Tag: augmentedreality

Mixed Reality – Hand inputs for mesh manipulation #2

Recognition of the pinch gesture for a more interactive and intuitive mesh manipulation. The pinch gesture confirms the start of the deformation, and the vertices will respond by how to further/close is the user’s hand to the mesh.

The push/pull deformation is realised by measuring, at each frame, the distance of the hand to the mesh and The push/pull deformation is realised by measuring, at each frame, the distance of the hand to the mesh and comparing it to the previous frame. This operation is achieved by using Unity RaycastHit functions.

if (hit.distance < distance)
        {
            VetexDisplacementByDistance("push");

        }
        else if (hit.distance > distance)
        {
            VetexDisplacementByDistance("pull");

        }

....

public void VetexDisplacementByDistance(string command)
    {
        ...

        foreach (var item in vertexDistancesDict.OrderBy(pair => pair.Value).Take(vertexNeighbours))
        {
            HandJointUtils.TryGetJointPose(TrackedHandJoint.Palm, Handedness.Any, out MixedRealityPose palmPose);

            if (command == "push")
                displacementDirection = ray.direction.normalized * -1;
            else if (command == "pull")
                displacementDirection = ray.direction.normalized;
        }
    }