Definition: Motion tracking is a set of methods that continuously measure the position and orientation of objects or body parts in space and send them to software. In augmented reality, virtual reality and mixed reality, motion tracking makes sure that the virtual scene follows each movement of the head, hands and device.

In B2B mechanical engineering, motion tracking is the basis for: stable placement of 3D models in AR, free movement around a machine in VR, operation with a controller or the hands, training with realistic manual tasks, and design reviews at a 1:1 scale.

Advantage: Users move naturally, and the virtual machine stays where it belongs.

English: Motion Tracking, Positional Tracking
German: Bewegungsverfolgung, Bewegungserfassung

How It Works

Sensors and cameras combined

Modern XR devices combine two measurement principles:

  1. Inertial sensors (IMU): A gyroscope and an accelerometer measure rotation and acceleration very quickly, but they drift over time.
  2. Optical detection: Cameras or infrared sensors find the absolute position in space, but at a lower rate.
  3. Sensor fusion: Software combines the two sources. The IMU supplies fast intermediate values, and the camera corrects the drift.
  4. Prediction: The system predicts the head position at the time the image appears, to compensate for the delay.

Latency as a quality characteristic

The time between a movement and the displayed image is called motion-to-photon latency. As a guideline for comfortable XR experiences, it should be less than approx. 20 milliseconds. Higher values make many users feel unwell, because the sense of balance and the image do not agree.

So the sensors used and their calibration directly decide how long a training session or presentation stays comfortable.

3DoF and 6DoF

Degrees of freedom

A body in space has six degrees of freedom: three rotations and three translations.

Characteristic 3DoF 6DoF
Records Rotation (pitch, yaw, roll) Rotation and position (forward/backward, left/right, up/down)
Typical sensors IMU only IMU plus cameras or external stations
The user can Look around Look around, walk around, bend down
Suitable for 360-degree videos, simple presentations Machine walkthrough, training, design review

In practice in mechanical engineering: If you want to walk around a system, look under a conveyor belt or stand at a control panel, you need 6DoF. Current standalone headsets and AR-capable smartphones and tablets (through ARKit and ARCore) work with 6DoF.

Inside-Out and Outside-In

Inside-out tracking

Cameras on the headset or mobile device observe the environment. Methods such as SLAM detect distinct points in the room and calculate the device's own position from them. Some devices also use depth sensors such as LiDAR.

Advantages: No setup of external hardware, mobile use (trade show, customer hall, workshop). Limits: Difficult in very dark rooms or rooms without structure. Controllers outside the camera's field of view are only estimated for a short time.

Outside-in tracking

External stations record the position of the device. In base station systems such as SteamVR Tracking, permanently installed stations send infrared signals that sensors on the headset and the controllers evaluate. Camera-based motion capture systems, on the other hand, observe reflective markers on the object.

Advantages: Very precise and stable, also with fast movements. Limits: Setup and calibration are necessary, and the use is limited to the prepared area.

Optical references

You can also anchor tracking to real objects: marker tracking uses printed codes or images, markerless tracking detects surfaces and objects without aids.

Controllers and Hand Tracking

Controllers

Controllers are tracked with their own sensors and infrared LEDs or with base stations. Buttons and triggers give clear inputs, and vibration gives haptic feedback.

Strength: Precise and reliable, ideal for long training sessions and the exact selection of small components.

Hand tracking

The cameras of the headset detect the hands and calculate a skeleton model of the joints. The OpenXR extension for hand tracking defines 26 joint points for each hand, the WebXR Hand Input Module for the browser defines 25. Gestures such as grabbing, pointing or a tap of thumb and index finger become inputs.

Strength: Nothing in the hand, easy start without explanation, good for trade show visitors and occasional users. Limit: Hands outside the camera range or hidden fingers decrease the accuracy.

B2B Applications

Sales: the machine at a 1:1 scale

The sales representative uses a tablet to place a packaging machine in the customer's hall. 6DoF tracking keeps the model stable in its position while both walk around the machine and check the distances to walls and paths.

Trade show: start without instruction

Visitors put on a headset and reach for virtual assemblies with their hands. With hand tracking, you do not have to explain the controllers. Inside-out devices do not need a complex setup on the booth.

Service: instructions on the real machine

A technician sees maintenance steps in AR directly on a CNC milling machine. Precise tracking makes sure that the markings stay on the correct component, also when the technician moves. Hand tracking permits operation without putting down the tool.

Training: practice manual tasks

In VR, employees practice the changeover of a machine. Controllers give precise grabbing and haptic feedback, and hand tracking practices movement sequences that are closer to reality. The software can evaluate the order and the duration of the steps.

Design review: check ergonomics

Engineers walk through a planned system in VR. With 6DoF, they check reach zones, lines of sight and maintenance access at real body height. For high accuracy, outside-in tracking in a fixed review room is also worth it here.

Challenges

Environment

Inside-out tracking needs sufficient light and visible structures. Shiny machine covers, mirrors, large surfaces in one color or very dark halls can interfere with the tracking. At trade shows, there are also many people in the camera image and changing lighting.

Moving reference surfaces

When a user stands on a moving platform or in a vehicle, the inertial sensors and the camera image contradict each other. For such cases, some devices have special modes.

Several devices in the same room

When several people must see the same virtual machine in the same location, their coordinate systems must be aligned, for example with shared spatial anchors or markers.

Best practice: Test the location in advance, check the lighting and have a fallback ready for critical demos, for example the same content in the web viewer.

Frequently Asked Questions (FAQ)

Is 3DoF sufficient for machine presentations?

Yes, for 360-degree videos or a model that you look at only from one fixed position. As soon as users move to see details from close up, 6DoF is necessary. Without position tracking, the image does not move with the body, and this quickly becomes uncomfortable.

Inside-out or outside-in: which is better for companies?

For mobile use at customer sites, at trade shows and in service, inside-out is more practical, because you do not have to set up anything. Outside-in is worth it for fixed rooms with high precision requirements, for example design review labs or ergonomics studies.

Does hand tracking replace the controllers?

Not completely. Hand tracking is suitable for navigation, simple selection and short demos. For precise inputs, long sessions and haptic feedback, controllers keep the advantage. Many applications support both.

How accurate is motion tracking in AR?

This depends on the device, the light and the environment. For presentation and visualization, the accuracy of current smartphones and headsets is usually sufficient. For overlays with exact dimensions on real components, additional references such as markers or object detection are used.

Does tracking record data about users?

Yes, cameras record the room, and hand and head data describe movements. The device usually processes these data locally. When movement data is stored or evaluated, for example in training, the rules for data privacy apply, including informing the employees.

Industry Standards and Technologies

Tracking methods: - SLAM – Simultaneous localization and mapping - Marker tracking – Tracking with image markers - Markerless tracking – Tracking without markers - Eye tracking – Tracking of the gaze - Hand tracking – Detection of the hands - Inside-out tracking – Cameras on the device - Outside-in tracking – External stations

Hardware and sensors: - Sensors – Measurement technology - LiDAR – Depth measurement with a laser - IMU – Inertial measurement unit - HoloLens – AR headset - visionOS – Apple's platform for spatial computing

Platforms: - ARKit – AR on Apple devices - ARCore – AR on Android - WebGL – 3D in the browser

Experience: - Augmented reality – Reality with added digital content - Virtual reality – Fully virtual environment - Mixed reality – Combination of real and virtual - Extended reality (XR) – General term - Immersion – Feeling of being inside the scene - Motion-to-photon latency – Delay between movement and image