Definition: The reality-virtuality continuum is a model that arranges all mixed forms of real and computer-generated environments on a continuous scale. At one end is the purely physical environment, at the other end a fully virtual world. Between them are augmented reality and augmented virtuality, which together make up the area of mixed reality.
In B2B mechanical engineering, the model helps with decisions: it shows how much reality and how much virtuality an application for sales, trade shows, service, training or design review actually needs.
Advantage: A common language for engineering, sales and management, instead of discussions about buzzwords.
English: Reality-Virtuality Continuum, Virtuality Continuum
German: Realitäts-Virtualitäts-Kontinuum, Virtualitätskontinuum
How It Works
Origin of the model
Paul Milgram and Fumio Kishino introduced the continuum in 1994 in the article "A Taxonomy of Mixed Reality Visual Displays" (IEICE Transactions on Information and Systems). In the same year, Milgram, Takemura, Utsumi and Kishino published a conference paper that describes augmented reality as a class of displays on this continuum. The goal was to bring order to display systems, which at that time had many inconsistent names.
The four zones
- Real environment: Only physical objects, no digital content.
- Augmented reality (AR): The real environment dominates, digital objects are overlaid.
- Augmented virtuality (AV): The virtual environment dominates, real elements such as live video or sensor data are integrated.
- Virtual environment: The entire scene is computer-generated.
Milgram and Kishino group everything between the two end points as mixed reality. The boundaries are fluid: what matters is which part dominates.
Three dimensions behind the scale
The model is more than a line. The authors describe three axes:
- Extent of World Knowledge: How much does the system know about the real environment (geometry, position, objects)?
- Reproduction Fidelity: How realistically does the system show virtual content?
- Extent of Presence Metaphor: How strongly does the user feel present in the displayed world?
In practice, you can translate the axes into specific requirements. AR assembly instructions need much knowledge about the real machine, so precise tracking, but not necessarily a photorealistic display. A VR tour of a plant, on the other hand, needs almost no knowledge about the real room, but high display quality and a strong feeling of presence.
Criticism and further development
Skarbez, Smith and Whitton revised the model in 2021 in "Frontiers in Virtual Reality". Their thesis: full virtuality is not possible, because the user's body always stays real. So today's VR is basically also mixed reality. They also put the user more at the center, for example through immersion and presence.
Where Web 3D, AR, MR and VR Are
| Format | Position on the continuum | Typical device |
|---|---|---|
| Web 3D viewer | Virtual object in a screen window, almost no relation to the environment | Desktop, tablet, kiosk |
| Mobile AR | Close to reality, few virtual objects | Smartphone, tablet |
| Mixed reality | Middle, virtual objects react to the captured environment | MR headset |
| Augmented virtuality | Close to virtuality, with real data or video | Headset, large screen |
| Virtual reality | Close to virtuality, environment hidden | VR headset |
A WebGL viewer is a special case: the content is purely virtual, but it appears in a small window and the real environment stays visible. The umbrella term for AR, MR and VR is extended reality (XR).
B2B Applications
Sales
In the first meeting, a 3D model in the browser is often sufficient. If the customer wants to check whether a packaging line fits into their hall, AR on site is useful: the real hall stays the reference point, and the machine is overlaid true to scale.
Trade shows
Large plants such as conveyor systems or press lines rarely fit on a booth. A kiosk with a 3D viewer reaches many visitors at the same time. A VR headset gives individual prospects a walk-through tour of the plant.
Service
For maintenance on the real machine, the real part is decisive. AR or MR instructions overlay work steps directly on the component. A fully virtual approach would lose the relation to the actual machine here.
Training
Operators can practice dangerous or rare procedures without risk in VR, for example a tool change on a CNC milling machine or malfunctions. For training on the real plant, AR is more suitable.
Design review
Design teams check assemblies in VR at a 1:1 scale or as a mixed reality hologram on the meeting table. When you connect a digital twin to live data, the result is augmented virtuality.
Choosing the Right Point on the Continuum
More virtuality is not automatically better. Three questions help you choose:
- Is the real environment part of the task? Yes: AR or MR. No: web 3D or VR.
- How many people do you want to reach? Many: browser and mobile. Few, but intensively: headsets.
- Which hardware does the user have? Browsers and smartphones are available everywhere, headsets usually are not.
Example: A manufacturer of filling lines first shows new customers a 3D model in the browser. In the quotation phase, the customer uses tablet AR to check whether the line fits into their hall. For the training of the operators before delivery, the manufacturer uses VR, and for maintenance during operation again AR on the real machine. So one project goes through several points of the continuum.
In practice, a common 3D model works well as the basis, which you deliver at different points of the continuum, depending on the situation.
Frequently Asked Questions (FAQ)
Is the continuum a fixed division into four categories?
No. The four zones are reference points on a continuous scale. Many applications are between two zones or change their position during use.
Where exactly is mixed reality?
In the original model, mixed reality includes everything between the two end points, so AR and AV. In today's usage, MR usually means applications in which virtual objects capture the real environment and react to it.
Is the model from 1994 still relevant today?
Yes. Research and industry still use it as a reference. The 2021 revision adds the role of the user, but it does not question the basic idea.
Can one application cover several points of the continuum?
Yes. Modern headsets with passthrough cameras switch between an AR-like display and full VR. A web application can also offer a 3D viewer and an AR mode.
What role does the continuum play in budget planning?
It makes requirements comparable. The more environment knowledge and display quality an application needs, the higher the effort for tracking, content and hardware usually is.
Industry Standards and Technologies
- Milgram & Kishino: A Taxonomy of Mixed Reality Visual Displays (IEICE, 1994) – original paper
- Skarbez, Smith & Whitton: Revisiting Milgram and Kishino’s Reality-Virtuality Continuum (2021) – current revision
- ISO/IEC 18039:2019 – reference model for mixed and augmented reality
- OpenXR – open interface for AR and VR devices
- WebXR Device API – AR and VR in the browser
- glTF 2.0 – exchange format for 3D content
Related Topics
XR basics: - Augmented reality – digital content in the real world - Mixed reality – merging of reality and virtuality - Virtual reality – fully virtual worlds - Extended reality (XR) – umbrella term - Spatial computing – spatial data processing - Augmented virtuality – virtual world with real elements
Perception: - Immersion – technical immersion - Immersive experience – effect on the user - Presence – feeling of being present
Technologies: - WebGL – 3D in the browser - SLAM – environment capture - Markerless tracking – tracking without markers - Real-time rendering – real-time display - HoloLens – MR headset
B2B applications: - Digital twin – digital replica of the machine - 3D product configurators – interactive configuration - Virtual prototype – testing before construction
