Spatial Computing – A Virtual World Springboard

A digital twin is the complete, holistic representation of a product, a machine or even an entire factory floor in a digital, machine-readable format. The detailed 3D model is the basis to which all further information is attached, from metadata on material, color or production steps to electronics, which can be simulated just like the software that controls the product and makes it smart.

A machine that is digitally defined can be put into operation and operated in a virtual space, with simulation algorithms ensuring that all components act in a physically correct manner. Parcels on a sorting conveyor belt are moved by conveyor belts, distributed via gates and drop to the ground if something is wrong – physically correct.

The term ‘spatial computing’ describes a form of IT use that interacts with the environment and allows users to immerse themselves in the virtual environment to a very great extent. Instead of working in front of the screen, you enter the virtual world and interact with the digital content as naturally as possible. Not with a keyboard, mouse or VR input device, but with natural hand movements and gestures. This means that, unlike in VR, the body and hands are part of the virtual space.

In addition, modern spatial computing technologies make it possible to include the environment in the ‘experience’ and to interact with it. The boundary between the real and virtual environment becomes blurred, with the software seamlessly projecting virtual content into the real environment in three dimensions.

Apple’s glasses are, in a way, a hybrid of VR and AR: on the one hand, the glasses are completely closed like VR glasses. On the other hand, the real environment is recorded by cameras and displayed in real-time on the displays inside – similar to AR glasses. The glasses are therefore ‘virtually transparent.’ There is even an outward-facing display built into the glasses that makes the wearer’s eye area, which is covered by the glasses and recorded by cameras inside the glasses, visible to the outside world.

Spatial computing now makes it possible to immerse yourself in the virtual world and interact with the digital twin. A digital control panel actually controls the machine, and pressing a button or turning a potentiometer triggers reactions in the machine. The virtual machine can be located in a real environment; for example, a new system can be ‘inserted’ into the existing factory building. The user can then move around the building and assess whether all the work areas are free or whether the machine would be better located half a meter further to the left or right.

In other applications, the focus is less on the real environment and more on the natural, manual interaction with the virtual system, e.g. in training. In trade shows, manufacturers of machining centers present virtual machines that apprentices can practice on – without damaging a valuable real machine in the event of a crash or when an end switch is overrun. If a cycle time is to be increased, this can be tested on the virtual twin before waste is produced in real production. Optimizations to the control software can be tested safely on the virtual machine, as can mechanical changes or retooling for a different part to be manufactured. And all this is done without the need for ‘crutches’ such as a mouse or VR controller, but quite naturally by hand.

Last but not least, maintenance work on the virtual machine can be planned: Which parts have to be removed to access a defective component, is a crane needed or is a forklift sufficient? Apple Vision Pro is very useful for this because it allows several people to work on the virtual model and see each other in the form of avatars. Can the two people needed to screw on an assembly fit into the confined space of a machine? Questions like this, which involve people, can be tested and answered using such hardware.

Spatial computing is the key to operating the digital twin and is therefore not hype, but the digital tool of the future, a natural tool in the digital world.