XR Technologies – Bold steps to new realities
The simulation of reality has developed vastly in the last decade, and the pace of the development is increasing. Virtual and augmented reality have become buzzwords and artificial intelligence (AI) is shaping up to change mankind for better or worse. Applications can be found in industry as well as in the gaming world. Where is simulation going and how will it shape our world?
Faster, better, and simpler are words that drive industry. Instant gratification and consumption are the order of the day. The more one knows about the world, the more one becomes aware of experiences that are missing. But what is an experience? What makes a trip more real than a simulation of the same trip? What is missing?
The birth of different realities
X Reality (cross reality / XR) encompasses multiple forms of artificial realities such as augmented reality, virtual reality and mixed reality.
Augmented reality, as the name implies, seeks to add artificial entities such as data, text, and objects to a physical environment. The entities are visible through a camera that recognises physical trackers and places the entities according to the location of these trackers.
Virtual reality is an artificial environment created with the intention to simulate self-presence in an environment with the use of a head-mounted display (HMD) that tracks head movements and position thanks to cameras or lighthouse boxes.
Mixed reality is a combination of both technologies. In mixed reality users experience a virtual environment, yet some of its components are anchored to real-life physical objects.
A collective interest
Communication is incredibly important for all living beings; visual inputs, sounds, smells, tastes and sensations are ways of gathering information from the environment, thereby increasing human chances of adapting and surviving. There is, naturally, no longer the need to survive in the wild. Humans are, nevertheless, highly dependent on information. It is not always important how the information is obtained, but rather how easily it is understood.
Information has become much more readily available to everyone – from gossip to messengers, newspapers, news channels, and the Internet. Basic knowledge is no longer a luxury and even academic diplomas are no longer an advantage in many fields in the same way that experience is.
Simulators have long been used as tools to gauge performance. The line between simulation and reality is shrinking quickly as new tools and software are developed. If simulations are combined with AI, completely different scenarios to test users can be developed. This allows learning in a more involved manner than with a textbook.
Room for both the old and the new
It always takes some time for new technologies to be adopted. The number of detractors usually peaks when the technology is new and untested; many fear the consequences and are blinded from the benefits. History has shown, however, that it is always a matter of time: vehicles did not make human legs redundant, emails are extremely convenient, but people can still write letters if they wish. Cell phones replaced many technologies, but specialised tools are still sold.
Standards are difficult to predict, before the Internet was invented, it was difficult to imagine it. VR and AI together will not replace the cinema or drivers and it will take time to reach conclusive opinions about it. Perhaps it is the fear of no longer needing human imagination or of losing more control to technology that makes some people reluctant to adapt.
Some doubt the viability of VR movies, claiming that it is impossible to keep an audience focused with such freedom. The theory is that they will likely miss important details of the story if they are easily distracted. However, these kinds of movies are possibly not meant to be seen once, or in a room with many people.
More than a game
There is still a stigma attached to games and the people who enjoy them. The biggest concerns are the inability to separate fiction and reality and the normalisation or celebration of actions that are not deemed acceptable in society.
The debate will probably rage for several more years, but attitudes are changing gradually. Games are not only the means to live a fantasy, but a means to engage with others, to test skills or develop them.
A game is a structured form of play: rules and boundaries are set, and an objective is explained. A well-crafted video game easily relays these rules and objectives to the user: how high one can jump, what can be interacted with, what rules end the game, and what is the objective.
Similarly, a well-crafted application that monitors health not only keeps track of numbers but can also challenge the user to reach an objective. It can provide an environment with rules to follow as well as visually appealing functionalities to help the user focus on the objective.
When considering realities, it is important to talk about immersion. After all, if one wants to build a machine, many factors such as the scale, parts, and procedure are important, but moving a machine is not always a cheap option, neither is having it maintained on site.
Nowadays, a virtual copy or digital twin of a machine can be created with the use of surrounding data to predict behaviours, or at the very least to gain a realistic and updated status of the machine.
A digital twin can bring many benefits to many industries; it is not always necessary to add artificial components to a machine as the data obtained is valuable itself. However, there is a branch that takes digital twins further: artificial Intelligence (AI), or more precisely neural networking, which can assist in troubleshooting machine behaviour by understanding patterns and suggesting solutions.
The term artificial intelligence was coined back in 1950, but the idea of artificial beings developing intelligence has been around for centuries. But even with the immeasurable advantage machines have over humans, in some fields the biggest disadvantage they have is the lack of pattern recognition.
Neural networking algorithms seek to emulate human brain-pattern recognition based on the comparison of data. How does a human recognise 1 as 1? What about if it was written as I? Effective algorithms do not provide a single answer to the above-mentioned questions, but a set of answers with a conclusion.
Everyone is an architect
Game engines have come a long way. Not only are there multiple options to create applications, but there is a vast number of sources to learn and improve the game skillset as well as tools to ease development.
Anyone can engage in creating their own environments or team up to tackle larger projects. It takes a few hours to grasp the basics, and minutes to publish even an empty space with a cube on multiple platforms, from a cell phone to a PC as standalone or virtual reality enabled.
The biggest players right now are Unity3D and Unreal Engine. They are constantly pushing their respective engines to appeal to different kinds of developers, including VR-focused ones.
As technology keeps evolving, it may no longer require hours to generate content. Taking a trip to an exotic island with fantastical creatures may be as simple as ticking a few parameters. In such a world, education will be both academic and empirical without costs and risks. In short, it will be a world where simulations can provide all the answers.
Will perfect simulation be possible in the future? It is currently possible to simulate images, sounds, smells, and tastes. Haptic feedback is slowly advancing, but why would one want to feel pain from a virtual source?
All available senses are important to humans and the joint perception makes experiences “real”. The fear of pain from haptic technology is understandable, but not much different to the fear of hearing a deafening or screeching sound or that of food that is met with immediate rejection. It is natural to try to avoid these sensations, but musicians and chefs need to be exposed to both good and bad music and food.
AI that learns from human feedback can provide unexpected, yet enjoyable or challenging scenarios that allows all senses to take part in experiences.
This article was first published in Elomatic’s Top Engineer magazine 2/2019