The Future of Artificial Intelligence (AI) – A 5 Year Roadmap

Implementation of AI amplifies the AR experience by using deep neural networks to replace traditional computer vision approaches; new features such as object detection, text analysis, and scene labeling are also an addition. Some suitable examples are Apple Live text and Visual Look Up. These AI-powered AR tools help a user copy and paste text, make a phone call, translate text, send emails, and run a search online. In the past, AR software used traditional computer vision techniques and algorithms to compare visual features between camera frames to map and track the environment. However, modern AR applications rely on deep learning to provide a more advanced and wholesome experience. Computer-generated objects coexist in a single virtual scene with the real world. It has been made possible by integrating multiple sensors such as camera(s), gyroscopes, accelerometers, GPS, etc. Thus to form a digital representation of the world that can be overlaid on the physical one. A 3D picture of the world must be constructed to virtually allow digital objects to exist alongside physical ones. The most common method among developers to combine AR and AI models is to take images or audio from a scene, run that data through a model, and use the model output to trigger effects within the scene. Here are a few examples:

• Image or scene labeling: Object labeling makes use of classification models in machine learning. It is done by running a camera frame through a model to match its image with an existing label and convert the information collected in an AR format. One such example is Volkswagen’s Mobile Augmented Reality Technical Assistance (MARTA), which labels vehicle parts and provides information about existing problems and instructions on how to fix them.

• Object detection: Object detection and recognition utilize Convolutional Neural Network (CNN) algorithms to estimate the position and extent of objects within a scene. A camera frame is passed to an AI model that estimates the position and distance of things within a scene. Location information is then used to form hitboxes and colliders that facilitate physical and digital objects. For example, IKEA places ARKit application scans surrounding environment, measures vertical and horizontal planes, estimates depth, and then suggests products that fit the space.

• Semantic segmentation and occlusion: While ARKit may provide generic people occlusion capabilities, a custom AI model can segment and occlude cars or other objects.

• Pose estimation: An AI model infers the position of objects like hands and fingers, which are used to control AR content. Yopuppet.com is one example.

• Text recognition and translation: An AI model detects, reads, and translates text in an image. Augmented Reality APIs are then used to overlay translated text back into the 3D world. Text recognition and translation combine AI Optical Character Recognition (OCR) techniques with a text-to-text translation engine such as DeepL. A visual tracker keeps track of the word and allows the translation to overlay the AR environment. Google Translate currently offers this functionality.

• Audio recognition: AI models listen to specific words that trigger AR effects. Automatic Speech Recognition (ASR) uses neural network audiovisual speech recognition, an algorithm that relies on image processing to extract text. Specific words trigger an image in the library labeled to fit the word description, and the image is projected onto the AR space. For example, a user says “Queen,” and a virtual crown appears on their head. An example is the Panda sticker app.

Visual data, an accelerometer, and gyroscopes are used to build a world map that is tracked for movement. However, traditional computers can still do most of these tasks since vision techniques make no use of machine learning. It is a whole new ballgame with innovative capabilities to seamlessly match reality with fiction. Launching AR-enabled consumer-facing applications like Pokémon Go and Snapchat has evolved to massive levels. It will continue to break new grounds in the future of Artificial Intelligence. According to Greenlight Insights, Augmented Reality devices and the content will hit revenue of a whopping $36.4 billion in 2023.