Combining Deep Learning Algorithms with Augmented Reality

What is Deep Learning?

Deep learning is a sub-discipline of artificial intelligence (AI) that focuses on automatically learning relevant features from complex unstructured data. Deep neural networks are mathematical models designed to mimic the functioning of the human brain. These neural networks, characterized by their layered architecture, allow for progressive processing of the received information. Using layers of information processing, they learn to recognize patterns and perform complex tasks. These tasks include image recognition, machine translation, speech recognition, and many others.

Deep learning grew popular in recent years because of its ability to process large amounts of data with high accuracy. The exponential growth in the amount of data available and the increase in computing power offered by GPUs (Graphics Processing Units) have made it possible for users to solve many complex problems and democratize the concept of deep learning.

Deep learning has capabilities used in many applications. This includes image and video recognition, speech recognition, machine translation, product and content recommendation, fraud detection, and many other areas.

What is the Purpose of Deep Learning?

The main objective of deep learning is to enable machines to process and analyze complex data autonomously. Indeed, data generated by humans and machines are often very large, unstructured, and difficult to analyze with traditional methods. It overcomes these difficulties by automatically learning relevant features from this data. That in turn allows machines to manually recognize patterns and characteristics that would be difficult to identify.

Deep learning also improves the accuracy of predictions and decisions using models trained on large amounts of data. The goal is to allow machines to interpret data based on what they have learned during the learning phase.

What is Machine Learning (ML)?

In general, machine learning refers to the ability of machines to learn from selected features extracted from that data manually by an operator using algorithms that detect patterns in the data and then use those patterns to make predictions. Machine learning can be supervised, unsupervised, or semi-supervised depending on how the data is labeled and used to train the algorithms.

The Difference Between Machine Learning and Deep Learning

On the other hand, deep learning is a dimension of machine learning that uses deep neural networks, often composed of multiple layers, to extract complex features from unstructured data, such as images, audio, or text. Deep neural networks can easily recognize diverse information and learn to represent data more abstractly than traditional methods.

The difference between machine learning and deep learning is that in machine learning, the developer defines the features to be used for analysis. In contrast, in deep learning, the network behaves like a feature extractor and makes a decision based on what it has extracted as information.

Deep Learning and Augmented Reality

When using augmented reality (AR), Deep learning techniques are beneficial because they allow machines to recognize and track objects in real time, which is essential for creating smooth AR experiences. Deep neural networks are trained on large amounts of visual data to recognize objects and scenes in the real world. This allows the machine to adapt to its environment.

Machines can then recognize, understand and adapt to the environment in which they are used, track objects in real time, and react quickly and accurately to user movements. These improvements in AR have paved the way for new, more immersive, realistic augmented reality experiences in many fields, including industry.

Deep Learning at DELMIA

Today, tracking initialization is a manual process using standard relocation or markers. In most cases, this works well, but sometimes it is not robust enough, and initialization can be difficult, especially when the work environment is subject to a combination of varying lighting conditions or changes in background or product appearance. In such cases, the learning process for standard initialization can be more difficult, time-consuming, and requires special skills.

With its deep learning capabilities, DELMIA makes the tracking initialization process for AR applications robust to variations in the working environment or product characteristics, avoiding repetitive manual adjustments each time changes occur.

The benefits of deep learning:

• Works day and night, resisting seasonal and punctual lighting variations
• Reuses the tracking model across multiple workspaces
• Adapts to multiple production lines
• Deploys or duplicates use cases on other sites easily
• Provides a more robust initialization than standard initialization on complex objects (e.g. several meters long)

DELMIA Deep Learning for Inspections

Today, deep learning is used in inspection either to make a decision on the status of the object (presence or absence of defect) or to feed another algorithm. That algorithm will use this first analysis and then be able to make a decision.

The most common models used in production are classification models. Classification models determine whether an object is present or absent, inverted with another known object, or if a known characteristic defect is present on the object (sufficiently visible). DELMIA Augmented Experience also uses object detection to identify objects not present in the 3D model provided by a customer. We also use object detection to verify an order between several objects. The inspection engine also uses semantic segmentation. Its role is to calculate a mask of the object of interest (a mask is a black image where only the pixels belonging to the object of interest will be). From this mask verify that an object has the expected shape and is close to the desired length.  

Finally, our DELMIA Augmented Experience Quality Inspection solution is equipped with OCR (Optical Character Recognition) capabilities. These verify that an expected text is in the right place and will verify serial numbers or read labels.

Contact our DELMIA Experts to learn more about these immersive experiences.