Augmented Reality Smart Glasses

Introduction

Remember when everyone said that augmented-reality glasses had failed horribly and died?

They didn’t.

They were simply reborn into a sector where privacy concerns and geek-averse fashion trauma do not matter so much.

Smart glasses have now taken on a new life in digital factories, supporting an initiative known as “Industry 4.0”.

What Is Industry 4.0?

Sometimes called the fourth industrial revolution, Industry 4.0 was coined in 2011 by the German government and aims to improve productivity in digital manufacturing. 

This involves consolidating the Internet-of-Things (IoT), Artificial Intelligence (AI) and Big Data directly onto the factory floor.

As Peggy Gulick, Director of Business Process Improvement at AGCO, a US-based agricultural machinery manufacturer, says: 

“Augmented-Reality Glasses don’t replace people; they allow people to be smarter, faster and better at their jobs”.

Also known as smart spectacles, smart eye glasses and AR glasses, there are many manufacturers available. However, rather than reviewing specific products, we are going to dive into the technology, its benefits and its applications.

The secret sauce of all smart glasses is the merging of virtual and physical worlds by a hardware / software technology called ‘Augmented Reality’ (AR).

What is Augmented Reality?

Augmented Reality is the superposition of digital information onto the viewable natural world, and requires the following core elements to function:-

• A digital camera, normally embedded into a device, such as a smartphone or head-mounted smart spectacles,
• AR software, which uses computer vision to interpret the real world and superimpose the digital model onto the user’s field of vision,
• Cloud-connectivity to a ‘digital twin’ (a 3-dimensional model) of the object being viewed in the real world, and
• Sensors, located on the real world object, which inform the digital twin what state it is in and what operations can be performed on it. This can furthermore be combined with information from the factory-level.

How This Works

The following diagram breaks down how it works:

1. The AR device (e.g. smartphone) uses computer vision to identify the real world object (e.g. an industrial robot). This can be achieved through snazzy machine learning that recognises the object, or some sort of marker (for example, a QR code).
2. The AR device then connects to the cloud-based digital twin of the real world object.
3. The real world object also connects to the digital twin, streaming its current state of play, and showing all the operations you can perform on it.
4. The data from the digital twin is sent to the AR device where the AR software merges the virtual model onto the real world object. You might visualise this as an automotive-style dashboard floating in your field of view above the robot and indicating its operational status, for example.
5. The user can now send commands to the digital twin via the AR device, asking the robot to perform a function, using either a virtual user interface provided by the software, or via voice commands.
6. Finally, these commands are sent from the digital twin to the real world object to be executed, and any resulting status messages are fed back to the digital twin.

What has this got to do with Smart Glasses?

Smart glasses bring the AR hardware closer to your eyes, so you can simultaneously view the real world object and the superimposed augmented elements.

Oh, and I forgot to mention: 

They allow hands-free operation, which is a Really Big Deal for industrial operators.

We could also achieve the same concept in very different implementations:

• a camera embedded into an automotive windscreen, allowing us to see information about speed, traffic delays and fuel consumption.
• a camera embedded into an aircraft cabin window to view geographical information superimposed on the terrain below, or 
• a camera embedded into a building facade, where AI estimates security risks for banks or jewellers through facial recognition.

However, none of these alternatives allow users to view the physical and virtual worlds simultaneously in a mobile, hands-free way. Consequently, smart glasses have become the preferred vehicle for AR.

Here is a non-exhaustive list of augmented-reality smart glasses on the market currently, listed in alphabetical order:

• Aleger Smart Glasses
• Augumenta Smart Glasses
• Microsoft Hololens 2
• RayBan Smart Glasses
• RealWear HMT-1 Smart Glasses
• Vuzix Smart Glasses

AR Smart Glasses – Survey

Rather than looking at the marketing hype coming from manufacturers, let’s look at a real-world survey done by the University of Skövde in Sweden into real operators using real AR devices in real manufacturing environments.

The study highlights the current challenges and future improvements that smart glasses will need to deliver to see greater market penetration. 

Importantly, this is done from the perspective of industrial operators and includes factors such as the importance of assembly instructions (i.e. the user interface), ergonomics (critical for safety), as well as design, support, and training.

The main challenges outlined in the survey point to:-

• out-dated assembly instructions with a lack of feedback on errors made during the assembly, which should ideally contain more multimedia and less text
• a limited field of view offered by the smart glasses
• how seamlessly the augmented reality is integrated into the real world
• product testing based on overly-simplified tasks executed by people who are not experienced industrial operators
• insufficient focus on the weight of the AR smart glasses and the weight distribution relative to the orientation of the user’s head, often resulting in fatigue from extended use, and
• limited battery life of the units themselves.

There are several ways to implement AR in mobile devices:

Projection-based AR

This involves a projector worn on the user’s head and uses retroreflective materials strategically placed in the environment to ease recognition.

Eye-multiplexed AR

This is where virtual elements are synchronised with the physical environment but not superimposed onto the real world.

Video see-through AR 

This technique combines virtual content with a real-time video stream. This method suffers from increased motion-sickness though and poor internet bandwidth can result in choppy video feeds.

Retinal Projection

In which thin parallel light beams are focused into the user’s eyes.

Optical see-through AR

Creates virtual elements in the user’s field of view by directing light through half mirrors or prisms. This is the most common commercially-available solution right now.

Case Study: TÜV-SÜD Global Risk Consultants

One organisation actively using AR smart glasses is the company TÜV-SÜD, headquartered in Germany, who provide Fire Protection Engineering, Boiler & Machinery Engineering and Natural Hazards Analysis as a remote service.

This enables remote participation with the client during site visits as well as real-time video calls and in-situ photographic documentation for risk assessment of industrial environments.

In this case, TÜV-SÜD use the Realwear HMT-1 for standard industrial environments and the HMT-1Z1 model when explosion protection is needed. Since the latter model has ATEX Zone 1 certification, this allows its use in wet, dusty, hot, dangerous and loud industrial environments.

Wrapping it up

To conclude, augmented-reality smart glasses have seen limited market penetration, despite the financial clout of manufacturers such as Microsoft and Google, but the technology does present very clear benefits that need further exploration and real-world validation.

The survey pointed out in this article certainly pushes the industry to compare real results for real users to identify real areas for improvement, but the work of integrating AR smart glasses into complex manufacturing environments has only just begun.

At Smart Glass World, we look forward to offering our readers further insights into the state of the market in years to come as this technology matures and receives the market penetration that it so richly deserves.

References

1. How Does Augmented Reality Work?, Harvard Business Review

2. The Battle of the Smart Glasses, Harvard Business Review