Augmented-Reality Smart Glasses for Industry 4.0

Augmented-reality (AR) smart glass is a nascent technology with clear benefits for digital manufacturing that needs real-world validation in order to enable the efficiency gains required by Industry 4.0

Remember when everyone said that Google Glass had failed horribly and died?

It didn’t.

It was simply reborn into a sector where privacy concerns and geek-averse fashion trauma does not matter so much.

Smart glasses have now taken on a new life in digital factories, supporting an initiative known as “Industry 4.0”.

What Is Industry 4.0?

Sometimes called the fourth industrial revolution, Industry 4.0 was coined in 2011 by the German government and aims to improve productivity in digital manufacturing, by consolidating the Internet-of-Things (IoT), Artificial Intelligence (AI) and Big Data directly onto the factory floor.

As Peggy Gulick, Director of Business Process Improvement at AGCO, a US-based agricultural machinery manufacturer, says: 

“Google Glass is a tool that doesn’t replace people; it allows people to be smarter, faster and better at their jobs”.

Also known as smart spectacles, smart eye glasses and AR glasses, there are now of course many large manufacturers, but rather than reviewing specific products, we are going to deep-dive into the technology, its benefits and the applications for industry.

The secret sauce of all smart glasses is the merging of virtual and physical worlds by a hardware / software technology called ‘Augmented Reality’ (AR).

How many people though really know what AR is and how it works?

Let’s find out.

What is Augmented Reality?

Firstly – is it our reality which is being augmented, or the virtual world which is being augmented? 

OK, let’s not go there.

Augmented Reality is simply the superposition of digital information onto the viewable natural world, and requires the following core elements to function:-

  • A digital camera, normally embedded into a device, such as a smartphone or head-mounted smart spectacles,
  • AR software, which uses computer vision to interpret the real world and superimpose the digital model onto the user’s field of vision,
  • Cloud-connectivity to a ‘digital twin’ (a 3-dimensional model) of the object being viewed in the real world, and
  • Sensors, located on the real world object, which inform the digital twin what state it is in and what operations can be performed on it. This sensorial information can furthermore be combined with information from the factory-level or from the business itself.

The following diagram breaks down how it works:

Augmented Reality Flow Diagram

  1. The AR device (e.g. smartphone) uses computer vision to identify the real world object (e.g. an industrial robot). This can be achieved through snazzy machine learning that recognises the object, or some sort of marker (for example, a QR code).
  2. The AR device then connects to the cloud-based digital twin of the real world object.
  3. The real world object also connects to the digital twin, streaming its current state of play, and showing all the operations you can perform on it.
  4. The data from the digital twin is sent to the AR device where the AR software merges the virtual model onto the real world object. You might visualise this as an automotive-style dashboard floating in your field of view above the robot and indicating its operational status, for example.
  5. The user can now send commands to the digital twin via the AR device, asking the robot to perform a function, using either a virtual user interface provided by the software, or via voice commands.
  6. Finally, these commands are sent from the digital twin to the real world object to be executed, and any resulting status messages are fed back to the digital twin.

What has this got to do with Smart Glasses?

Good question. 

In fact, smart glasses are really only there to bring the AR hardware closer to your eyes, so you can simultaneously view the real world object and the superimposed augmented elements.

Oh, and I forgot to mention: 

They allow hands-free operation, which is a Really Big Deal for industrial operators.

We could also achieve the same concept in very different implementations:

  • a camera embedded into an automotive windscreen (like a heads-up display), allowing us to see information about speed, traffic delays and fuel consumption.
  • a camera embedded into an aircraft cabin window, to view geographical information superimposed on the terrain below, or 
  • a camera embedded into a building facade, where AI software could estimate security risks for banks or jewellers and inform staff of nearby known assailants through facial recognition.

Industry 4.0 robot AR smart glasses

None of these however offer the benefits of simultaneous viewing of physical and virtual worlds in a manner which accompanies the user and is hands-free, hence the popularity of smart glasses as a vehicle for AR.

We will purposefully stay away from reviewing the many competing products, since that information can get out of date very quickly, and there are far better resources doing that already.

Instead, here is a non-exhaustive list of augmented-reality smart glasses on the market currently, listed in alphabetical order:

  • Aleger Smart Glasses
  • Augumenta Smart Glasses
  • Google Glass
  • Microsoft Hololens 2
  • RayBan Smart Glasses
  • RealWear HMT-1 Smart Glasses
  • Vuzix Smart Glasses

AR Smart Glasses – Survey

Rather than looking at the marketing hype coming from manufacturers, let’s look at a real-world survey done by the University of Skövde in Sweden into real operators using real AR devices in real manufacturing environments.

The study highlights the current challenges and future improvements that smart glasses will need to deliver to see greater market penetration. 

Importantly, this is done from the perspective of industrial operators and includes factors such as the importance of assembly instructions (i.e. the user interface), ergonomics (critical for safety), as well as design, support, and training.

The main challenges outlined in the survey point to:-

  • out-dated assembly instructions with a lack of feedback on errors made during the assembly, which should ideally contain more multimedia and less text
  • a limited field of view offered by the smart glasses
  • how seamlessly the augmented reality is integrated into the real world
  • product testing based on overly-simplified tasks executed by people who are not experienced industrial operators
  • insufficient focus on the weight of the AR smart glasses and the weight distribution relative to the orientation of the user’s head, often resulting in fatigue from extended use, and
  • limited battery life of the units themselves.

There are several ways to implement AR in mobile devices:

Projection-based AR

This involves a projector worn on the user’s head and uses retroreflective materials strategically placed in the environment to ease recognition.

Eye-multiplexed AR

This is where virtual elements are synchronised with the physical environment but not superimposed onto the real world.

Video see-through AR 

This technique combines virtual content with a real-time video stream. This method suffers from increased motion-sickness though and poor internet bandwidth can result in choppy video feeds.

Retinal Projection

In which thin parallel light beams are focused into the user’s eyes.

Optical see-through AR

Creates virtual elements in the user’s field of view by directing light through half mirrors or prisms. This is the most common commercially-available solution right now.

Case Study: TÜV-SÜD Global Risk Consultants

One organisation actively using AR smart glasses is the company TÜV-SÜD, headquartered in Germany, who provide Fire Protection Engineering, Boiler & Machinery Engineering and Natural Hazards Analysis as a remote service.

This enables remote participation with the client during site visits as well as real-time video calls and in-situ photographic documentation for risk assessment of industrial environments.

In this case, TÜV-SÜD use the Realwear HMT-1 for standard industrial environments and the HMT-1Z1 model when explosion protection is needed. Since the latter model has ATEX Zone 1 certification, this allows its use in wet, dusty, hot, dangerous and loud industrial environments.

Wrapping it up

To conclude, augmented-reality smart glasses have seen limited market penetration, despite the financial clout of manufacturers such as Microsoft and Google, but the technology does present very clear benefits that need further exploration and real-world validation.

The survey pointed out in this article certainly pushes the industry to compare real results for real users to identify real areas for improvement, but the work of integrating AR smart glasses into complex manufacturing environments has only just begun.

At Smart Glass World, we look forward to offering our readers further insights into the state of the market in years to come as this technology matures and receives the market penetration that it so richly deserves.

References

1. How Does Augmented Reality Work?, Harvard Business Review

2. The Battle of the Smart Glasses, Harvard Business Review

Need vendor-neutral advice choosing smart glass for your next project?

From Our Blog