The technology promises to transform how we work, and it’s already having an impact in professions ranging from dentistry to aviation.
Sven Holtorf, like most dentists, appreciates that his patients don’t exactly look forward to visiting him. “Every time you’re going to the dentist, it’s fear, it’s uncomfortable, it’s pain, it’s not very much fun,” he says. But Holtorf recently invented something that he hopes will help, just a little bit, to ease the discomfort. And it’s not a special drill or a new kind of anaesthetic. It’s a set of glasses.
Holtorf’s device is the eyeCAD-connect system – a bit of augmented reality software developed by his firm, iDENT, that runs on a pair of smart glasses built by technology firm Epson. When Holtorf dons the glasses during a dental examination, he still sees his patient’s mouth – but superimposed over that, the glasses show real-time data coming in from his dental scanner. Once the scan is complete, Holtorf can send the data off to the lab, where technicians create a virtual model and design whatever crown or cap the patient needs.
In the past, when a dentist was scanning a patient’s teeth, they used the camera with one hand while they focused their attention on a nearby screen – much like during an ultrasound. But with the glasses, says Holtorf, the dentist can scan while looking directly at the patient, and the scans show up instantly on the augmented reality glasses. “So we avoid looking to the monitor while we’re scanning, and we don’t lose contact with the patient, and for me, that’s the idea,” he says.
From the dentist’s office to oil sand pipelines and fighter jet construction, augmented reality is creeping into workplaces all over the world. Used well, it can help workers do their job more safely and efficiently. But right now, manufacturers are facing a bit of a challenge in designing products that people will really use.
Gabe Batstone, the CEO of NGRAIN, a company that makes software for augmented reality devices, tells me some of his favourite experiences seeing the products in action. On a site in the oil sands in Canada, he was able to walk through the process of installing a clamp onto a pipe without any prior experience. He simply held his smartphone up to the stretch of metal and followed the onscreen instructions. On the phone’s screen, Batstone could see both the actual pipe he was looking at, captured using the phone’s camera, and an overlay of the steps by which he should install the clamp. “I walked through the procedure as someone who knows nothing about the clamp. A regular person can walk up to a piece of technology and feel like they can almost do that job.”
Lockheed Martin is another one of NGRAIN’s customers. Workers in Fort Worth, Texas, have to do a visual inspection on some of their planes (the F-35 and the F-22 in particular) to ensure that each jet’s stealth technology is still functioning. Once again Batstone said that he was able to walk down to the factory floor and run through the checklist without any prior experience. “I looked at it, and I was like ‘wow this is very straightforward.’”
But while these systems might make it easy for aeroplane engineer novices like Batstone to perform certain tasks, working out exactly what useful information to give professionals performing those same tasks is more difficult. The first thing companies have to figure out is exactly which tasks their workers perform that could benefit from the addition of augmented reality. “It’s not about replicating the real world on a screen, which I think is what many of us might think,” Batstone says. “It’s about contextualising the world and segmenting it.” Figuring out what information a company has, and what they can show their workers to help them do their jobs better, takes a lot of research and thought.
Then, companies like NGRAIN and iDENT have to figure out exactly how to take that information and create an augmented reality setup. This usually involves designing an image recognition system, so that the glasses or tablet devices can “recognise” what they see through their camera. “Very similar to facial recognition software but for machines,” Batstone says. Except that facial recognition might be easier than some of the systems they’re designing. “[Faces] are well suited to differentiation, big grey hunks of metal are much harder,” he says.
And when it comes to Holtorf and his dental glasses, there was an added complication: lag time. With things like oil pipelines and stealth jet inspections, a half second lag between what the camera sees and what is projected isn’t a deal breaker. But when Holtorf is poking around in someone’s mouth, there has to be an instant connection between what he sees through the glasses and the dental scans.
Holtorf and his colleagues solved this problem in two ways. First, they used a local network, rather than an internet connection, which can involve sending the data through remote servers, introducing a more pronounced lag. Incidentally, this local network approach helps solve another possible issue: security. “That way, also, the case sensitive data will stay inside, so we can guarantee that all the sensitive data will be inside the office,” he says.
The second strategy was to drop the resolution of the scan images. “We had to go a little bit down in resolution, to get no lag, but so far it’s okay for our case.” Holtorf says that in the next generation of the device, they’re hoping to get better resolution on the images.
But all these technical challenges are perhaps easier to overcome than the human one. No matter how amazing your system is, if your customers don’t use it, it’s worthless. “The people who are doing these jobs have been doing this job for a long time and haven’t used your technology,” Batstone says. Convincing people to use a tablet or put on a set of glasses to help them do a job they’ve been doing for years can be tough. Some of the early attempts at augmented reality glasses failed to really deliver because workers simply refused to wear them.
For Holtorf, acceptance was actually the easy part. His team showed their device recently at a conference for dentists, to overwhelming support. “Everybody who uses a scanning device said ‘Okay that’s great, that’s really great.’ Even very experienced users.” And Holtorf says his patients notice the difference too. “Most of the patients said ‘Yeah every time you’re doing this you’re at me, and not the machine.’”
When I ask Holtorf if he has thought up other medical uses for the glasses he laughs. “We’ve got a lot of ideas, of course.” He imagines them being beneficial during delicate surgeries, for instance, where a doctor might be searching for something hidden beneath the skin. But for now, Holtorf says they’re focused on making their dental glasses as perfect as possible.
So, the next time you're sitting in your own dentist's chair, keep an eye out to see if he or she is wearing an unusual pair of glasses. They could be looking at more than just your teeth.
BBC Future | By Rose Eveleth