Editor’s note: This article was co-written by DEG UI Engineer Anna Scheuler.
In July 2016, the software development company Niantic released the long-awaited mobile game, Pokémon GO. This mobile game uses characters from a world-renowned franchise (started in 1996), but with a new twist. In this game, players are placed on a map—based on their current location—and by walking around their surroundings, a user has an opportunity to run into a wild Pokémon. Augmented reality is utilized to display a Pokémon in the user’s environment that the user can then interact with.
This and other products, like Snapchat filters, have brought augmented reality into the forefront of modern technology experiences. So, how do we make it our new reality?
What is augmented reality?
Augmented reality (AR) is a form of technology that overlays virtual objects over real-world environments. It uses a device’s camera to see a user’s current environment and puts a specially created object into the scene, visible on the device’s screen.
It differs from virtual reality (VR) in that VR is a completely immersive, virtual experience. It also differs from mixed reality (MR) in that this environment allows users to interact with the virtual objects.
How we tried to use AR
For the past two years, members of the DEG UI, Creative, and Marketing teams have come together to celebrate Halloween by putting on a “Conference Room of Horrors.” This is a showcase to demonstrate new and experimental technologies to other DEG associates, implemented as web applications with a spooky theme. This year, we attempted to integrate augmented reality.
For 2018, our plan was to utilize AR to create something unique and engaging for all associates, remote or in-office. We wanted users to be able to point their phone camera at any open area and then have a scary character appear in that space. This would allow people to take pictures with their favorite scary character (i.e. Pennywise or the Headless Horseman) in their current environment.
What we ended up building was a web application that asks the user to point their device camera at a specific marker. Once the device recognizes the marker image, a friendly spider object would appear.
A recurring issue in ecommerce is that customers cannot see a product’s detail or test how it works in their space. By using AR, it can increase the consumer’s confidence and allow them to be more comfortable purchasing products online.
Though we got a project up and running, there were many roadblocks along the way. The first challenge was choosing the proper AR library.
We wanted to make this experience web-based to be accessible to the largest audience, so we had to find technologies that allowed us to do that. We knew that augmented reality existed for native apps with technologies like ARCore or ARKit, but we didn’t want to deal with the hassle of adding apps to the Google and Apple stores. Plus, this approach didn’t gel well with the overall “Conference Room of Horrors” experience, which was web-application based.
Web-based technologies that utilize ARCore and ARKit exist but come with their own limitations, mostly a lack of browser support. The second roadblock we faced was getting a stable development environment set up in which to build an ARCore- or ARKit-based augmented reality experience.
In order to create that environment, developers have to download an experimental app called WebARonARKit for iOS, and WebARonARCore is needed for Android. For users on iOS, only iOS11 was supported, which was an issue since at the time the most recent iOS version was iOS12.
For Android, the experimental browser required the most recent Android version (Android O), as well as a compatible phone and certain Chrome flags enabled. On both platforms, there was enough setup, that we felt our audience would quit the process before even getting the application working.
After much research, we decided to use A-Frame-AR, an AR web framework built with declarative HTML. This library has solid documentation, examples, and uses HTML elements to implement augmented reality. Since web AR is still being developed, having detailed documentation and practical examples heavily influenced our selection. The framework was easy to get started with but challenged us to go beyond what was available in the tutorials.
Using AR will enhance your customers’ online shopping experience, as well as create immersive campaigns and promotions for marketing. It will help you form the new reality.
A third roadblock we faced was trying to create a markerless AR experience. In our working application, the user has to have a separate image (marker) to point their camera at to make the 3D model appear. With markerless AR, the user would just point their camera in any direction and a 3D object would materialize. This type of experience would be more usable for all DEG associates, especially remote associates, because it only required the user to have their phone. Unfortunately, we didn’t find any web-based AR technologies that allowed us to create a markerless experience, so we had to think of something else using the tools we currently had.
So, instead of going with “no markers,” we pivoted to try a multi-marker experience. We found out that A-Frame allows you to use multi markers, which gives your phone the ability to “learn the area” based on how the markers are set up. Also, multi markers made 3D objects more stable, and allowed users to scale those objects to any size they wanted.
We thought taking a picture with a life-sized scary character would be a cool (and more frightening) experience. Unfortunately, the user interface to “learn the area” and place 3D objects in your environment was a bit clunky and hard to use. This, again, would have taken a bit of time to set up which could have led users to be frustrated and bail before experiencing the AR part of the app.
Even though we went through all these roadblocks, we believe that as soon as Web AR is supported on more browsers, we will be able to create these experiences with far fewer issues.
How it can benefit us in the future
With AR being a revolutionary technology that will change how people view reality, there will be many ways to enhance the experience for companies, as well as for their clients. One way we can leverage AR is by creating a preview on product images for ecommerce websites. This preview would be a 3D object that users can place in the environment around them. This will allow users to see and evaluate an item as if they were in a physical store.
A recurring issue in ecommerce is that customers cannot see a product’s detail or test how it works in their space. This can lead to doubt and frustration, causing a customer to leave the site without continuing the checkout process. By using AR, it can increase the consumer’s confidence and allow them to be more comfortable purchasing products online.
AR can also help us when it comes to creating marketing campaigns and promotions. Marketers will be able to create digital experiences that tell a story and better communicate the brands messaging to the consumer.
Companies such as IKEA, Home Depot, Pepsi, and L’Oreal have all utilized AR in their advertising campaigns and have seen positive gains in views, user engagement, and conversions. In fact, Pepsi had a campaign that went viral and ended up becoming one of YouTube’s most viewed advertising campaigns.
When consumers come across these types of interactive experiences, engagement increases significantly. For example, brands that adopted augmented reality early in their campaigns have seen a 30% increase in user engagement, and have also seen an increase in the length of time spent consuming content (Source: Blippar).
The new reality
Augmented reality is a cutting-edge technology that is slowly working its way into mainstream applications. Taking advantage of this technology will be a learning process as more information continues to come out and browser support becomes better. Using AR will enhance your customers’ online shopping experience, as well as create immersive campaigns and promotions for marketing. It will help you form the new reality.