I was recently posed an interesting problem. What sort of technology could we use to generate an interactive training/tutorial application on a webpage that can be viewed on a phone? Sure we could develop an app, but then we’d have to force people to download the training.
Sounds like an interesting problem to solve, right? From a technology perspective, the website part isn’t all that scary. Even having to support mobile browsers is also something we’ve done before. But what about the interactive scene itself? We’ll probably need something rendered with WebGL. As this isn’t the type of technology that clients typically have a need for, we don’t see that too often. So I began researching technologies that might be good fits for this problem.
Personally, the last time I worked with WebGL was over five years ago, but that was only 2D scenes. For this problem, we’re likely going to need something that can handle 3D. Let’s take a look at what advancements were made in the last five years. To help me get started, I was directed toward a few engines that might fit, Unity and 8th Wall. So we’ll take a look at those first.
Problem
Before I delve too deeply at potential solutions/technologies, let’s dig into the requirements a bit and identify what sort of ramifications they may have.
From their ask, we can identify the following requirements:
- Interactive Application
- Browser-based
- Mobile Friendly
As this may eventually end up on a client project, there is also the implicit requirement:
- Can we use this technology in a commercial product?
Interactive Application
Interaction can take several forms here. If we wanted something simple, we could do something where each step in a workflow had some sort of animation, and the user would click buttons to traverse the individual steps. If we wanted something more immersive, we would have the users manipulate a scene, zoom in and out, rotate and move objects. As the users interact with the system, it would prompt them to perform a set of instructions. Once completed, users are prompted with the next set, repeating until the task is finished.
If we went with a step-by-step workflow, leveraging a graphics library or even just recording static content on each step in the workflow would both be potential solutions. For a more immersive environment, we’ll likely need some sort of graphics library. I opted to focus on the latter, as that would be a more challenging endeavor.
Browser-based
Ok, so having an interactive webpage where they can manipulate a scene, likely means some degree of HTML5 Canvas rendering. Most browsers support WebGL2, and there are a lot of libraries out there that leverage it. This feels incredibly doable!
Mobile Friendly
Paired with the previous requirement, this means that at least the mobile versions of Chrome and Safari need to be considered. Luckily most default mobile browsers currently support WebGL2, so we’re somewhat safe on that front. The issue then becomes more about resource management, as something that renders in desktop browsers may not be performant when viewed on a phone. Still, with improving technologies on every generation of phone, this is becoming less of an issue.
Can we use this technology in a commercial product?
This is a very important requirement. Ultimately this technology may end up in a product being sold by a client, so they’ll need to be able to do so legally. If they can’t, there’s no sense in looking into the technology all that deeply.
Technology Investigations
Now, let’s take a look at some technologies that can render 3D scenes in a browser:
Unity
Unity is a powerful gaming engine that supports both 2D and 3D experiences. It supports C# natively, and is supported on numerous different platforms, including Android and iOS.
So does it check our boxes?
Interactive Application?
Yes.
Browser-based?
Yes, Unity can be loaded in browsers.
Mobile Friendly?
Yes, but not in browsers. Per their documentation, the engine is still too resource intensive to be performant on many mobile browsers. My takeaway from this is that it might work (at least on some devices), but Unity doesn’t guarantee anything in terms of quality or performance.
Can we use this technology in a commercial product?
Unity has several licensing plans available. There are some restrictions about what plans can be used for commercial products, but we don’t need to go into it here.
Conclusion
Because it’s not mobile browser friendly, it doesn’t seem like Unity is what we want in this instance.
8th Wall
8th wall is a platform developed by Niantic to provide WebAR support for browsers (specifically mobile browsers). They attempt to make things easy and work “out of the box” by providing online development/deployment tools that would let you easily publish your project into the 8th Wall Ecosystem. Self-hosting is still possible at higher tiers in their pricing model, but most of their documentation and workflows are assuming that the developers are using the cloud-based tools that 8th wall provides.
In practice, 8th Wall libraries are written to enhance and extend other WebGL libraries written in JavaScript. They support applications written with A-FRAME, three.js, BabylonJS and several others.
So let’s go through the checklist again:
Interactive Application?
Yes, 8th Wall libraries extend JavaScript WebGL libraries to add more AR-friendly interactions.
Browser-based?
Yes
Mobile Friendly?
Yes
Can we use this technology in a commercial product?
8th Wall has its pricing model here: https://www.8thwall.com/pricing, which does include options for Commercial Licensing.
Conclusion
This seems a bit more promising, but the problem didn’t have any requirement around AR/VR capabilities. The specific licensing requirements for the prospective application are also still unknown. So what about the libraries that it extends? What can they do?
A-FRAME
A-Frame is a framework written around three.js(https://threejs.org/), that provides a declarative syntax to add A-Frame controls on a website. Essentially A-Frame is written by importing a script, and then adding A-Frame specific tags to the page as HTML, making it incredibly simple to spin up a new web page.
Its design leverages an entity component system (https://aframe.io/docs/1.4.0/introduction/entity-component-system.html), which allows the re-use of component modules across multiple entities (namely the objects in the scene). For modeling, A-Frame can natively load .obj and .gltf models.
Does it check our boxes?
Interactive Application?
Yes, many of the examples on their home page have some degree of interaction. Camera controls are available out of the box, but object-level behaviors will need to be custom.
Browser-based?
Good here too. It’s also a JavaScript library that uses WebGL to create scenes on a page.
Mobile Friendly?
In terms of browsers, A-Frame supports most default mobile browsers (https://aframe.io/docs/1.4.0/introduction/vr-headsets-and-webxr-browsers.html#which-browsers-does-a-frame-support).
Can we use this technology in a commercial product?
A-FRAME is licensed under the MIT License, which means that it is allowed to be used in commercial product.
Conclusion
A-Frame is generic, forcing users to develop their own interactions by adding new modules to entities (like drag/drop for instance), but still feels like its focus leans more towards VR. There are examples for non-VR, so still a good candidate for what we’re trying to do.
BabylonJS
BabylonJS is another JavaScript based WebGL gaming framework. Unlike some of the other technologies we’ve looked at so far, it’s more geared towards the 2D/3D modeling space. That said, it doesn’t shut the door on the AR/VR capabilities, they’re just not the focus.
It’s an expansive framework, with hooks for many standard interactions. Having support for pointer-events is a plus. It also tries to cater to a lot of different use cases. It supports .glTF, .glb, .obj, .stl and custom .babylon model types.
Does it check our boxes?
Interactive Application?
Yes, custom behaviors can be added to models and the scenes through callback hooks that are defined in the framework.
Browser-based?
Yes again. It’s also a JavaScript library that uses WebGL to create scenes on a page.
Mobile Friendly?
Per the documentation, it supports any browser that has WebGL/WebGPU support. Some features will not work if the browser does not support WebGL2.
Can we use this technology in a commercial product?
BabylonJS is licensed under the Apache License 2.0, which means that it can be used in a commercial product.
Conclusion
BabylonJS is more familiar for those who have used graphics libraries in the past. It has good documentation, and an easy-to-use sandbox to try things out without having to invest too much into spinning up infrastructure. Definitely a solid choice.
What’s next?
On paper, both A-Frame and BabylonJS seem promising and could likely solve the problem. Both are flexible JavaScript libraries that will leverage WebGL to render complicated scenes. Both can be hooked back into 8th Wall (if desired). So where do we go from here? Of the two options, BabylonJS feels a bit more flushed out. But we won’t really know how well it works until we try it, so the next step is going to be using it to prototype something.
