XR stands for Extended Reality. WebXR Device API provides access to input and output capabilities commonly associated with Virtual Reality (VR) and Augmented Reality (AR) devices. It allows you develop and host VR and AR experiences on the web. In this post, we will be learning about WebXR, how it works on the browser, and how to build our first VR application with React and WebXR.
The WebXR Device API is an API designed to access the sensors and displays of immersive hardware hardware such as VR headsets, augmented reality headsets, and a wide range of immersive devices. It. works by knowing what the user is looking at from the devices sensors.
Some instances where WebXR can be deployed to are:
- 360deg and 3D videos
- 3D object and data visualization
- Artistic experiences
WebXR is a WebGL (Web Graphics Library) based API. This means you can’t render 3D experience with HTML and CSS, it’s purely restricted to WebGL. WebGL is a JavaScript API for rendering high-performance interactive 3D and 2D graphics within any compatible web browser without the use of plug-ins. The bad news is that WebGL is very verbose, it takes a lot of work to just render a simple cube. The good news is we have some JavaScript libraries that abstracts all that complexity. Some of those libraries are: PlayCanvas, three.js, A-FRAME, babylonjs, react-three-fiber, react-xr, e.t.c Let’s go ahead and build a 3D scene with some object using react-three-fiber.
React-three-fiber is a JavaScript library for creating 360degree immersive VR experiences. It uses the same declarative concepts in React and use the threejs engine for 3D rendering. react-three-fiber is an abstracts the complexity of threejs and makes it more accessible for React developers to build VR experiences.
Pre-requisites
You would need to have the following to flow with this guide:
- Nodejs >=v14 installed
- Knowledge of JavaScript and React
- A code editor (preferably VS Code)
- Basic Threejs knowledge. Go through this.
- Basic knowledge of react-three-fiber. Go through the documentation
The complete code for this article is on CodeSandbox.
Getting Started
Let’s get started by creating a React app with create-react-app. Run this command on your terminal:
npx create-react-app react-xr-app
When it’s done installing, open the project directory with your code editor.
Go ahead and delete all the files from the src folder except App.js and index.js.
Install the following packages:
npm install three @react-three/fiber @react-three/drei lamina react-merge-refs
Code language: CSS (css)Now paste this code snippet
//This code is gotten from Three.js templates
//App.js
import * as THREE from 'three'
import { Canvas } from '@react-three/fiber'
import { Sparkles, Shadow, ContactShadows, Billboard, Environment, BakeShadows, OrbitControls } from '@react-three/drei'
import { LayerMaterial, Depth } from 'lamina'
const App = ({ env = 'https://res.cloudinary.com/sammy365/raw/upload/v1658828418/nat-callaghan-GOxq9KQBSw0-unsplash_lu0jbk.hdr' }) => (
<div style={{ height: "80vh" }}>
<Canvas shadows camera={{ position: [0, 0, 12], fov: 60 }} height="800">
<hemisphereLight intensity={0.5} color="white" groundColor="black" />
<Environment files={env} ground={{ height: 5, radius: 40, scale: 20 }} />
<Sphere color="white" amount={50} emissive="green" glow="lightgreen" position={[1, 1, -1]} />
<Sphere color="white" amount={30} emissive="purple" glow="#ff90f0" size={0.5} position={[-1.5, 0.5, -2]} />
<Sphere color="lightpink" amount={20} emissive="orange" glow="#ff9f50" size={0.25} position={[-1, 0.25, 1]} />
<ContactShadows renderOrder={2} color="black" resolution={1024} frames={1} scale={10} blur={1.5} opacity={0.65} far={0.5} />
<BakeShadows />
<OrbitControls autoRotateSpeed={0.85} zoomSpeed={0.75} minPolarAngle={Math.PI / 2.5} maxPolarAngle={Math.PI / 2.55} />
</Canvas>
</div>
)
const Sphere = ({ size = 1, amount = 50, color = 'white', emissive, glow, ...props }) => (
<mesh {...props}>
<sphereGeometry args={[size, 64, 64]} />
<meshPhysicalMaterial roughness={0} color={color} emissive={emissive || color} envMapIntensity={0.2} />
<Glow scale={size * 1.2} near={-25} color={glow || emissive || color} />
<Sparkles count={amount} scale={size * 2} size={6} speed={0.4} />
<Shadow rotation={[-Math.PI / 2, 0, 0]} scale={size} position={[0, -size, 0]} color={emissive} opacity={0.5} />
</mesh>
)
const Glow = ({ color, scale = 0.5, near = -2, far = 1.4 }) => (
<Billboard>
<mesh>
<circleGeometry args={[2 * scale, 16]} />
<LayerMaterial
transparent
depthWrite={false}
blending={THREE.CustomBlending}
blendEquation={THREE.AddEquation}
blendSrc={THREE.SrcAlphaFactor}
blendDst={THREE.DstAlphaFactor}>
<Depth colorA={color} colorB="black" alpha={1} mode="normal" near={near * scale} far={far * scale} origin={[0, 0, 0]} />
<Depth colorA={color} colorB="black" alpha={0.5} mode="add" near={-40 * scale} far={far * 1.2 * scale} origin={[0, 0, 0]} />
<Depth colorA={color} colorB="black" alpha={1} mode="add" near={-15 * scale} far={far * 0.7 * scale} origin={[0, 0, 0]} />
<Depth colorA={color} colorB="black" alpha={1} mode="add" near={-10 * scale} far={far * 0.68 * scale} origin={[0, 0, 0]} />
</LayerMaterial>
</mesh>
</Billboard>
)
export default App;
Code language: JavaScript (javascript)Here, we start by creating the environment with react-fiber Canvas. If you notice, you can see a cloudinary image, that 3D image will be represented as our 3D environment.
<Sphere color="white" amount={50} emissive="green" glow="lightgreen" position={[1, 1, -1]} />
<Sphere color="white" amount={30} emissive="purple" glow="#ff90f0" size={0.5} position={[-1.5, 0.5, -2]} />
<Sphere color="lightpink" amount={20} emissive="orange" glow="#ff9f50" size={0.25} position={[-1, 0.25, 1]} />
Code language: HTML, XML (xml)const Sphere = ({ size = 1, amount = 50, color = 'white', emissive, glow, ...props }) => (
<mesh {...props}>
<sphereGeometry args={[size, 64, 64]} />
<meshPhysicalMaterial roughness={0} color={color} emissive={emissive || color} envMapIntensity={0.2} />
<Glow scale={size * 1.2} near={-25} color={glow || emissive || color} />
<Sparkles count={amount} scale={size * 2} size={6} speed={0.4} />
<Shadow rotation={[-Math.PI / 2, 0, 0]} scale={size} position={[0, -size, 0]} color={emissive} opacity={0.5} />
</mesh>
)
Code language: JavaScript (javascript)This snippet renders the Spheres. There is a Glow effect on the Spheres, this part of the code implements that:
const Glow = ({ color, scale = 0.5, near = -2, far = 1.4 }) => (
<Billboard>
<mesh>
<circleGeometry args={[2 * scale, 16]} />
<LayerMaterial
transparent
depthWrite={false}
blending={THREE.CustomBlending}
blendEquation={THREE.AddEquation}
blendSrc={THREE.SrcAlphaFactor}
blendDst={THREE.DstAlphaFactor}>
<Depth colorA={color} colorB="black" alpha={1} mode="normal" near={near * scale} far={far * scale} origin={[0, 0, 0]} />
<Depth colorA={color} colorB="black" alpha={0.5} mode="add" near={-40 * scale} far={far * 1.2 * scale} origin={[0, 0, 0]} />
<Depth colorA={color} colorB="black" alpha={1} mode="add" near={-15 * scale} far={far * 0.7 * scale} origin={[0, 0, 0]} />
<Depth colorA={color} colorB="black" alpha={1} mode="add" near={-10 * scale} far={far * 0.68 * scale} origin={[0, 0, 0]} />
</LayerMaterial>
</mesh>
</Billboard>
)
Code language: JavaScript (javascript)
This line of code adds the rotating orbit effect
<OrbitControls autoRotateSpeed={0.85} zoomSpeed={0.75} minPolarAngle={Math.PI / 2.5} maxPolarAngle={Math.PI / 2.55} />
Code language: HTML, XML (xml)If you followed the links on the pre-requisites, you’ll have a fair understanding of what’s going on here.
Save and run your app with npm run start. Navigate to http://localhost:3000 on your browser and you will be presented with this screen:
https://www.dropbox.com/s/8ige6ek8uielelv/webxr.webm?dl=0
Awesome. You can find more examples using react-three-fiber here.
The WebXR Devive API is still on active development, these are some features we expect to come soon:
- DOM Overlay API for better accessibility
- Lighting estimation using Computer vision
- Hand interactions.
In this article, we introduced the WebXR Device API by explaining what it is, we also outlined some examples where WebXR can be deployed to and we discussed how to build a WebXR experience. We went further to explore a React library that abstracts three.js rendering engine called react-three-fiber. We went on to build a simple sample app with the library and concluded with highlighting WebXR features we are to expecting soon.
I hope you picked up a thing or two from this article.
Happy Coding!