TL;DR
This article covers how to implement a 360˚ video player in JavaScript. You can find the complete code on the following link:
https://gist.github.com/bepro-dev/a640f0c2048d4d02a063476ee8a2a3a3
And here is the final result.
BEPRO Video Player
We (BEPRO) install multiple cameras next to a pitch to film matches or trainings for our clients. The number of cameras installed at each pitch is between two and four, which is determined by the distance between the cameras and the ground, the height of the cameras and the angle at which the cameras look down to the ground.
Images taken by each camera are stitched based on the feature points in the overlapping area in each image and processed into a panoramic image that can capture the entire pitch on a screen.
However, these panoramic images have a very wide field of view (FOV) which makes videos look distorted. To give a better viewing experience to our users, we developed a special video player to display videos without distortion. You may have had a chance to take a look at our 3D video player if you have already tried our service or have visited the BEPRO demo website.
Unlike the original panoramic video, the videos played with BEPRO’s 3D Video Player have no distortion and the video can be zoomed in/out and users can freely rotate their view to see wherever inside the pitch.
BEPRO has implemented this video player in multiple languages for each platform. We’re going to take you through the steps you need to take to implement a 3D video player in JavaScript, which can then be used in a web browser.
Fundamentals
BEPRO’s 3D Video Player basically works the same as a general 360˚ video player.
However, as mentioned earlier, the shape of the ground captured by each camera differs depending on the location and conditions where the cameras are installed. Accordingly, BEPRO’s panoramic video has various FOVs depending on the pitch, which makes it hard to apply the mechanism of general 360˚ video player which have FOV of 360 degrees horizontally and 180 degrees vertically. Therefore, in BEPRO’s 3D Video Player additional processing is required to use only a certain part of the sphere, not the entirety of it.
In this article, we’re going to practise with the example of the general 360˚ video player which is more commonly used.
3D Engine
You can easily implement a 360˚ video player if you use a 3D Engine. It’s not necessary to have experienced a 3D engine to follow this tutorial, but please read the brief introduction of a 3D Engine below if you want to understand the basics of it.
Basic concepts in a 3D Engine
Scene, Renderer, Camera
A scene, a renderer and a camera are needed to display something with a 3D Engine. A scene is like a virtual 3D space. After placing 3D objects and a virtual camera in a scene, you can render the image captured by the virtual camera on the screen through a renderer.
Perspective Camera, Orthographic Camera
There are two types of cameras — perspective and orthographic. A perspective camera is used to display 3D graphics while an orthographic camera is used to display 2D images although the scene itself is 3D.
Geometry, Material, Mesh
To create a 3D object which will be shown in the scene, a geometry, a material and a mesh are needed.
In a 3D Engine, each object consists of multiple triangles and a geometry contains information of vertices and faces of these triangles.
A material decides how the surface of an object will be rendered. The object can be painted with a solid colour, a complex pattern or multiple colours by using a Shader, or covered with an image as a texture.
A mesh is a 3D object made up of triangles, which is the unit of an object that is actually displayed in the Scene. In THREE.js, the material defines the surface of the geometry, and the position, size and direction of the object are set through the mesh.
Implementing 360˚ video player through a 3D Engine can be carried out in the following steps:
- Create a scene
- Create a sphere
- Place a perspective camera in the centre of the sphere
- Put a video as a texture on the inner surface of the sphere
Now, let’s look at each step in more detail with working code.
Implementing a 360˚ Video Player
To implement a 360˚ video player, we are using THREE.js (https://threejs.org/) which is a popular JavaScript 3D Engine.
1. Create an HTML file and install THREE.js
(https://threejs.org/docs/#manual/en/introduction/Installation)
THREE.js can be installed via a package manager like npm or from CDN. We are going to use r128 version which is the most recent version at the moment(18/03/2022) served by cdnjs.
- CDN
2. Create a Scene
First of all, we need to create a scene to display something with a 3D Engine. A camera and other objects will be added to this scene.
3. Place a Camera and add a Renderer to the screen
We can create a camera anytime before rendering, but let’s create it first and then add a renderer to the screen so that interim results can be checked during the tutorial.
The value of near and far parameters of PerspectiveCamera are distances from the camera respectively and objects only between near and far are captured by the camera and displayed on the screen. The frustum between Near Clip and Far Clip in the image below is the area which will be shown on the screen.
Although we didn’t pass any arguments to the constructor of WebGLRenderer, it provides some options by default to optimise image quality and performance. You can check the details in THREE.js documentation.
Now we can see a black rectangle on the screen.
4. Create a sphere
The three values we passed to the SphereGeometry constructor are radius, widthSegments and heightSegments. The radius doesn’t affect the visible shape since we are going to place a PerspectiveCamera in the centre of the sphere, but it has to be bigger than near and smaller than far of PerspectiveCamera to actually be displayed on the screen.
widthSegments and heightSegments define how many triangles the sphere should be divided into horizontally/vertically, and the larger the number, the better the image quality but of course the performance decreases.
Compare the quality depending on different widthSegments and heightSegments values
- widthSegments=8, heightSegments=4
- widthSegments=32, heightSegments=16
- widthSegments=32, heightSegments=16
- widthSegments=128, heightSegments=64
5. Put a video on the inner surface of the sphere
Let’s check the sphere before putting a video.
The whole screen looks blue since the camera is inside of the blue sphere. Let’s move the camera a bit so that we can see the sphere from outside. If the position or rotation values are not given separately, the camera is located on (0, 0, 0) and looking at -z direction by default. As we’ve set radius of the sphere to 15, let’s move the camera farther than 15 to +z direction.
To cover the sphere with a video, we can use a VideoTexture.
You can see the material.side is set to THREE.BackSide. Triangles consisting the surface of a sphere are facing outside by default and if we put a texture on them, it can only be seen from the outside of the sphere. But as we put the camera inside of the sphere, the material should be set to BackSide - to make the triangles face inward towards the centre of the sphere.
6. Start Animation Loop
As the screen is updated only once when the function renderer.render() is called, it must be updated for every frame using the function renderer.setAnimationLoop() to show the video being played.
Now you can see the undistorted video being played instead of a black rectangle.
7. Zoom in/out and rotation
It’s time to implement the essential features of a 360˚ video player, zoom in, zoom out and rotation. These can be implemented by controlling the virtual camera.
Zoom in/out
Zoom in/out can be implemented by changing the FOV value of the camera. Reducing the FOV does not change the size of the renderer but it narrows the visible area which enlarges the screen, while increasing the FOV reduces the screen by enlarging the visible area.
Rotation
Every 3D object including the camera have position and rotation values and all their default values are {x: 0, y: 0, z: 0}. To rotate an object, use the function .rotateX(), .rotateY() and .rotateZ() in Object3D. These functions rotate the object based on the x, y and z axis respectively.
Now you can zoom in/out by rolling your mouse wheel and drag the screen to see other parts of the image.
Result
You can find the final result here and check the complete code in the following link. Compare it with your code in case yours doesn’t work properly.
https://gist.github.com/bepro-dev/a640f0c2048d4d02a063476ee8a2a3a3
.avif)
