Forest Data Visualization using Virtual Reality and Augmented Reality
Introduction
LiDAR known as light detection and ranging is a remote sensing technique featured prominently in the forestry industry as a way to provide information on forest structural characteristics such as canopy heights, forest density and growth rates. This information is gathered by emitting lasers that will bounce off surfaces and return providing X, Y, Z positions of those surfaces and plot them as points in a 3D space. However, much of this data is often viewed through 2D or 2.5D displays which cannot represent the data fully as it is inherently 3D in nature. This projects aims to examine how forest LiDAR data can be viewed and visualized in a true 3D format using VR and AR mediums. This will allow users to examine the data in its full spatial complexity and provide unique perspectives not possible when viewing through a screen.
An example of a forest LiDAR point cloud. The colours here represent the elevation of the points with red being the highest and dark blue being the lowest. This point cloud was created from a LiDAR dataset of the Lake Tahoe forest using OpenTopography.org point cloud viewer.
Virtual Reality Design
A virtual reality experience was setup in Unity using the SteamVR plugin. It allows the user to fly around different data sets of a forest. These data sets include:
1. A structure from motion (SfM) textured model of a small area in a local forest processed from an array of overlapping pictures.
2. The raw LiDAR point cloud data imported into Unity.
3. A LiDAR derived mesh that was processed from the point cloud information to create a terrain height map image.
3 cubes displaying the different scenes act as buttons for the user to change scene. In the VR interface they are attached to the controller in the users left hand so they can use the right controller to change scenes anytime.
The user begins on the LiDAR mesh scene with ability to teleport and fly around. The process of turning a point cloud into a mesh results in a simplification of the terrain, making trees in the scene look like giant spikes.
The point cloud scene as seen from above, a perspective given by the user's ability to fly around. VR also allows you to view different scales of data, for instance this dataset had been scaled up to the size of a small room in VR, meaning you could walk around it and sit down in it if the VR tracking capability exists.
The SfM forest scene provides a textured micro model of the forest allowing for the depiction of the space as it looks in real life. SfM is most useful at creating models of solid surfaces such as that of the tree trunk and forest floor, it however, struggles at being able to 'see through' or around objects such as leaves, as the model is processed from only what the camera captures in front of it.
The ability for the user to move anywhere on any scene allows them to view unique perspectives of the data such as this one from the bottom of the tree trunk in the SfM scene.
Augmented Reality Design
A augmented reality application was created in Unity using the Vuforia Engine. 2 AR tracking pages are setup side by side which allow for the viewing of the 2 LiDAR datasets through a camera or phone application. Both datasets are the same as 2 and 3 in the VR application which consist of the same patch of forest.
Using a webcam the 2 datasets are being tracked simultaneously and appear on the computer. Slider functionality is also shown.
Augmented reality provides another means of viewing datasets, here the forest can be viewed as a point cloud and a mesh simultaneously for quick comparison. Using AR technology allows the user to see the datasets through a camera as if they were in front of them.
Discussion
VR allows users to immerse themselves in the data and explore it with ease, by moving around, through, looking down from above, or up from below. It allows for the representation of the data in its true 3D form where patterns and perspectives not previously seen are revealed. However, the comparison between the point cloud and the LiDAR mesh exposes the limitations of forest representation where realism can easily be lost, especially in mesh format, as the forest turns to spikes. On the other,
point cloud data would be extremely useful in these interfaces if they could be combined with functionality to change point cloud textures and attributes and allow researchers or forestry experts to view datasets and change them on the fly. SfM also shows potential, as a close to real forest visualization, with the textures and shapes appearing as they are in reality. If this method were to be perfected, in all aspects, it would prove to be very useful as a tool for spatial simulation or analyses of forests at or close to the ground level, including phenomena involving individual trees.
The AR interface option currently resides as a more accessible option to data visualization and can be more easily combined with spatial phenomena functionality. When using the slider for directional light in AR, a major drawback of the point cloud in this space is that the directional light does not act upon it at any angle. The mesh therefore is better suited to operate as a platform for spatially rigorous simulation and representation that can be useful for visualizing many phenomena. Nonetheless, the ability to visualize datasets in front of you, twist them around or hold them close for a better look, highlight the usefulness for this type of interface to succeed as a tool for users to conceptualize their data.
Overall, these interfaces may be used as an aid to data representation and visualization which prove the usefulness, as well as the problems, VR and AR might have in the future of forestry, remote sensing and GIS.
This project was completed by Riley Smith in Spring 2019 for GEOG 457 Geovisualization Interfaces at SFU, with assistance from Ian Lochhead and Dr. Nick Hedley.