Design of a Naturalistic Navigational Virtual Reality Using Oculus Rift1

Our team has recently developed a virtual reality navigational (VRN) environment for spatial cognition assessment [1]. In order to make it more naturalistic, we aim to enhance the design by creating 3D immersive environment. Oculus Rift is a new affordable virtual reality head mounted display with a high view angle capability, which provides a very immersive view for the user. Using the Oculus Rift in addition with our own optical flow and laser sensors in Fig. 1, we can obtain orientation and height of the user's head, and incorporate it in our VRN design.In this study, we have redesigned our VRN environment using Oculus Rift as shown in Fig. 3. Using the VRN environment and the Oculus Rift, it is possible to develop applications for Alzheimer's research, treating phobia, rehabilitation, and studying different aspects of cognitive and spatial abilities.Oculus Rift device [2] includes a gyroscope, an accelerometer and a magnetometer. We combine the information from these sensors through a process known as sensor fusion to determine the orientation of the user's head in the real world, and to change the user's virtual perspective accordingly. In Oculus Rift, rotation is maintained as a unit quaternion, which we convert it into a rotation matrix of Euler angles to obtain data in terms of pitch, roll, and yaw.Sensor fusion does the integration of the angular velocity data from the gyroscope. In each tiny interval of time, a measurement of the angular velocity arrivesω=(ωx,ωy,ωz)where component is the rotation rate in radians per second about its corresponding axis. For example, if the sensors were set on the center of a spinning hard disk that sits in the XZ plane, then ωy is the rotation rate about the Y-axis, andωx=ωz=0When the Oculus Rift is rotating around some arbitrary axis, the length of ω is given as:ℓ=√(ωx2+ωy2+ωz2)Angular velocity vector, ω, we obtain:There might be some difference between the actual Rift orientation and the calculated Rift orientation, called drift error. Drift error can be divided into two independent plausible errors. The first error is a tilt error, which changes the orientation in either the pitch or the roll component. This error creates confusion as to whether the floor is level in a game. Tilt error is reduced by the accelerometer data, which is used to estimate the gravity vector. The second error is a yaw error, which creates uncertainty about the direction that the user is facing. This error is only about the Y-axis, and is overcome using magnetometer data. These three corrected values (pitch, yaw, and roll) are then given to the camera parameter of the game engine, which helps in rotating the VRN object in the direction opposite to that of the user's head.To get a more naturalistic view, we performed the rendering process. For the Oculus Rift, we are required to render in split- screen stereo, with half of the screen used for each eye. When using the Rift, our left eye sees the left half of the screen, whereas the right eye sees the right half. Thus, we render the entire scene twice; using two perspective position-orientation, for each eye.The lens in the Rift magnifies the image to provide an increased field of view, but this comes at the expense of creating a pincushion distortion [1] of the image. This radical symmetric distortion can be corrected through software by introducing a barrel distortion [2] that cancels it out.Our distortion mode is expressed in terms of the distance r from the center of the image. It is helpful to think of polar coordinates (r,θ) for a point in the image. A distortion model performs the following transformation:(r,θ)→(f(r)r,θ)where f(r) is the scaling function given by the following standard model:f(r)=k0+k1r2+k2r4+k3r6In other words, the direction θ is unharmed, but the distance r is either expanded or contracted. The four coefficients k0, k1, k2, and k3 control the distortion. They are all positive in the case of barrel distortion. With properly chosen coefficients, the pincushion distortion is canceled off by a barrel distortion (Fig. 2).We have used fragment shaders and vertex shaders [3] to do the rendering process and overcome the distortion. Shaders are part of a programmable pipeline. They both run on the graphics card's graphics processing unit. The vertex shader finally sets a 4D float vector and projects it into a 2D screen. The fragment shader sets the color of the fragment. For example, if we take a triangle with three vertices, a fragment is the data provided by the three vertices for the purpose of drawing each pixel in that triangle. This gives us flexibility to play around with the image, and produce naturalistic pictures.After performing the entire process mentioned above we will obtain the screen as shown in Fig. 3. With this in place, head tracking was established using external sensors and a camera in which the lens was focused towards the ground. The Lucas–Kanade method was used for calculating optical flow [4].Interfacing Oculus Rift with our VRN environment resulted in a more pragmatic picture of our designed virtual building. Output has shown very high accuracy between the user's head motion and the motion of the image. By using properly chosen coefficients, barrel distortion was able to nullify the effect of pincushion distortion. We divided the screen into two halves, but on the Rift these two screens become one. Since Oculus covers the eyes fully, this will not allow the user to see anything except the view projected on the laptop monitor; thus causing an immersion in the virtual environment. In our game engine, if the user is standing outside and looking up, he/she can see the sky, and if looking down he/she will see the grass. Similarly, if the user moves close to the building using keyboards/joystick or our wheelchair [5] then he/she feels as if he/she is getting physically closer to the house. When the player goes inside the building, he can view the 3D virtual building in a highly naturalistic form.The integration of our recently designed VRN environment with Oculus Rift has shown great results in terms of providing a naturalistic immersed virtual reality environment, in which the user can navigate and feel present in the environment. We believe, with this technology, we can assess several aspects of human brain functions in a natural way, and thus gain profound understanding of our brain's mechanism.As for future work, we plan not only to apply this version of our VRN with Oculus Rift to our pool of volunteers for spatial cognition assessment but also to interface the Rift with the translational motion detectors on the body of the user and enhance the naturalistic quality of our virtual world significantly. We believe, with this tool we can revisit many neuroscience findings of human brain function.This study was supported by Natural Sciences and Engineering Research Council (NSERC) of Canada. Also the stipend of the second author's salary was provided by MITACS Canada.

Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.