Chapter 1: Introduction to the Project

This report outlines the outcomes of a completed project focused on the dual-handed manipulation of molecular models using cost-effective virtual reality (VR) tools within WebXR-compatible web browsers.

Grantee: Luciano A. Abriata

Funder: Hasler Stiftung

Initially, the project aimed to utilize a commercially available low-cost infrared camera to track users' hands for manipulating virtual molecules. However, rapid technological advancements necessitated a pivot towards a more sophisticated and affordable VR solution: the Oculus Quest 2, a standalone VR headset priced at 400 CHF that features exceptional hand-tracking capabilities and a Chromium-based WebXR-enabled browser. This transition, approved by the Hasler Foundation on August 31, 2021, significantly enhanced user immersion.

The adapted project had three primary objectives, all of which were successfully achieved, leading to peer-reviewed publications and practical applications. These objectives included: (1) developing a WebXR application that integrates web graphics and simulation libraries for viewing and manipulating virtual molecules with bare hands on Oculus Quest 2; (2) leveraging WebRTC for seamless browser-to-browser communication to facilitate collaborative sessions; and (3) assessing user experiences with a group of scientists familiar with molecular modeling.

In terms of technical execution, we encountered and resolved several challenges, ultimately achieving smooth manipulation of molecular structures on various devices, including the Oculus Quest 2. However, for larger molecular systems requiring complex calculations, we identified the necessity for external computational resources. Additionally, we noted the urgent need for reusable controllers to create user interfaces in VR, as current libraries do not meet this requirement. We may seek additional funding from the Hasler Foundation to address these issues in future proposals.

Chapter 2: Project Goals and Achievements

The project's goals were centered on the development of a tool enabling experts, educators, and students to collaborate in immersive environments using augmented or virtual reality (AR/VR) to explore molecular structures and data effectively.

Aim 1: Develop a WebXR App

The first objective was to create a WebXR application allowing users to manipulate virtual molecules directly within their browsers using their hands. This involved both utilizing pre-made VR objects and constructing molecular representations from atom coordinates.

The video showcases hand and gesture detection capabilities using WebXR and Three.js, illustrating the app's functionality.

Aim 2: Enable Collaborative Sessions

The second objective focused on employing WebRTC to facilitate real-time collaboration between users manipulating the same virtual molecules. This endeavor included creating sender and receiver applications, with the sender app handling calculations and the receiver app visualizing the results.

This video discusses out-of-the-box solutions for cross-platform AR/VR experiences using Web technologies.

Aim 3: Assess User Experience

The third aim involved evaluating user experiences by comparing the efficiency of molecular manipulation tasks performed in VR against traditional setups. The findings revealed notable improvements in task completion times when using VR tools.

Summary of Results

The project successfully demonstrated that immersive VR tools enhance users' ability to manipulate molecular structures more intuitively. Users reported that the key benefits stemmed from the 3D nature of the environment and the ability to use both hands for interaction. Observations from collaborative sessions indicated that users could work together without interfering with one another's tasks.

Chapter 3: Future Prospects

Our work has laid the groundwork for future research and development in the realm of VR-based molecular modeling and education. However, we identified challenges that must be addressed, including the need for advanced computational resources for larger molecular systems and the development of user-friendly graphical interfaces.

The success of this project has opened avenues for securing additional funding to further enhance our VR capabilities and user experience, ultimately contributing to the advancement of educational tools in chemistry and structural biology.

Conclusion

In summary, the project has successfully merged technology with education, establishing a robust framework for interactive, immersive experiences that significantly benefit the fields of molecular modeling and chemistry education.