Is Immersive Online Content (IOC) Approach in the Form of Virtual Reality (VR) an Effective Method for Generating High-Quality Online STEM Labs?

Project Rationale

There is a gap between faculty willingness to hold online STEM laboratory courses and student demand for high-quality online instruction. While online courses increase student flexibility and access to course content and instruction (Pellitier, 2022), faculty resistance is based on the lack of high-quality resources, the predominant use of canned labs that lack faculty presence, and the conception that students only learn laboratory techniques with traditional hands-on, in-person experiences (Becker, 2017). To bridge the gap between student demand for high quality online instruction and faculty resistance to STEM laboratories due to lack of high-quality hands-on experiences, this study aims to implement and measure the effect of immersive online content in the form of virtual reality STEM labs using faculty assessment of student learning gains using laboratory report grades, perception of student learning gains, student virtual engagement, student confidence in scientific communication and technical learning skills, and student perception of applicability in future careers.  

Project Context

This study was conducted on 10 distance learning undergraduate students taking a fully online Neurophysiology course at Florida Atlantic University. The course covers concepts of cellular neuroscience, the process of scientific research and communication, and the technical skills of electrophysiology using immersive online content in the form of virtual reality laboratories with probing questions with just-in-time feedback, and simulations. Florida Atlantic is a Hispanic-serving institution serving 30,000+ students across 6 campuses that span 110 miles along the southeast coast of Florida. A recent FAU Graduating Student Survey demonstrated that students want more flexibility and choice of course modality, which is consistent with national trends (Caron & Muscanell, 2022). In a recent BCSSE Survey,  46-48% of students reported caring for at least one dependent, 85% of transfer students commute over 1 hour to campus each day one way, and 52-58% of first-year and 71-74% of transfer students work at least 12hr/week and 74-79% expect stress in balancing school and other activities. This study aims to bridge the gap between traditional in-person STEM laboratory offerings and student demand for flexible, high-quality instruction by assessing the effectiveness of immersive online content in the form of virtual reality on faculty assessment of learning gains and student perception of learning.  

Supportive Literature

While enrollment in online courses during the COVID-19 pandemic reached a record high, demand for online courses has continued to accelerate with online and hybrid courses seeing considerable growth of 36% and 20% respectively (Richard et al., 2023). The increase in demand for online courses is attributed to the increased flexibility and accessibility of the online course modality (Caron & Muscannell, 2022; Müller, 2023) as well as a changing student demographic in higher education. During the pandemic, students experienced learning flexibility that they continue to value (Pellitier, 2022). Additionally, the average age of the undergraduate student is rising from 20 in 2013 (Yale Common Data Set 2013-2014) to 23 in 2023 (Berg et al., 2024) and many undergraduate students report caring for dependents, long commutes to campus, working at least part-time, and incurring stress when trying to balance school and other activities (BCSSE Survey, 2023). Our undergraduate student population is shifting toward a more and more non-traditional student demographic.

During the pandemic, faculty also adopted online and hybrid teaching modalities and while many faculty saw benefits in adopting such course formats, concerns remained regarding the suitability an online format for STEM courses that required hands-on learning (Bashir et al., 2021). Faculty resistance to online STEM labs is primarily due to the lack of quality resources and the perception that students require traditional in-person, hands-on experiences to learn STEM laboratory concepts and techniques (Becker, 2017). However, students often prefer the flexibility and accessibility of online courses, including STEM labs (Means et al., 2013). Online courses also increase student access to course content and instruction, which can be particularly beneficial for non-traditional and underserved student populations (Means et al., 2013).

Considering the increased demand for accessibility and flexibility in course offerings from undergraduate students coupled with faculty resistance to adopt online and hybrid course modalities that provide accessibility and flexibility, there is a need to develop effective, high-quality solutions, especially for courses like STEM labs where traditionally, learning by doing is the primary teaching modality. Previous studies have explored the use of immersive online content, in the form of VR-based simulations, can effectively engage students and improve their learning outcomes and motivation compared to text-based learning (Merchant et al., 2014). Virtual reality has also been shown to increase student’s perception of the applicability of their learning to future careers (Makransky et al., 2019). To address the gap between faculty resistance to adopt online STEM labs and student demand for high-quality online instruction, this study aims to create a high-quality immersive VR-based STEM laboratory experience in an upper-level undergraduate neurophysiology STEM lab.

Research Methods

Course Description

Neurophysiology is a Course-Based Undergraduate Research Experience (CURE) where undergraduate students and dual enrolled high school students learn electrophysiology techniques using invertebrates. The course is structured with conceptual lectures on cellular neuroscience in combination with wet lab experiments. Students conduct experiments, collect and analyze data, and communicate their findings in the format of a scientific journal article.

Creation of Immersive 360 Videos

Course modules were planned using a blueprint of the course objectives aligned to activities and assignments. Based on the in-person laboratories and laboratory objectives, a storyboard was created for each recording. Storyboards included necessary 360 videos, 2D videos, questions, and answers that served as the production outline of the final immersive 360 video for a particular lab. A GoPro Max camera was used to create the 360 VR videos with narration of the neurophysiology laboratories. 2D recordings of electrophysiology data were captured using a screen recorder (Mediasite). Short introductory lectures and explanatory feedback were also recorded in 2D. After all necessary videos were recorded, Cenario VR was used to process the immersive 360 video, embedding 2D videos, guiding cues, probing questions, and just-in-time video feedback into the 360 recording of the laboratory (Figure 1). Immersive 360 videos were created for each laboratory and embedded into Canvas modules. Students were provided with 2-D lectures on content and raw data files for analysis. After completing the immersive online laboratory, students wrote a laboratory report in the format of a scientific journal article with an introduction, methods, results, discussion, and references sections which was graded using an adapted rubric (Bakshi, et al., 2016).

Figure 1. Elements of Immersive 360 Videos. Immersive 360 videos were created by embedding elements into the VR recording of the laboratory experiment. Embedded elements included 2D explanatory videos (A), guiding cues (B), probing questions (C), and just-in-time video feedback into the 360 recording of the laboratory (D).

Assessing the Effectiveness of Immersive 360 Videos

Student Perception of Learning Gains

Students took a post-course survey that asked a combination of Likert scale and open-ended questions on learning gains in the following areas: content knowledge, technical skills, and scientific communication to assess overall student learning gains. The overall research question for this study being “Is Immersive Online Content (IOC) Approach in the form of 360-virtual reality (VR) videos an effective method for generating high-quality online STEM labs?” This overall question was subdivided into three categories: Content Knowledge Learning Gains, Technical Skills Learning Gains, and Professional Application of Skills. Likert scale questions were set on a 6-point scale with 0 being the lowest score and 5 being the highest score.

Achievement Data on Scientific Writing

Students’ understanding of the concepts investigated in the course were assessed using a validated rubric (Bakshi, et al., 2016) that was adapted to the Neurophysiology curriculum that assess student achievement in the following areas: Introduction, Methods, Results, Discussion, References, and Overall Format. Student grades in the asynchronous online course were compared to the in-person Neurophysiology course to compare overall student achievement across the two course modalities.

Results

Assessment of Student Perception of Learning Gains

The immersive 360-videos had a positive impact on the student learning experience. When asked about their overall learning experience in the Neurophysiology course that incorporated virtual reality and immersive online content, 75% of students reported having a great to excellent experience. One key factor addressed in the post-course survey was the student’s perception of content knowledge learning gains to assess course content quality and effectiveness. Lectures cover nervous system function at a cellular level, including the biological basis of the electrical activity of neurons, including factors that affect neuronal function such as temperature, axon diameter, and the extracellular environment of the neuron. 87.5% of students reported that the use of virtual reality greatly enhanced or extremely enhanced their comprehension of complex electrophysiology concepts, and the same percentage reported that the immersive 360-VR experiences helped them to retain those concepts better than other forms of learning (Figure 2).

Neurophysiology is a lecture and laboratory course, with the acquisition of technical skills being a key component of the course. The survey asked questions to address to what extend student’s perceived improvement in technical skills, acquired through the VR-based online course, correlated with students' confidence in performing hands-on procedures as well as troubleshooting technical issues when compared to traditional learning methods. When asked how much their technical skills in electrophysiology have improved as a result of the VR-based online course, 75% of students reported great to excellent improvement.

Figure 2. Student Perception of Content Knowledge Learning Gains. This graph represents the percentage of students that reported great (4) to excellent (5) learning gains in the categories of Overall Learning Experience, VR Impact on Understanding, and Concept Retention.

When asked about their confidence in performing electrophysiological techniques in person after practicing the techniques in the VR environment, students reported the most confidence in elements that were repeated throughout the course like the use of equipment or the preparation and pinning of organisms. These elements were a part of the 5 immersive online labs developed. One student stated “I am highly confident that I can apply these techniques, especially in the areas of animal preparation and equipment setup. These were covered so thoroughly in the videos and reiterated throughout our assignments that I learned these techniques the most.” The same sentiment was repeated in student open responses with 100% of responses mentioning procedures that were repeated throughout the experiments were the skills they were the most confident in and 57% of students mentioned that they were most confident in these skills because they were reiterated throughout the labs.

Immersive 360 videos also included troubleshooting that naturally occurred during the recorded experiment. These experiments were conducted on live, fully anesthetized organisms and there is variability that occurs with each preparation, some of which requires troubleshooting to resolve issues. These instances were recorded as the experiment was conducted and included in the final immersive 360 video so students could better understand what types of issues arise when conducting electrophysiology experiments with both the animals and equipment. When asked about their confidence in troubleshooting technical issues related to electrophysiology equipment or procedures after completing the course, 37.5% of students reported great to excellent confidence, 50% reported moderate confidence, and 12.5% reported little confidence. “The 360 videos served as the time a lab administrator would come around and check up on your technique. It made me confident in talking about the methods and intro to the experiments, but it left me uncertain on troubleshooting.” Of all the student learning gains assessed in the survey, troubleshooting scored the lowest regarding student confidence.

Figure 3. Student Perception of Technical Skills Learning Gains. This graph represents the percentage of students that reported great (4) to excellent (5) learning gains in the categories of Technical Skill Development and Troubleshooting Proficiency.

Students did report they were likely to apply the skills they gained in the immersive online course to achieve their future academic and professional goals. When asked how likely they would be to use the technical skills you gained in this course in your future academic or professional pursuits, 50% of students reported a 4 and 50% of students reported a 5 (to a great extent). Students referenced application of skills to becoming a medical professional, scientific researcher, graduate degrees and future courses. When asked if students thought they would apply the skills they learned in future goals, one student stated “Yes, especially coming from an aspiring Doctor. For academic growth, it opens avenues for research, specialized coursework, and collaboration. Professionally, it's applicable in medical fields like cardiology and neurology, biomedical engineering, and pharmaceutical research for innovations in medical technology and drug development.”

Figure 4. Student Perception of Future Application of Learning Gains. This graph represents the percentage of students that reported application of their skills to a good (4) or great (5) extent to future professional and academic pursuits. Students referenced applications to their goals in the medical, research and academic fields.

When asked to reflect on the VR-interface usability, 100% of students that spoke of benefits mentioned the immersivity of the videos. “It is very immersive and gives you a great way to connect with what it is you're doing for the experiment.” “The clear benefit was how immersive it was, being able to interact was much more beneficial than just watching a recording.” While the immersive nature of the videos as a benefit was mentioned in 50% of all responses, 62.5% of all responses mentioned glitches in the 360-environment or lack of features like fast forward and rewind. “You have to start the video from the beginning so if anything happens during the 360 video, you have to start over from the beginning. This caused many students to have to rewatch videos.”

The importance of producing high-quality online course options for students was apparent when students were asked about why they chose the online course modality. 100% of students surveyed referenced the accessibility of the online format including scheduling conflicts with other courses or work or issues with commuting, where the online format provided them with the flexibility to be able to balance their other academic, professional and personal activities while still being able to access the course when it best fit their schedule. “I opted to take the course online to better fit my work schedule, I am a non-degree seeking student who works a part-time and full-time job. Being able to access work when I was free is the only reason I could take this course.” “I did not want to take this course online, but my schedule was tight. The reality is that science communication is 99% online, and will continue to be so, so adapting to this modality is extremely important. This format provided enough leeway for me to succeed in all my courses, and frankly, I think I learned the most this semester than I have in any other semester in my life. It just provided me with the convenience to manage my time in a schedule that suited me and was not prearranged to be at a certain time of the week.” The increased demand in online STEM courses is supported by the enrollment of the online section which had a total of 9 students enrolled compared to the in-person section that had 4 students enrolled with 1 student in the in-person section receiving an incomplete for the semester.

When comparing actual achievement in the form of grades between the in-person and online course modalities, there was no difference in student grades. The average overall grade for the in-person section was 89% compared to 90% in the online section. Interestingly, online students scored 5% higher on laboratory reports compared to the in-person section while the in-person students scored higher on the comprehensive final exam compared to the online students. The in-person section scored an average of 82% on their laboratory reports while the online section scored an average of 87% on their reports. This same trend was not reflected in final exam grades. The comprehensive final exam consisted of multiple choice, fill in the blank, and short answer questions that covered course content knowledge, methods and experimental setup, and data analysis topics. The in-person section scored an average of 95% on the final exam compared to 80% for the online section. The student receiving an incomplete for the semester was not included in the grade calculations.

Implications

The overall findings of this study demonstrate the importance of developing high-quality online STEM courses, especially STEM labs. The online course section had more than double the enrollment of the in-person section for this semester, demonstrating the student demand for online courses which was further supported by student statements on the accessibility and flexibility that the online course option offered, even for students that preferred to take in-person courses.

Students also valued the immersivity and interactivity of the 360-videos, which contributed to their perception of learning gains and confidence in technical skills building that they could apply to future professionals and academic goals as a result of taking the course. Interestingly, students were most confident in skills that were repeated throughout each laboratory of the course like animal preparation and use of equipment. These findings further support the model of spaced repetition in learning, where interventions are repeated in multiple instances over time (Lehcen, et. al., 2021). The skills students mentioned, animal anesthetization, pinning, equipment setup, etc. Are all essential skills of electrophysiology, which the students learned through repetition. Spaced repetition is an important factor to consider when developing high-quality immersive online courses in the future. This may also be why students felt less confident in troubleshooting as it was organically recorded over the course of the experiment and occurred at random. An additional module on troubleshooting or more immersive elements during instances of troubleshooting within the 360-enviroment could provide students with more interaction with these events and feedback on the troubleshooting process that could boost their confidence in this area.

There were differences in student achievement between in-person and online sections of the course. While in-person learners performed better on the comprehensive final exam, online students performed better on their laboratory reports. Overall performance on the final exam suggests that in-person learners retained more course content while the increased scientific writing output reflected in the online student performance could be attributed to the increased flexibility of the course. Students had more time in their schedules because they took the online course modality that they could then dedicate to writing better scientific reports.

One additional finding of this study shed light on areas of improvement. The program used to create the immersive 360-videos, CenarioVR, lacks some functionality with regard to fast-forwarding or rewinding the 360-video. If students miss something or want to review a part of the experiment, or if there is a glitch that forces them out of the immersive 360-video, there is no way to fast forward or rewind, forcing students to start over. This can be mitigated by breaking 360-video segments into short pieces; however, this segmentation could disrupt the logical flow of the experiment, making it harder for online students to mentally connect the steps of the experiment in order.

Faculty, especially those that teach STEM labs, have been historically resistant to the adoption of online courses, however, this does not match the increased demand by students for high-quality online instruction. The use of immersive 360-videos leveraged existing technology to make faculty-faculty-created courses more immersive and engaging for students without compromising student comprehension or skill development. “The greatest impact this class has had on my academic career is illustrating axons' electrophysiology via 360-videos and NIA simulations. These resources helped me visualize and interact with axons in a way that facilitated learning and constant review of electrophysiology that I did not believe was feasible in an online environment.”

References