Virtual Reality And Spatial Cognition Research Paper

In this research paper, we give an overview of applications of virtual reality in behavioral experiments on human cognition. We focus on spatial cognition, i.e., navigation and spatial memory.

1. Applications

The very first application of virtual reality was the training of air force pilots in flight simulators. The first computer-graphics based flight simulator was built in the 1970s. By the 1980s, with better software and hardware, the market for virtual reality technology had grown far beyond military use. As we enter the twenty-first century, virtual reality is a standard method for manipulating the action–perception cycle underlying normal behavior in all autonomous agents.

Contemporary virtual reality systems are widely used. Examples include:

1.1 Visualization

Visualization is of high importance for interpreting data for many technical or scientific applications. An excellent overview is given in van-Dam et al. (2000). Examples come from the fields of art, medicine, architecture, and archaeology. For example, members of the Computer Science Department at Brown University have been collaborating with archaeologists to develop the Archave system which makes it possible to visualize the results of excavations. On the basis of this visualization, further analysis of the data is possible. Another example comes from the field of car design. In virtual realities car-designers and engineers are not only capable of presenting future models but can also examine functional aspects. For example in a packaging simulation, one can study the assembly of an engine or the possibility to dismantle single pieces of the engine for repair. Furthermore, crash tests can be observed from unusual perspectives: One can place a virtual camera right in the middle of the engine during a crash test.

1.2 Training And Education

There are many applications of virtual reality to training and education, especially in the field of medicine. For example, many practising surgeons completed their training before the so-called laparoscopic era of the 1990s. Laparoscopic or minimal invasive surgery has dramatically changed the way surgery is performed, which makes it necessary to teach even very experienced physicians. For this reason, there are many training simulations in the field of endoscopic surgery, for example of the knee (Muller et al. 2000). In this simulation students can act on virtual reality tendons and bones with realistic instruments.

A further application is the various driving simulators—for example, for aeroplanes, helicopters, motorcycles, trucks, and even wheelchairs. In the case of aeroplanes and helicopters, the purpose of training in a flight simulator is not to substitute for the training in the real world completely but to familiarize students, for example, with the handling of the cockpit, instrument displays, navigational tasks, or modifications of existing instruments.

1.3 Behavioral Experiments

Computer graphics has been used for many years in the generation of experimental stimuli to examine the visual system. With virtual reality, the sensory system and related behavior can be investigated, that is, it is possible to examine a whole action–perception cycle. If one focuses on perception alone, it is always hard to decide which feature of the measured sensor performance is responsible for a certain behavior. The following are easily measurable behaviors in virtual worlds: reaction-time measures, pointing direction, grasping behavior, eye movements, or navigation ability. For the experimenter, the advantage of the virtual world compared to the real world lies in the possibility of controlling the environment very strongly, in the flexibility and ease with which different environments can be constructed, and in the ease with which experimental data can be saved and visualized.

2. Spatial Cognition: Experiments In Virtual Reality

Many fruitful experiments in the field of spatial cognition have been performed with virtual reality. A comprehensive overview of the advantages and disadvantages of performing navigation experiments in a virtual world is given in Peruch et al. (1998). On the one hand, it is possible to build up totally new environments, which are unknown to subjects. Thus unbiased measurements of navigational ability are obtained. Mallot et al. (1998) describes a maze—like city called ‘Hexatown,’ where each junction is a connection of three streets, resulting in a hexagonal structure (Fig. 1). The regularity of the ground plane has the advantage that the structure of the city does not give the subject any cues in a navigational task. During the experiments the subjects sit in front of a computer screen and move through the town by pressing the buttons of a computer mouse. The results suggest that the subjects acquire a cognitive map during exploration even in the absence of physical motion. On the other hand, it is possible to reconstruct existing cities or buildings to investigate in virtual reality the spatial memory acquired in real environments or vice versa (Fig. 2).

Witmer et al. (1996) investigated the transfer of knowledge based on training in a virtual world to knowledge acquired in the real world. They concluded from their experiments that the resulting configuration knowledge is very similar.

As mentioned previously, one advantage of experiments in a virtual world is the high flexibility and controllability they provide. In older navigational studies the experimenters prepared videofilms or slide presentations in order to control the environment that the subjects had to explore. In other studies model towns were built using toy houses and the perspectives from within the town were caught in a camera drive. This latter method resembles the fashion in which virtual worlds are built up today. With software very similar to the computer-aided design (CAD) programs used by architects, the surroundings can be modeled. The interaction with a user resembles a camera drive moving through the model. Using this type of software it is much easier to make subtle changes. For example, it is possible to move objects during the experiments (Aginsky et al. 1997), to change the lighting conditions (Mallot et al. 1998), or to change the topography of the environment, for instance the slant (Mallot et al. 2000).

One set of navigational experiments has focused on the question of the comparability of the results obtained in the virtual world to results obtained in the real world. Ruddle and colleagues (1997) replicated a distance-estimation and pointing task in a virtual world that was originally carried out by Thorndyke and Hayes-Roth (1982) in the real world. The results turned out to be very similar. However, Peruch et al. (1997) found in a path-integration task that subjects in a virtual world made a different systematic error compared to subjects in the real world (Loomis et al. 1993). The latter tend to compress the range of actually produced turns and distances in such a way that large values are underestimated and small values are overestimated. Subjects in the virtual world underestimated all values. One has to keep in mind that the sensory input in the virtual world is different from the input in the real world. For instance, in the Loomis et al. (1993) study the subjects were blindfolded during the experiments, that is, perception was based on vestibular and proprioceptive input alone. On the other hand, in the Peruch et al. (1997) study the behavior was based only on visual input.

Recently, there have been efforts to combine the visual input in the navigation experiments with adequate proprioceptive and vestibular stimuli. Most of the studies carried out so far have taken place in ‘desktop virtual reality,’ where subjects sit in front of a computer screen or a projection wall and press the buttons of a computer mouse or move a joy stick. By doing this, they navigate through the virtual world. In a study of Chance et al. (1998) subjects performed a triangle-completion task in which they walked two sides of a triangle and were then requested to return to the starting point, completing the third side. During walking, the visual input was presented on a head-mounted display. The performance was better than in ‘just turn’ group. The latter could only physically turn and carried out translatory movements in the virtual world.

A very interesting paradigm is the combination of virtual reality with functional imaging studies (PET or fMRI) to identify regions in the brain involved in spatial cognition. One exciting finding is that there is an activation of the hippocampus proper (besides several other regions in the brain), which has already been found in animal experiments. In studies of rat hippocampal pyramidal cells, so-called ‘place cells’ have been identified, that fire when the animal is in a specific location (O’Keefe and Dostrovsky 1971). In similar fashion, Maguire et al. (1998) found in a PET study with human subjects that the right hippocampal formation shows increased activity during exploration of a virtual town.

3. Simulation Sickness

A major problem with virtual reality is simulation sickness, which was initially documented in a helicopter flight simulator. The most common symptoms are: general discomfort, sweating, disorientation, nausea, and vomiting. Some people exhibit all of these signs whereas others exhibit only a few or even none. In an analysis of data from 10 US Navy and Marine Corps flight simulators, it was found that approximately 20 to 40 percent of the pilots experienced at least one of the symptoms (Kennedy et al. 1989). The main theory explaining simulation sickness is the ‘Cue Conflict Theory,’ which implies that the main causes are discrepancies between the senses concerning information about body orientation and motion.

Bibliography:

1. Aginsky V, Harris C, Rensick R, Beusmans J 1997 Two strategies for learning a route in a driving simulator. Journal of Environmental Psychology 17: 317–31
2. Chance S S, Gaunet F, Beall A C, Loomis J M 1998 Locomotion mode affects the updating of objects encountered during travel: The contribution of vestibular and proprioceptive inputs to path integration. Presence 7(2): 168–78
3. Kennedy R S, Lilienthal M G, Berbaum K S, Baltyley D R, McCauley M E 1989 Simulator sickness in US Navy flight simulators. Aviation, Space and Environmental Medicine 60(1): 10–16
4. Loomis J M, Klatzky R L, Golledge R G, Cicinelli J G, Pellegrino J W, Fry P A 1993 Nonvisual navigation by blind and sighted: Assessment of path integration ability. Journal of Experimental Psychology General 122(1): 73–91
5. Maguire E A, Burgess N, Donnet J G, Frackowiak R S I, Frith C D 1998 Knowing where and getting there: A human navigation network. Science 280: 921–4
6. Mallot H A, Gillner S, vanVeen H A H C, Bulthoff H H 1998 Behavioral experiments in spatial cognition using virtual reality. In: Freksa C (ed.) Spatial Cognition: An Inter-disciplinary Approach to Representing and Processing Spatial Knowledge. Springer, Berlin, pp. 447–67
7. Mallot H A, Mochnatzki H F, Steck S D 2000 Geographic slant improves navigation performance in virtual environments. Investigative Ophthalmology & Visual Science, Suppl. 41(4): 44
8. Muller W, Bockholt U, Lahmer A, Voss G, Borner M 2000 VRATS – Virtual reality arthroscopy training simulator. Radiologe 40(3): 290–4
9. O’Keefe J, Dostrovsky J 1971 The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely moving rat. Brain Research 34: 171–5
10. Peruch P, May M, Wartenberg F 1997 Homing in virtual environments: Effects of field of view and path layout. Perception 26: 301–11
11. Peruch P, Gaunet F 1998 Virtual environments as a promising tool for investigating human spatial cognition. Current Psychology of Cognition 17(4–5): 881–99
12. Ruddle R A, Payne S J, Jones D M 1997 Navigating buildings in ‘desktop’ virtual environments: Experimental investigations using extended navigational experience. Journal of Experimental Psychology: Applied 3(2): 143–59
13. Thorndyke P W, Hayes-Roth B 1982 Differences in spatial knowledge acquired from maps and navigation. Cognitive Psychology 14: 560–89
14. vanDam A, Forsberg A S, Laidlaw D H, LaViola J L, Simpson R M 2000 Immersive VR for scientific visualization: A progress report. IEEE Computer Graphics 20(14): 26–52
15. Witmer B G, Bailey J H, Knerr B W, Parsons K C 1996 Virtual spaces and real world places. Transfer of route knowledge. International Journal of Human–Computer Studies 45: 413–28