Teaching with Virtual Humans: Dealing with A Self Representation in the Virtual Environment

Get Complete Project Material File(s) Now! »

Immersion and Presence Factors in Virtual Environments

Immersion and presence are two terms currently used when talking about virtual reality. Even if related, these two terms should not be confused. Immersion is about the technology. Vianin de-fined it as the technical interface between the man and the machine not concerning psychological states of the user [Vianin1995]. Eﬀectively, immersion concerns the interaction of the subject with the virtual environment. These interactions are possible throughout sensorial information (vision, audition, smell, kinesthetic, . . . ). These interactions, necessary for the human-machine coupling are created throughout behavioral interfaces. These interfaces are divided in two cate-gories, motor interfaces (motion capture, sensors, computer vision, . . . ) and sensorial interfaces (referring to the five senses, like screen and 3D glasses, for example). Fuchs et Al. [Fuchs2006] proposed a third category concerning sensorimotor interfaces like, for example, force feedback interfaces. The quality of these interfaces will influence the sensation of immersion in the virtual environment of the subject. What diﬀerentiates immersion from presence, that will be defined next, is that in immersion the psychological aﬀect of the subject is not involve.
On the other hand, presence can be defined as the psychological sensation of being in an environment created using immersion technologies. That sensation of presence is not equally induced for all user of a same immersive environment [Slater1993]. That definition was subject to many variations. Barfield and Weghorst [Barfield1993] talk about virtual presence and defined it as a subjective and hypothetical conscious state of being implied in a non-present environment. Steuer [Steuer1992] instead talk about « telepresence » and defined it as the experience of a presence in an environment mediated by a communication mean. In that case, presence refer to natural perception of an environment as telepresence refer the perception of a mediated environment.
The sensation of presence is not unique to virtual environments. According to Hendrix [Hendrix1994], presence can also appear when reading a book, watching a movie or a theater act. Presence is therefore not something new. However, a high degree of presence is something important to reach in virtual reality applications.

Presence Factors

Slater and Usoh [Slater1993] classified the presence factors in two categories. The first category concerns the external factor and is closely link with the immersion technology used to create the virtual environment. The second category concerns the user internal state.

External Factors

The external factors are related to the material used to create the virtual environment. According to Steuer [Steuer1992] there is five important points to consider:
• the quality and richness of the sensorial information transmitted by the environment;
• the coherence of the virtual environment;
• the degree of interaction between the subject and the environment;
• the fidelity of the reproduction of the subject virtual body (avatar);
• the latency of the system (as close to zero as possible) between the subject action and the reaction cause in the virtual environment.

Internal Factors

The internal factors are related to the internalization processes of the users. These factors con-sequently largely varied from one subject to another [Psotka1995]. That finding was confirmed by a study by Hodgins et Al. [Hodgins1998] analyzing the influence of geometric models on perceptions. That study showed that for some users the action felt more realistic on a textured polygonal model than on a stick figure model as other users did not perceived any diﬀerence.
The level of presence is therefore highly dependent on the individual, especially if the graphical quality of the environment is low. The presence level depends on the subject implication and on his capacity to supplement the virtual environment with his own representation of the world. However, it seems possible to study presence on a population and to draw general conclusion about the capacity of a virtual environment to induce presence.

Presence Measure Methods

Presence is a multidimensional complex phenomena and it is consequently not possible to mea-sure it using only one kind of measures. Furthermore, subjective measures are not always reliable and pose a validity problem [Barfield1993]. The notion of presence itself is not perfectly defined. However, many methodologies to measure it have been created, and Hendrix divided these into objective and subjective measures [Hendrix1994].

Objective Measures

Regarding the objective measures, Barfield and Weghorts [Barfield1993] outlined some cate-gories:
• Physiologic indicators: muscular tension, cardiovascular and ocular response, skin conduc-tion, . .
• Physiometric indicators: cortical responses, pupil diameter, . . .
• Performance indicators: precision, speed, . . . [Slater1996]
• Conflict resolution indicators: the capacity of the subject to adapt to the conflictual situations generated by the presence of two environments, the physical world and the virtual environment.
• Subject disorientation degree indicators: time to re-adapt to the physical world.

Subjective Measures

Subjective measures mostly rely on questionnaires to evaluate the degree of presence of an individual in a virtual environment [Witmer1998, Usoh2000, Slater1999]. Questionnaires alone cannot measure all factors influencing presence, it is therefore important to combine subjective and objective measures [Hendrix1994]. Furthermore, Slater et Al. have point out the diﬃculty of formalizing questions to assess presence of the virtual experience of a participants [Slater2007] [Slater2004].

Presence and Performance

The study of the relationship between presence and performance has major impact in virtual reality. Knowing the presence factors that can possibly improved the performance of a task execution in teleoperation has a direct impact on the conception of these environments. To establish the link between performance and presence, it is necessary to identify a list of behavioral interfaces that improved the performance of a specific task [Draper1996]. The problem is not to know if presence influences the performance, but to determine the combination of media required to induce a reaction similar to that that would be induced in real life situation [Slater1996]. If that combination is known it is be possible to train individual in virtual reality to prepare them to be performant in real life situations. For example, to ameliorate the eﬃciency of a surgeon in the operation room [Freysinger2002]. However, that is not an absolute rule. In some cases, a high level of presence might induce a dependance to the virtual environment to perform the task. Situations were transfer from one environment to another are required should consequently be carefully study. Sections

resents more details about that.

However, establishing the level of performance in virtual environments, gives information about the level of presence. Consequently, if an individual correctly performed a task in virtual reality, that implies that his presence and implication level were suﬃcient [Slater1999]. In fact, the definition of presence (being there) implies « being able to act there » [Sanchez-Vives2005].
Furthermore, performance and presence can also be related in another manner. The eﬃciency of a simulation can be evaluated by its degree of fidelity [Stroﬀregen2002]. That degree of fidelity being defined by the diﬀerence between the subject behavior in the virtual environment and the real world situation. System fidelity can therefore be divided in two categories: the fidelity of the experience, and the fidelity of the action performed in the virtual environment. The first category refers to the notion of presence. The second category is a measure of the functional fidelity. Functional fidelity is considered high if action performed in the virtual environment is close to that that would be performed in the real world [Morice2008].
The notion of performance is extremely important in the domain of sports. Bideau et Al. [Bideau2003, Bideau2004a] and later Vignais et Al. [Vignais2009a, Vignais2010] have studied the reaction of handball goalkeeper to virtual throwers in order to establish if the level of performance was the same in virtual reality than in a real situation. They were able to establish that the level of performance, and consequently of presence, was suﬃcient to use virtual reality as a study tool for the visual information retrieval activity of an handball goalkeeper.

Body Ownership

Body ownership can be seen as special cases of presence, where the presence to evaluate is that of the subject in a virtual human present in the virtual environment. That virtual human is a projection of the subject in the virtual environment and can be seen either in first person view, normally throughout the usage of and HMD where one can look at himself (a virtual self), or in third person view, often by being represented in the virtual environment throughout the usage of a mirror or shadow metaphor.
Body ownership is a known phenomena outside the virtual reality context and is often illustrated by the rubber hand illusion [Tsakiris2005]. An illusion where a subject places his arms on a table, one of his arm is hidden and replaced by a rubber arm. The subject sees the rubber hand being touch by a feather while his real arm is also touched by a feather. After a certain time, if the rubber arm is threaten, the subject demonstrates the same physiological and physical reactions as if his real arm was threaten.
The same experiment has been done in virtual reality with a virtual arm [Sanchez-Vives2010, Yuan2010]. Moreover, it has also been demonstrated on the whole body in an experiment where male subjects see themselves in first person view in the body of a female character [Slater2010] and in third person view throughout the usage of a virtual mirror [González-Franco2010]. The protocols used in the first study is similar to that of the rubber hand illusion. In the case of the mirror experiment, synchronous and asynchronous response of the mirrored avatar to the subject action are used as a stimulus for the body ownership illusion to operate. In both case, the virtual body is then threatened to induce a reaction.
In these experiment, ownership is assessed using the same method as for other type of presence, namely throughout physiological measures, questionnaires, and interviews. The illusion of body or limbs ownership in virtual reality is used, for example, in the treatment of chronic pain [Hoﬀman2000] and for phantom limbs treatment [Murray2007, Mercier2009].

READ Socialisation of the law of contract in judicial decision making

Simulation of Virtual Humans

The most frequently used method of teaching in sports is teaching by demonstration [Desmurget2006]. That method implies a human model on which the learner take the necessary information to build his own representation of an action or gesture. The information the learner retrieves are mostly visual information obtained by the observation of the model combined with audio infor-mation provided by the discourse and the sounds (breathing, clapping, hitting, sounds due to displacement, etc. . . .) accompanying the demonstration. In the case of a virtual learning envi-ronment the model is a virtual human. The level of fidelity of the rendering of that information is therefore crucial. However, the actual technologies do not permit to model all the variables implied in human actions. Simplifications and choices must consequently be made with the risk of degrading and even altering the rendering of the information required by the learner.

Table of contents :

Introduction
1 State of the Art
1.1 The Usage of Virtual Reality in Sports Teaching and Training
1.1.1 A Brief History of Virtual Reality: From Simulators to Virtual Environments
1.1.1.1 Virtual Reality Definition
1.1.1.1.1 Technological Definition
1.1.1.1.2 Conceptual Definition
1.1.2 Immersion and Presence Factors in Virtual Environments
1.1.2.1 Presence Factors
1.1.2.1.1 External Factors
1.1.2.1.2 Internal Factors
1.1.2.2 Presence Measure Methods
1.1.2.2.1 Objective Measures
1.1.2.2.2 Subjective Measures
1.1.2.3 Presence and Performance
1.1.2.4 Body Ownership
1.1.3 Simulation of Virtual Humans
1.1.3.1 Human Body Representation
1.1.3.2 Morphological Adaptation
1.1.3.3 Motion Representation
1.1.3.3.1 Frame By Frame Representation
1.1.3.3.2 Key-Framed Based Representation
1.1.3.3.3 Signal Processing
1.1.3.4 Motion Production
1.1.3.4.1 Motion Capture Systems
1.1.3.4.2 Motion Capture Data Treatment
1.1.3.5 Section Conclusions
1.1.4 Interacting with Virtual Humans
1.1.4.1 Perception of Virtual Humans
1.1.5 Sports in Virtual Reality: A Study Tool
1.1.6 Teaching and Training Motor Skills in Virtual Environments
1.1.6.1 Video Games
1.1.6.2 Exertion Interfaces
1.1.6.3 Enactive Training Accelerators
1.1.6.4 Teaching by demonstration Using Virtual Humans
2 General Methodology
2.1 Building a Virtual Environment to Evaluate the Relevance of Using Virtual Human for Teaching Motor Skills
2.1.1 The Choice of the Gestures
2.1.1.1 Karate: A Martial Art
2.1.1.2 Three Isolated Basic Gestures
2.1.1.2.1 Tsuki: A Frontal Punch
2.1.1.2.2 Mae geri: A Frontal Kick
2.1.1.2.3 Soto uke: A Frontal Defense
2.1.2 The Design of the lesson
2.1.2.1 The Choice of the Teaching Method
2.1.2.2 The Different Moments of the Lesson
2.1.2.2.1 Video Warmup
2.1.2.2.2 Lesson
2.1.3 The Design of the Virtual Environment
2.1.3.1 Virtual Dojo
2.1.3.2 Virtual Trainer
2.1.3.2.1 Graphical Design
2.1.3.2.2 Motion Capture
2.1.3.2.3 Motion Retargeting on the Virtual Instructor .
2.1.3.2.4 The Virtual Instructor Voice
2.1.3.3 Technological Environment
2.1.3.3.1 Projected Virtual Environment
2.1.3.3.2 Artificial Stereoscopic Vision
2.1.3.3.3 Head Tracking For Egocentric Vision
2.1.4 Studies Participants’ Profiles
3 Teaching with Virtual Humans: A Comparative Study of Teaching in Traditional, Video, and Virtual Environments
3.1 Introduction
3.2 Methodology
3.2.1 Three Different Learning Environments
3.2.1.1 Learning in a Traditional Class with an Instructor
3.2.1.2 Learning from a Video of the Instructor
3.2.1.3 Learning from a Virtual Human Representation of an Instructor
3.2.2 Evaluation by an Expert
3.2.2.1 Initial Evaluation
3.2.2.2 Final Evaluation
3.3 Results
3.4 Conclusion
4 Teaching with Virtual Humans: Dealing with A Self Representation in the Virtual Environment
4.1 Introduction
4.2 Methodology
4.2.1 Mirror Feedback
4.2.1.1 Real-Time Full Body Motion Tracking and Mapping
4.2.2 New Evaluation Methods
4.2.2.1 Refinement of the Expert’s Evaluation Grid
4.2.2.1.1 Multi-Evaluators Comparison
4.2.2.1.2 Validity of the Grading Scale
4.2.2.2 Evocation Interview
4.2.2.2.1 Interviews Methodology
4.3 Results
4.3.1 Evaluation by Experts Results
4.3.1.1 Tsuki
4.3.1.2 Mae Geri
4.3.1.3 Soto Uke
4.3.1.4 Comparison with Experts’ Scores
4.3.2 Evocation Interview Results
4.3.2.1 Without Mirror Group Results
4.3.2.1.1 First Training Interview Synthesis
4.3.2.1.2 Third Training Interview Synthesis
4.3.2.1.3 Individual Learning Summary
4.3.2.1.4 Group Conclusions
4.3.2.2 Mirror Group Results
4.3.2.2.1 First Training Interview Synthesis
4.3.2.2.2 Third Training Interview Synthesis
4.3.2.2.3 Individual Learning Summary
4.3.2.2.4 Group Conclusions
4.3.2.3 Interviews Conclusions
4.3.2.3.1 The Role of the Indicators of the Learning Environment
4.3.2.3.2 Participants’ Learning Process Management .
4.4 Discussion
4.5 Conclusion
5 Standardization of the Performance Evaluation: The Necessity for an Automatic Evaluator
5.1 Introduction
5.2 Methodology
5.2.0.4 Motion Capture of the Evaluations
5.2.0.5 Motion Retargeting of the Evaluations
5.2.0.6 Motion Segmentation in Trials
5.2.0.7 The Choice of the Instructor Reference Trials
5.2.0.8 Automatic Evaluation Based on Biomechanical Parameters .
5.2.0.8.1 Dynamic Time Warping
5.2.0.9 Statistical Analysis of the Distance Matrix
5.2.0.9.1 Distance from the Instructor Reference Trials .
5.2.0.9.2 Intra-subject Variability
5.2.0.9.3 Intra-subject Symmetry
5.2.0.10 Testing of the Automatic Method Evaluator
5.2.0.10.1 Distance From a Panel of Experts
5.2.0.10.2 Distance Between Two Captures of the Instructor .
5.2.1 Results
5.2.1.1 Tsuki
5.2.1.2 Mae Geri
5.2.1.3 Soto Uke
5.2.1.4 Comparison with the Evaluation of Expert Level Subjects
5.2.1.4.1 Distance Between Two Captures of the Instructor .
5.3 Discussion
5.4 Conclusion
6 General Discussion
6.1 Summary of the Studies
6.2 Future Work: Toward a Virtual Coach
6.2.1 Toward an Automatic Evaluator
6.2.2 Toward a more Plausible Virtual Learning Environment
Conclusions and Perspectives
Contributions
References
List of Figures
List of Tables
7 Appendix
7.1 First Study Evaluation Grids
7.1.1 Tsuki Evaluation Grid in French
7.1.2 Tsuki Evaluation Grid in English
7.1.3 Mae Geri Evaluation Grid in French
7.1.4 Mae Geri Evaluation Grid in English
7.1.5 Soto Uke Evaluation Grid in French
7.1.6 Soto Uke Evaluation Grid in English
7.2 Second Study Expert’s Evaluation Grids
7.2.1 Tsuki Evaluation Grid in French
7.2.2 Tsuki Evaluation Grid in English
7.2.3 Mae Geri Evaluation Grid in French
7.2.4 Mae Geri Evaluation Grid in English
7.2.5 Soto Uke Evaluation Grid in French
7.2.6 Soto Uke Evaluation Grid in English
7.3 Second Study Evaluation by Experts Results
7.3.1 Tsuki Evaluation by Experts Results
7.3.2 Mae Geri Evaluations by Experts Results
7.3.3 Soto Uke Evaluations by Experts Results
7.4 Second Study Evocation Interview Results
7.4.1 Without Mirror Group
7.4.1.1 First Training Session
7.4.1.2 Second Training Session
7.4.2 Mirror Group
7.4.2.1 First Training Session
7.4.2.2 Second Training Session
Publications
7.5 Thesis Related Publications
7.6 Other Publications