Project topics and materials
Get Complete Project Material File(s) Now! »
Virtual environments, mixed reality, virtual reality
In the literature, many terms are used to describe and define the different virtual environment technologies. All these technologies aim to provide to their users a virtual environment where they can visualise and interact with some content, which can be either virtual and/or real. To describe such VE technologies in a generic way, the term mixed reality can be used since it encompasses the definitions of all the kinds of environments that provide to their users a ”reality”, which is in fact an alternative to our physical world. It means that this term includes the concept of virtual reality – which is close to full virtuality, the one of augmented reality – which is closer to the reality, and all the other intermediate possibilities, such as augmented virtuality [139]. To sum up, MR technologies allow for the creation of environments that combine both real elements from our physical world with fictive elements from virtual worlds, regardless of the relative proportion of each of these two kinds of elements in the environment [105] [104]. Figure 2.1 shows this concept called virtuality continuum.
Figure 2.1: The reality–virtuality continuum – Milgram et al. courtesy [105]
MR technologies aim to put in relationship a virtual environment and our real world, through some input and output systems that are provided to the users. These systems allow for the establishment of communications and of actions between the two kinds of environments and the users. In one of their study, Burdea et al. [31] represented explicitly this fact, as shown in Figure 2.2. Indeed, in this figure the arrows represent both these relationships and the interactive workflow of communication between the users and the virtual reality technology, notably when they need to perform a task within this VR environment. In this regard, Rubio-Tamayo et al. highlighted an essential aspect about this relationship: this communication between the users and the virtual and real environments, through input and output systems, is mostly based on our senses [144]. This explains why our visual perception is particularly important and essential for us in VR experiences; nonetheless, other human senses can be used in VR, such as our touch or motion sense through haptic devices, as shown in Figure 2.3.
Figure 2.2: A VR system: real world, virtual world and users interacting through its input and output devices – Burdea et al. courtesy [31]
Figure 2.3: Virtual environment experience is a sensorial experience – Rubio-Tamayo et al. courtesy [144]
Virtual environments are characterised by the choices made about the content they display and its associated visualisations, and the interactions they offered to their users [29]. To clarify what these two main elements of a virtual environment are, these definitions can be used:
• The VE content is the content that composes the VE itself: this is usually some 3D visual content such as 3D models, but it can also be some audio content for example. This content can be created by designers or be issued from domain-specific field data.
• The VE interactions are the interactions that the users have with and within the VE. They are possible thanks to VR interaction techniques, which are the ways or the interaction techniques built by the VR developers to provide to the users these capacities to interact.
Bryson et al. called these visualisations choices ”the overall environment and the information presentation metaphors” choices, and called them for the interaction ”the interaction metaphors choices” [29]. They also noticed that these two main characteristics – visualisations and interactions – that are employed to describe a virtual environment could also be used when thinking about the device requirements or the device performance constraints since they are usually visual display constraints and interactivity constraints. About the design of the virtual environments, both human and device characteristics may have some influence on it. This can be explained since VEs are in fact advanced human-computer interfaces. Thus, as Hettinger et al. noticed, virtual environments should be designed thinking on the future users and their aims – e.g. following a user-centred design – since VE main purpose is to improve and extend human abilities [72].
Virtual reality can offer a lot to its users, notably through an improvement of their imagination, due to a virtual environment that improves their interaction capabilities, and provides a deeper sense of immersion. Indeed, Fuchs et al. described the interaction and the immersion as ”the two keywords of virtual reality” [56], and, more recently, Mutterlein added a third ”i”, with the concept of the imagination of the VR users, and called these three ”i’s” the three pillars of virtual reality [113].
Figure 2.4 shows this principle about virtual reality. Moreover, as these ”i’s” are what make VR much more than a simulation according to Zyda et al. [198], the use of VE technologies should also improve the users’ performance on their expert tasks. Alshaer et al. showed that the virtual environments can improve the spatial abilities of their users, and more precisely in their study the spatial memory and navigation abilities during an exploratory walkthrough task [3]. It has been shown that these kinds of abilities may be essential for performing some tasks, and that VR allows the users to perform these tasks better, inclusively in many different contexts such the aeronautic [97] or the aircraft industries [123].
Figure 2.4: The three i’s of virtual reality, adapted from Fuchs et al. [56] and Mutterlein [113]
For these reasons, mixed reality technologies are currently widely used in different kinds of industries.
They have also been used for many different purposes, such as for example in the mining industry for training as Wyk and De Villiers noted it [175]. Virtual reality can also be used to perform training sessions for complex tasks such as assembly tasks and industrial maintenance [58] [118]. Nonetheless, it would be reductive to think that VR can only be used for training. Indeed, in such environments, users’ behaviours can be evaluated for other purposes and tasks. For example, user experiments can be conducted when VR users must validate the design of industrial workstations, to evaluate their collaboration on a designing task [63] [126]. In that sense, design or planning tasks are usually good candidates for virtual reality since they may require coordination and collaboration between several experts, which are two concepts that are enhanced in VR [103] [92]. In particular, some VR techniques enhance this collaboration by providing multiple representations of objects, and by showing a different one to each user according to their own expertise, notably in CAVE devices [93]. To conclude, mixed reality technologies can be used for diverse purpose in different kinds of industries, and in particular for performing design tasks, either individually or in collaboration.
Finally, about the design tasks performed in VR, one major challenge is to reflect the decisionmaking choices taken during the VR sessions on the design model that is used by the experts on their domain-specific tools. To address this issue, in the context of the AEC industry, Du et al. presented a zero-latency environment, based on AEC software and a VR application, to assist design decision- making about a building [43]. Their virtual environment improved both the collaboration and the individual user task performance of the AEC stakeholders involved in this design task, by allowing for earlier decision-making, thanks to the VR technology, and thanks to AEC software, which are presented in the next section.
Building Information Modelling usages and benefits
In the AEC industry, the Building Information Modelling is a methodology that has been more and more used since the last decades [147]. This methodology and its related software, specific to the AEC industry, have helped the different stakeholders of building projects for completing some of their tasks. One of the main purposes of BIM is to centralise all the data about a building from its conception to its renovation or its recycling. For this purpose, BIM allows all the different stakeholders to share and to participate in a common digital model. Thus, to enhance the collaboration, BIM tools allow for the consultation and modification of the building models, the costs supervision documents, and the planning of the construction. Nonetheless, BIM is more than a simple database and also relies on software that allow for the design of building models thanks to 2D plans and 3D virtual mock-ups. These models can be for example structural, architectural or mechanical, electrical and plumbing models, and are usually designed in BIM authoring tools such as Autodesk Revit or Graphisoft Archicad.
All these models and also the data from other sources – e.g. simulations, text documents, videos or point clouds from the field – contribute to the global digital model. Figures 2.5, 2.6 and 2.7 show some different representations of BIM data to understand the forms that BIM can take and what BIM can be: 3D design models, 2D design models, and 3D models with point clouds from the real site, in these examples from Johansson et al. [84], Costa et al. [37] and Mill et al. [106].
Figure 2.5: 3D mock-ups used in BIM tools – Johansson et al. courtesy [84]
Figure 2.6: 2D models to create the building design – Costa et al. courtesy [37]
Figure 2.7: A point cloud and a 3D model of an existing building – Mill et al. courtesy [106]
By extension, this BIM global model, which centralises all the building data, may be considered as a digital twin of the real structure. A digital twin may be defined as ”a digital asset [i.e. a data set that describe an asset that is not necessarily physical] on which services [i.e. repeatable business activities that have a specified outcome] can be performed that provide value to an organization” – ISO norm ISO/TS 18101-1:2019. This point is actually debated in the literature, and accepted or not depending on the studies. As for Khajavi et al. [87], a BIM model is not a digital twin and would only become it if the data collected thanks to the Internet of Things technologies (IoT) and the information coming from machine learning computation could be considered as included in the BIM methodology – and as for them, they cannot be considered as such. They specified that, as for them, a digital twin model is a BIM model enriched with these kinds of data – IoT data [87]. Volk et al. also expressed this possible ambiguity about the understanding of BIM scope, which can variate [178], as shown in Figure 2.8.
In any cases, the different studies in the literature about BIM agreed that its use does allow for a better follow-up of the construction steps and has actually improved the work of the building projects stakeholders, spreading the idea that BIM has benefits and that these benefits can be evaluated, to show how it can help the AEC professionals.
Table of contents :
Abstract
1 Introduction
1.1 Architecture, Construction and Engineering and Building Information Modelling
1.2 Computer science technologies and virtual reality in the AEC industry
1.3 Issues for designing and evaluating a virtual environment and its user interactions
1.4 Description of our contribution about the designing of VEs
1.5 Document outline
2 State of the art
2.1 Virtual environments, mixed reality, virtual reality
2.2 Building Information Modelling usages and benefits
2.3 Benefits of using BIM-based virtual environments for AEC tasks
2.4 Ambiguity about the ”design evaluations” noun group in the AEC literature
2.5 Usability evaluation on the design of a virtual environment
2.6 Required analyses and studies to design appropriately VE interactions
2.6.1 Explanations on the importance of the design of VE interactions
2.6.2 User task analysis
2.6.3 VE interaction techniques study
2.7 User-centred design methodologies for VE interactions
2.8 Conclusions
3 Experiments on user-centred design methodologies with AEC case studies
3.1 Objectives
3.2 Worksite construction supervision case study
3.2.1 Description of the case study
3.2.2 Participants
3.2.3 User task analysis
3.2.4 Expert study, guidelines, and prototyping
3.2.5 User evaluation
3.2.5.1 Experimental protocol and results
3.2.5.2 Discussion
3.3 Clash resolution during design coordination case study
3.3.1 Description of the case study
3.3.2 Participants
3.3.3 User task analysis
3.3.4 Expert study, guidelines, and prototyping
3.3.5 User evaluation
3.3.5.1 Experimental protocol
3.3.5.2 Results
3.3.5.3 Discussion
3.4 First approaches to improve the user-centred design methodologies
3.4.1 Observations from the preceding case studies and proposals
3.4.2 Application to the architectural walkthrough case study
3.5 Conclusions
4 Scientific issues
4.1 Research problem
4.1.1 Statements about VE and BIM benefits
4.1.2 Analyses of existing user-centred design methodologies for VEs
4.1.2.1 Starting point: previous knowledge about existing user-centred design methodologies for VEs
4.1.2.2 Analysis of Gabbard et al. user-centred design methodology for VEs
4.1.2.3 Analysis of Chen et al. user-centred design methodology for VEs
4.1.2.4 Statements about existing user-centred design methodologies for VEs
4.1.3 Conclusions about the research problem and research issues
4.2 Research questions
4.3 Hypotheses
4.4 Research scope
5 Methodology
5.1 Overview of our methodology and of its steps
5.1.1 Origins of our methodology
5.1.2 Steps of our methodology
5.2 Step 1: build a model of the domain-specific task
5.2.1 Overview
5.2.2 Human information processing and classification in kinds of primitive tasks
5.2.3 Our hierarchical decomposition model and hierarchical task analysis system
5.2.3.1 A hierarchical decomposition for a main input task
5.2.3.2 Our hierarchical model
5.2.3.3 Our semi-automated system
5.2.4 Conclusions
5.3 Step 2: determine proposals of VR interaction techniques
5.3.1 Overview
5.3.2 Step 2.1: Identification of the type of VR interaction and the task characteristics
5.3.3 Step 2.2: Characterisation of the VR interactions
5.3.4 Step 2.3: Determination of the characteristics of the appropriate VR interaction technique(s) for a VR interaction in terms of usability
5.3.4.1 Overview
5.3.4.2 Taxonomies of characteristics of VR interaction techniques
5.3.4.3 Deduction rules between interaction characteristics and interaction technique characteristics
5.3.4.4 Output of the step 2.3: appropriate VR interaction techniques characteristics in terms of usability
5.3.5 Step 2.4: Translation of the VR interaction techniques characteristics into a list of proposed VR interaction techniques
5.3.6 Conclusions
5.4 Step 3: create VR prototypes and run user evaluations
5.4.1 Overview
5.4.2 Step 3.1: Choice of the VR interaction technique(s) to be prototyped
5.4.3 Step 3.2: Development of the prototypes with the VR interaction techniques
5.4.4 Step 3.3: Preparation of the user evaluations
5.4.4.1 Overview
5.4.4.2 Quality factors for a user evaluation of usability
5.4.4.3 Measures and scores for evaluating the usability of VR interaction techniques
5.4.4.4 Scores weighting to obtain global balanced scores of usability
5.4.4.5 Our process for preparing the usability user evaluations of interaction techniques
5.4.5 Step 3.4: Realisation of the user evaluations and determination of the final VR interaction techniques
5.4.6 Conclusions
5.5 Conclusions on our methodology
6 Evaluation of our methodology: an experiment with AEC experts
6.1 Objectives
6.2 Evaluation, criteria, and numerical levels linked to the values of the criteria
6.3 A website as a concrete support for the experiment
6.3.1 Overview of the website
6.3.2 Step 1 on the website
6.3.3 Step 2 on the website
6.3.4 Conclusion on the website
6.4 Case study 1: preventive hazard identification
6.4.1 Description of the case study
6.4.2 Task of the case study: preventive hazard identification
6.5 Case study 2: design review of a plant room – space requirements aspect
6.5.1 Description of the case study
6.5.2 Task of the case study: design review of a plant room – space requirements
6.6 Experimental protocol
6.7 Results for the case study 1 – preventive hazard identification
6.7.1 Completion of the steps of the methodologies and time spent
6.7.2 Autonomy and questions for completing the steps
6.7.3 Functionalities
6.7.4 List of required interactions
6.7.5 VR interaction techniques recommended characteristics
6.7.6 Proposals of VR interaction techniques
6.8 Results for the case study 2 – design review of a plant room
6.8.1 Completion of the steps of the methodologies and time spent
6.8.2 Autonomy and questions for completing the steps
6.8.3 Functionalities
6.8.4 List of required interactions
6.8.5 VR interaction techniques recommended characteristics
6.8.6 Proposals of VR interaction techniques
6.9 Discussion
6.9.1 Usability criterion about our methodology
6.9.2 Autonomy criterion about our methodology
6.9.3 Functionalities results – similarity criterion
6.9.3.1 Case study 1: hazard identification
6.9.3.2 Case study 2: design review of a plant room
6.9.4 List of required interactions – similarity criterion
6.9.4.1 Case study 1: hazard identification
6.9.4.2 Case study 2: design review of a plant room
6.9.5 VR interaction techniques recommended characteristics – similarity criterion
6.9.5.1 Overview of the results and possible kinds of differences of results
6.9.5.2 ”Compatible differences” in the results – cases 1 and 2
6.9.5.3 ”Incompatible differences” in the results – cases 1 and 2
6.9.5.4 Similarity of VR interaction techniques characteristics results
6.9.6 Proposals of VR interaction techniques – similarity criterion
6.9.6.1 Case study 1: hazard identification
6.9.6.2 Case study 2: design review of a plant room
6.9.6.3 Similarity of proposals of VR interaction techniques results
6.9.7 Similarity criterion about our methodology: final assessment for H1 and H2
6.10 Conclusions about our three criteria of evaluation on our methodology
7 Conclusions and perspectives
7.1 Research question addressed
7.2 Contributions
7.3 Implications of our contribution
7.4 Limitations
7.5 Perspectives for our methodology
List of publications
Bibliography
Appendices
