Games for visual aesthetics learning

Get Complete Project Material File(s) Now! »

Chapter 3 – Methodology

Design-based research

As opposed to a particular set of evaluation methods, design-based research (DBR) is a methodological paradigm encompassing a series of related approaches that orients the way a research study is conducted (Barab & Squire, 2004; van den Akker, Gravemeijer, McKenney & Nieveen, 2006; Reimann, 2011). DBR was first introduced as ‘design experiments’ (Collins, 1992; Brown,1992) and has since been described under various names in the literature including ‘design-based research’, ‘design research’, ‘design science’, ‘design studies’ and ‘development research’.
Wang and Hannafin (2005) defined DBR as:…a systematic but flexible methodology aimed to improve educational practices through iterative analysis, design, development, and implementation, based on collaboration among researchers and practitioners in real-world settings, and leading to contextually-sensitive design principles and theories. (pp. 6-7)
Van den Akker et al. (2006) further identified five essential characteristics of DBR. Some of the rationale and philosophy behind each characteristic follow.

Characteristic 1: Interventionist

Since its inception, the central idea of DBR is to conduct educational research that advances not only the theory but also the practice in education (Collins et al., 2004). To strengthen the bridge between theory and practice, DBR necessitates that theory-driven design studies are conducted in real-world settings where learning normally takes place. While DBR may also be implemented in informal environments (e.g. Reisman, 2008), this research is particularly concerned with technological intervention in formal educational settings, such as classrooms.
Cobb, Confrey, diSessa, Lehrer and Schauble (2003) termed the natural classroom setting metaphorically as the ‘learning ecology’, with interacting elements such as the following contributing to the complex environment:…tasks or problems that students are asked to solve, the kinds of discourse that are encouraged, the norms of participation that are established, the tools and related material means provided, and the practical means by which classroom teachers can orchestrate relations among these elements. (p.9)
Situating the design study within its local context, therefore, leads to a more holistic understanding of its relevance to the learning process as well as its practicality for teachers and students in naturalistic settings (Brown, 1992; Barab & Squire, 2004).

Characteristic 2: Iterative

To deepen the understanding of the local context of an educational intervention, DBR also calls for an iterative research approach. The initial design and insights evolve through multiple cycles of analysis, development, evaluation and refinement over an extended period. These cycles often include sub-studies within a larger study, such that each sub-study includes “its own complete cycle of enquiry and sound chain of reasoning” (McKenney & Reeves, 2012, p.15).

Characteristic 3: Process-oriented

When integrating new learning technologies such as digital games into an educational context, Amiel and Reeves (2008) urged that such technological intervention should be understood as a process rather than a mere independent variable in the learning study. This process-oriented approach of DBR entails the shift of focus from “a black-box model of input-output measurement” (van den Akker et al., 2006) to understanding and refining the intervention as learning unfolds.

Characteristic 4: Utility oriented

For the intervention to eventually be harmonious with the learning ecology, it needs to be – as McKenney and Reeves (2012) put it – “rooted in, and not cleansed of, the complex variation of the real world” (p.15). Hence, DBR places a strong emphasis on utility; requiring the design to be practical enough to be carried out and used in the complex and messy real-world environment. The virtue of the design is largely measured by this practicality (van den Akker et al., 2006).

Characteristic 5: Theory oriented

What differentiates DBR from similar methodologies is it is strongly anchored to theory. Not only is the initial design of the intervention grounded, in theory, the design process also plays a significant role in the pursuit of theory development (Edelson, 2002). As further highlighted by Barab and Squire (2004): “The design is conceived not just to meet local needs but to advance a theoretical agenda” (p.5). DBR therefore, necessitates moving beyond merely testing a theory or reporting how the design works in a real-world context. It is conducted to gather evidence that can further refine design principles or generate new theories (Cobb et al., 2003).
Edelson (2002) suggested three types of theories that design can generate.
•Domain Theory – a descriptive theory about the world as generalized from some portion of the problem analysis. DBR can contribute to this theory in the form of context (by characterizing the opportunities and challenges that come with the design context) or outcomes (by characterizing the set of outcomes associated with the learning intervention, be it a desired or an undesired outcome).
•Design Framework – a prescriptive, generalized design solution that comprises a collection of coherent design principles characterizing what a design artefact must have in order to achieve a particular goal (or goals) within its particular context.
•Design Methodology – a prescriptive, generalized design procedure that comprises guidelines for the process rather than a product.
In summary, the DBR paradigm can also be described as pragmatic, grounded in theory as well as real-world practice, interactive, iterative and flexible, integrative, and contextual (Wang & Hannafin, 2005).
The next subsection describes the vertical slicing method that was used in this research in conjunction with this methodological paradigm.

Vertical slicing method

Vertical slicing is one of the methods used in agile software development. It is “the complete yet portioned implementation of a functional, valuable and demonstrable feature through all technology layers” (Ratner & Harvey, 2011, p.240). Using this method, a software project follows an iterative development of a thin vertical slice driven from e.g. the presentation to the data layer through to completion (Figure 3.1a). This can be contrasted with the more traditional horizontal approach (Figure 3.1b) to software development where each technology layer is built layer by layer.
A completed vertical slice is similar to a vertical prototype where both are high in fidelity, have great depths, and a narrow scope. However, a vertical slice is slightly different in that the narrow section or ‘slice’ implemented is a carefully selected, fully functional feature of the software that is both valuable and clearly apparent to its stakeholders.
One analogy that has been used to illustrate this point is portrait painting (e.g. Keith, 2011). Borrowing this analogy, let us say the finished software is like the finished painting of Mona Lisa 23 (Figure 3.2a). If the artist were to use the vertical slicing approach, the portrait would be painted to completion portion by portion. Hence, the first ‘vertical slice’ should be a portion of the portrait painted to its ‘finished state’.
Both Figure 3.2b and Figure 3.2c fulfilled this criterion. However, the portion selected in Figure 3.2c has more value to its stakeholder compared to Figure 3.2b as featuring the head is more meaningful than presenting e.g. a random vertical strip.
To ensure that portions meaningful to the stakeholders or users are selected, the feature for each vertical slice is usually described in the form of a ‘user story’
– typically a short sentence that captures the software requirement from the user’s perspective: e.g. “I as a (role) want (function) so that (business value)”
(Cohn, 2004, p.81). Splitting the software project into a series of small user stories is desirable as it allows for a more rapid iteration of building and user testing each vertical slice (Ratner & Harvey, 2011).
The vertical slicing’s capacity to support rapid iteration matches well with the progressive refinement approach of DBR. The application of both DBR and the virtual slicing method in respect to this research is covered in the next section.

General research approach

The following outlines the general approach of this research as informed by the five characteristics (described earlier in Section 5.1.1) of the DBR paradigm.
1.Interventionist: all versions of the prototype developed in this research were trialled in actual undergraduate HCI courses. They were treated as part of students’ classroom learning activities and were conducted in collaboration with the course instructor(s).
2.Iterative: the research follows iterative-incremental cycles of study, with each cycle also building on the empirical findings from its previous cycle.
3.Process oriented: the initial design is left flexible to allow for any deliberate changes necessary, especially after analysing students’ feedback and learning outcomes in each study round. All the design decisions that have to be made as part of the design process were recorded and reported (see Chapters 4, 5 and 6). As suggested by Edelson (2002), these decision-making processes and observations of their consequences represent learning opportunities that can lead to better-informed design decisions in future. As such, this thesis also takes into account the researcher’s observations and reflections throughout the process.
4.Utility oriented: the design of the intervention takes into account its utility for both students and instructors. It was designed to fit within standard lab-based tutorials as well as self-directed online learning. The design was also faithfully revised based on students’ feedback from its classroom use.
5.Theory oriented: the designs of a new visual aesthetics learning module (covered in Chapter 4), as well as technological interventions (covered in Chapter 5 and 6) are driven by Constructivist learning theories and principles that are relevant to visual aesthetics learning.

1 Introduction
1.1 Background and motivation
1.2 Research question, aim and objectives
1.3 Thesis structure
1.4 Publications and Presentations
2 Games for visual aesthetics learning
2.1 Curriculum: visual aesthetics and design learning.
2.2 Theoretical framework: constructivism
2.3 Instructional strategy: game-based learning
2.4 GBL as the instructional strategy for visual aesthetics
2.5 Chapter summary
3 Methodology
3.1 Background
3.2 General research approach
3.3 Specific research activities
3.4 Chapter summary
4 Module design 
4.1 Adopting the SOLO taxonomy in the module design
4.2 DELMo: the new SOLO-based learning module
4.3 RD1: Reflections on DELMo’s implementation
4.4 Chapter summary
5 Design Eye v.1: Supporting the SOLO-based approach 
5.1 Technological intervention design
5.2 Implementation
5.3 Ev1: Evaluating Design Eye v.1
5.4 Chapter summary
6 Design Eye v.2: Leveraging game-based learning 
6.1 Exploring serious mini-games
6.2 Implementing the serious mini-games4
6.3 Ev2: Evaluating the serious mini-games
6.4 Design revisions for Design Eye v.2
6.5 Ev3: Evaluating Design Eye v.2
6.6 Chapter Summary
7 Discussion and conclusions
7.1 Reflections in fulfilling the research objectives
7.2 Research implications
7.3 Conclusions
GET THE COMPLETE PROJECT
Instructional strategies for the visual aesthetics of user interface design