Project Topics
Get Complete Project Material File(s) Now! »
The Use and Utility of Modelling Tools in Water Resources Management
Modelling tools are now a common part of water resources management and planning. But what exactly makes a model useful? Who are they useful to and what are they useful for? While interest and investment in modelling tools have increased in recent decades, their specific role in the decision-making process in practice remains somewhat ambiguous. This chapter attempts to explore these questions by analysing the diversity and design of water models in the examples of the PIREN-Seine in France and the CRC for Water Sensitive Cities in Australia.
“Error 404: User Not Found”
Major advances in computing power coupled with an emphasis on evidence-based policy have largely contributed to the popularity of modelling tools across major sectors. The rise of public policy tools in general – from simple checklists and decision trees to cost-benefit analysis and computer models of varying complexity – has been motivated by mounting pressure to ground decisions in best available evidence to address increasingly wicked environmental policy problems (Nilsson et al., 2008). In Europe, the systematic collection and use of evidence are perceived as ‘Better Governance’ (European Commission, 2003), while recent policy assessment systems specifically require the use of policy tools, favouring those that employ a quantitative approach (e.g. models) (Radaelli, 2004).
The advantage of modelling tools is that they help make sense of complex processes, while at the same time, transposing wicked problems into technical, ‘manageable’ solutions. This has led to large-scale investments in a wide range of modelling tools. For example, the European Commission’s 6th and 7th Framework Programmes have funded a number of research projects on policy assessment methods and tools, investing heavily in computer-based tools such as models (European Commission, 2003). From a scientific perspective, models are still one of the only tools available capable of analysing, evaluating and predicting the behaviour of environmental systems. From a management perspective, the exploratory and predictive capacities of models coupled with their basis in scientific evidence make them efficient policy instruments.
Despite increasing interest and potential advantages that modelling tools can offer and the wide range of tools that now exists to meet a multitude of needs, the use and adoption of models in practice continues to lag behind (Argent, 2004; Bach et al., 2014; Hipel and Ben-Haim, 1999; Liu et al., 2008; Marlow et al., 2013). This suggests that there is a mismatch between the growing supply of modelling tools and the demand in practice.
In a review of integrated urban water models, Bach et al., (2014) summarised current barriers as: 1/ model complexity, 2/ user friendliness, 3/ administrative fragmentation, and 4/ communication. In an assessment of participatory modelling, Carré et al., (2014) highlighted issues related to “institutional capacities and constraints”, described as differences in professional cultures as well as financial and political constraints (Turnpenny et al., 2008), a mismatch in perspectives between modellers and stakeholders (Radaelli, 2004), and the inability of models to meet the expectations of its end-users (Riousset, 2012).
In the case of PIREN, a diversity of modelling tools has been developed over the past 30 years. Yet, much like a computer will display a ‘404 Error’ code when the server is unable to find the requested resources, a search for examples of model uses in practice often returned the same response: ‘Error 404: User Not Found’. At the same time, PIREN practitioners reported a high level of satisfaction when it came to modelling, referring to models as ‘useful’ tools that were integral to their work. How then, can this inconsistency be explained?
The source of the error came down to a problem of binary thinking. To understand this discrepancy, it was necessary to first explore the common dichotomies of use vs. utility and research vs. operational models. Throughout the interviews, it became apparent that there was no clear distinction between these terms among participants. While for some, ‘using’ a model referred to physical manipulation of the model itself, others considered ‘use’ as making use of its results, sometimes without understanding the inner workings of the model. Conversely, some actors felt that they had “nothing to do with models” even if later they cited examples of how models have played a role in their work or of contributing to their development (e.g. providing input data). Sometimes, this was due to a distinction between their individual involvement and that of the institution they represent. There also appeared to be ambiguity around who uses what and to what extent, as there was often a mismatch between how one actor described another’s involvement and how the individual described their own involvement in the modelling process.
In terms of research vs. operational models, some made the distinction between the two according to application (i.e. the same model but applied in different contexts), while others perceived the difference as a matter of design (i.e. operational models are typically more ‘user-friendly’ and adapted to the needs of practitioners). Participants with the latter perspective often felt that research models tended to be more scientifically sound, since they were developed by researchers, whereas operational models were more adapted to answer management questions, since they were typically tailor-made by consultancies according to a design brief (‘cahier des charges’).
Distinguishing Between Use, Usage and Utility
In general terms, use is understood as “a method or manner of employing or applying something”, whereas utility is defined as “the quality or state of being useful” (Merriam-Webster Dictionary), which suggests an underlying motivation. For example, a model developer designs a model with a specific use in mind (e.g. to simulate stormwater runoff). However, its utility is determined not only by how it is used but also by whom it is used. Whereas the model may have been developed for its perceived utility in estimating the amount of water that is infiltrated and the amount of water that flows into the catchment during a rainfall event, an elected government official may in fact, use the model to estimate sanitation needs or justify a large budget, and understand its utility as a way of pushing through a political agenda and satisfying constituents (Commenges and Deroubaix, 2017).
In the context of this analysis, we further distinguish between use, utility and usage. Whereas use is understood as the technical or physical act, practice or activity of those developing or employing the model or its results, utility can be seen as the performance or outcome, while usage is considered the strategy employed by the user regarding the model. In other words, use is what is done in or to the model itself (e.g. entering lines of code or data, making a simulation, changing the parameters, retrieving results); usage is what the user intends to do with the model or its outputs (e.g. participating in the development of the model or justifying and action or decision), and; utility is what the model allows the user to do and has more of an impact on decision processes (e.g. understand or explain biological, chemical or physical processes, justify an action or decision, communicate to different audiences).
This thesis focuses primarily on the use and utility of modelling tools in the context of water resources management, while usage is evaluated more indirectly. The main reason is due to time constraints. Since this was the first analysis of this nature, it was necessary to begin with retracing the history of how and why models were developed before we could delve into the strategies of different actors, which requires a more in-depth analysis into the motivations of each actor and institution. Another constraint was that usage is difficult to characterise and quantify without first understanding use and utility, as it was seldom explicitly articulated. Even if several interviewees considered that models were useful or that they were used frequently, they were often unable to cite specific examples. This could also be due to the fragmentation of roles and responsibilities within institutions, which separated the individuals doing the modelling from those taking decisions, who may have been outside of the study context.
Research Models vs. Operational Models: Two Sides of the Same Coin?
This study initially followed the hypothesis that models can be divided into two general categories: research and operational models. Operational modelling was distinguished by a practical application or purpose, particularly in the context of decision-making (Hipel and Ben-Haim, 1999; Makropoulos et al., 2008; Tomasoni, 2014). Operational models can be explanatory or predictive, enhancing understanding of management or policy problems by allowing practitioners to explore multiple scenarios, evaluate and assess risk, and make forecasts based on scientific knowledge and empirical data. The results of these models are applied to address real problems and are used by decision-makers to develop, evaluate and assess public policies, regulatory standards, public works or urban projects.
In contrast, research models were characterised by a primary objective of deepening scientific understanding and can be used to test scientific hypothesis. Whereas operational models may be more accessible to non-expert users, research models tend to be highly academic, necessitating scientific expertise to operate and understand. In this sense, operational models are more in line with the idea of decision support tools (e.g., Argent et al., 2009; Giupponi, 2007; Matthies et al., 2007; Shim et al., 2002; Willuweit and O’Sullivan, 2013), as opposed to research models, which are typically not designed specifically for practitioners to use themselves. Research model outputs as well as the model itself often stay within the academic realm and may not have a direct link to practical applications.
In practice, however, the distinction between the two may not be so clear-cut. First, PIREN models were developed by researchers to deepen scientific understanding, while also aiming to (directly or indirectly) support management and planning decisions within a designated water basin. This suggested that the two categories are not necessarily mutually exclusive. Second, there was no consensus among actors on what differentiates a research model from an operational model. Whereas some actors made the distinction according to usability (e.g. operational models are more user-friendly), others considered the context of application to be the distinguishing factor.
The diversity of responses among interview participants suggested that this distinction was more a question of perspective than design. In other words, the difference between a research and an operational model was dependent on context. Specifically, where and how the model was applied, by whom, and for what purposes? At the same time, model design – what processes were taken into account, which actors were included in its development and to what extent, its ‘user-friendliness’, etc. – were all found to have an influence on a model’s use and utility.
Exploring the Use and Utility of Modelling Tools at the Science-Practice Interface
The ambiguity surrounding the terms above has largely contributed to the confusion over what makes a model ‘useful’. This has led to the common view that models must be used ‘directly’ and adopted by practitioners in order to be useful. This would partially explain why some authors continue to report that models are rarely used as decision support tools in water resources management (Carré et al., 2014; Riousset, 2012; Uthes et al., 2010).
Current literature follows this perspective (Makropoulos et al., 2008; Willuweit and O’Sullivan, 2013), which can be explained in terms of substantive rationality and procedural rationality, the former supposing that better technology leads to better decisions, while the latter supposes that better technology enhances the decision-making process itself (Commenges et al., 2014). Whereas substantive reality sees the function of a model as directing decision-makers towards a rational decision by identifying specific policies that can produce optimal outcomes (Ascher, 1981), procedural rationality sees the role of modeling to support the decision-making process by reducing the effort required to make decisions (Todd and Benbasat, 1992).
However, the lack of use and adoption by practitioners does not necessarily negate its utility. Our empirical examples show that models are perceived to be useful even in cases where they are not directly adopted. In PIREN, for example, the majority of actors believe that models are useful despite the fact that there are only two examples of models being fully adopted by practitioners. The example of the CRC for Water Sensitive Cities also shows that different types of models have different uses and utilities. For example, while earlier models were intended to serve more short- to medium-term management and planning purposes, a number of new modelling tools are designed for long-term scenario planning under deep uncertainty. Both purposes may be useful, depending on the stage of the decision-making process.
Table of contents :
GENERAL INTRODUCTION
Water in the 21st Century: Evolving Paradigms of Science and Management
The Paradox of Complexity and Systems Thinking
The Paradox of ‘Evidence-Based Policy’
The Paradox of Transboundary Collaboration
Continued Challenges at the Interface of Science, Practice and Policy
Context of the Thesis
A Call for (Restrained) Introspection
Challenges and Limitations
Thesis Objectives
Two Contrasting Cases: Examples from France and Australia
Evolution of Analysis
Thesis Structure
CHAPTER ONE THE USE AND UTILITY OF MODELLING TOOLS IN WATER RESOURCES MANAGEMENT
Error 404: User Not Found
Distinguishing Between Use, Usage and Utility
Research Models vs. Operational Models: Two Sides of the Same Coin?
Exploring the Use and Utility of Modelling Tools at the Science-Practice Interface
Article Use And Utility: Exploring The Diversity And Design Of
Water Models At The Science-Policy Interface
Natalie Chong | 2019
CHAPTER TWO RECONCILING UNCERTAINTY IN THE HYDROINFORMATIC PROCESS
Part One Exploring Perspectives and Approaches to Uncertainty
Understanding Uncertainty
Uncertainty Framework
Source of Uncertainty
Level or Type of Uncertainty
Nature of Uncertainty
Perspectives and Approaches to Uncertainty in the PIREN-Seine
Article Reconciling uncertainty in the hydroinformatic process:
Exploring approaches, perspectives and the spaces between
Part Two Dealing with Uncomfortable Knowledge in ‘Evidence-
Based Policy’
The Production and Utilisation of Knowledge
Objective Facts in Subjective Spaces
Governing by Numbers: Scientific Truths and the ‘Illusion of Precision’
Dealing with ‘Uncomfortable Knowledge’
Article Eyes wide shut: Exploring practices of negotiated ignorance
in water resources modelling and management
CHAPTER THREE THE ROLE OF BOUNDARY ORGANISATIONS
The Social Construction of Boundaries
The Evolving Nature of Science: New Modes of Knowledge Production
The Emergence of Boundary Organisations
Exploring the Role and Functioning of Boundary Organisations
Enhancing the Use and Utility of Knowledge and Tools
Reconciling Uncertainty in Model-based Decision Support
Beyond Boundary Organisations: Towards a Multi-scalar Approach to Boundary Work
Are Boundary Organisations Necessary?
Stabilising the Paradigms of Knowledge Production
CONCLUSIONS AND PERSPECTIVES
Models: Solution or Delusion?
Models as a ‘Boundary Object’
Models as Boundary Objects in a ‘Scoping’ Approach
Models as Boundary Objects in a ‘Fixed-Frame’ Approach
Key Factors Contributing to Models as a ‘Boundary Object’
What Qualifies as Adequate Representation?
Technological Representation: Is More Data Necessarily Better?
Social Representation: Who’s In, Who’s Out?
Towards Digital Catchments: A Happy Compromise?
BIBLIOGRAPHY
