Project topics and materials
Get Complete Project Material File(s) Now! »
Expert elicitation
Experts hold a wealth of contextual information that can be very useful in situations where little or no data are available. Applications of expert elicitation are varied and range from species richness estimates for coral reefs [44] and water quality modelling [46], to predicting the presence of the brush-tailed rock-wallaby [47], amongst others.
Elicitation can happen face-to-face or through interviews, phone calls, etcetera. Face-to-face interviews or workshops are more advantageous as it eliminates communication errors. This method also makes it easier to recognise when an expert does not understand something so that certain aspects can be explained to them in a dierent way [44]. Experts might be highly rated in their respective fields, but that might not translate into providing accurate and reliable probability assessments [44]. The value of the elicited knowledge depends on the expert’s level of expertise, and ability to be objective.
Eliciting knowledge, especially probabilities, from experts is a challenging task. Uusitalo [38] states that one of the challenges of working with BNs, especially with expert knowledge, is to extract knowledge from experts in such a way that it can be turned into probability distributions. Experts find it dicult to provide probabilities without data, and some feel uncomfortable thinking about distributions instead of point estimates and confidence intervals [35, 38]. It could be that they fear being misrepresented (such as in an academic publication or a news article), or that they are unsure of how to quantify their own knowledge. Instead of supplying hard numbers, experts prefer to adjust a certain empirical estimate up or down [44], which is what was done in this study. This elicitation technique will be described in Chapter 5.
The following five papers illustrate the dierence between eliciting knowledge directly and eliciting knowledge indirectly. The direct or structural approach to expert elicitation is the easiest approach from the researcher’s perspective [45]. This entails asking the expert directly for values corresponding to their belief in parameters. An indirect approach is followed when the experts are not familiar with probabilities. James et al. [45] believe that an indirect approach allows for greater accuracy. Examples of where the indirect approach is followed include the papers by James [45] and Fisher [44].
Both James et al. [45] and Fisher et al. [44] develop software tools for eliciting expert knowledge, named Elicitator and ElicitN respectively. The tool developed by James et al. [45] shapes the quantification of expert knowledge so that it can be used as a prior in Bayesian regression. The tool provides a graphical interface for the experts to share their knowledge, and provides quick graphical feedback, which is said to result in better elicitations. The expert is asked for the best estimate of the probability of a certain scenario [45]. The expert is also asked to provide upper and lower quantiles as well as estimates of the upper and lower bounds for the probability of that scenario. Lastly the expert has to supply a confidence rating for every probability.
Fisher et al. [44] use their software tool, ElicitN, to estimate species richness in coral reefs. The software tool helps to elicit probability distributions from the experts by capturing the expert’s “best guess” as well as the variability present to calculate likely intervals. The authors state that the first step in designing an elicitation process is to develop a statistical model that represents the underlying model [44]. As with James et al. [45], the authors of Fisher et al. [44] elicit subjective probability distributions from the experts that capture their uncertainty. Experts were also asked for their “best guess” of the “most likely” value. This value was also elicited as the mode, and not the mean or the median. The experts could choose whether they wanted to answer the questions as a percentage or as a multiplicative factor. A total of six parameters are elicited from the experts.
Another example of where an indirect elicitation method was used is illustrated by Denham [48]. Their approach is called an “indirect predictive P-method of elicitation” and was designed specifically for ecological modelling. The method uses an interactive map-based Geographical Information System (GIS) tool. The tool is used to elicit site-based parameters for a regression model. The expert selects a subset of the available features in the software tool, as well as a specific point on the map. A pop-up dialogue box then presents the expert with the dierent variable states at that location (much the same as with the grid for the rhino poaching problem). The expert is asked to consider 100 similar sites and to then supply a value for the median. These values present a conditional probability table (CPT) for the nodes. The expert also has to supply a lower bound and an upper bound for assigned probabilities.
Van Houtven et al. [46] developed a protocol to combine ecological and economic processes, thereby integrating environmental modelling, expert elicitation, and “nonmarket valuation methods”. They illustrate the protocol with a case study of nutrient loads to lakes and lake water quality in North America. Das [49] devises a strategy to populate the CPTs as well as lighten the knowledge acquisition load by using weights to quantify the strength of the causal links between parent nodes and child nodes.
These weights are elicited from experts and are used in a weighted sum algorithm to populate the CPTs. The weighted sum algorithm lightens the load on the expert in the sense that the expert has to come up with fewer probability distributions than usual. This increases the consistency as well as the expert’s confidence. This approach is very similar to what was done in this study.
In the next few paragraphs, papers are cited in which the process is described whereby CPTs are populated. Wang [50] developed a conditional probability tree (CPTree) as an attempt to simplify the elicitation process. It presents a node’s conditional probability table (CPT) as a tree view where the expert can expand or collapse the list of nodes and states according to his preference. The author also developed a shrinkable conditional probability table (sCPT) that gives a dierent view of the CPTs.
Both tools contain graphical tools for elicitation based on bar charts and pie charts. Developing a network with the help of experts is a daunting task, which is one of the reasons why a rhino poaching network structure was developed before presenting it to a panel of experts. Experts state their beliefs and encode those into probabilities. Sometimes, however, the probabilities can be data-driven and objective [50].
CHAPTER 1 INTRODUCTION
1.1 INTRODUCTION
1.2 PROBLEM STATEMENT
1.2.1 The problem
1.2.2 The facets of rhino poaching
1.2.3 Previous and current approaches to the problem
1.2.4 Why a new approach is needed
1.3 RESEARCH OBJECTIVES
1.4 THESIS STATEMENT
1.5 DELINEATION AND LIMITATIONS
1.7 UNDERLYING ASSUMPTIONS
1.8 SIGNIFICANCE OF THE STUDY .
CHAPTER 2 LITERATURE REVIEW
2.1 INTRODUCTION
2.2 THEORY BASE
2.2.1 Bayesian networks
2.2.2 Expert elicitation
2.3 BAYESIAN NETWORK APPLICATIONS
2.4 RHINO POACHING2
2.5 CONCLUSION
CHAPTER 3 THEORY AND CONCEPTS
3.1 INTRODUCTION
3.2 PROBABILITIES
3.2.1 Joint and marginal probabilities
3.2.2 Conditional probabilities
3.3 BAYES THEOREM
3.3.1 The frequentist approach versus the Bayesian approac
3.4 BAYESIAN NETWORKS
3.5 POSTERIOR, LIKELIHOODS, AND PRIORS
3.5.1 Probabilities versus likelihoods
3.5.2 Conditioning versus dependence
3.6 COUNTERFACTUALS
3.7 CONCLUSION
CHAPTER 4 DESIGN OF THE STUDY
4.1 INTRODUCTION
4.2 OVERALL APPROACH
4.2.1 Survey-based research
4.2.2 Secondary data analysis
4.2.3 Statistical modelling
4.3 ADDITIONAL CONCEPTS
4.3.1 Workshops and expert knowledge
4.3.2 Routine Activity Theory
4.3.3 Transdisciplinary research
4.4 DETAILED METHODOLOGY
4.4.1 Research instruments
4.4.2 Expert knowledge and GIS data
4.5 LIMITATIONS
4.6 ETHICS
4.7 CONCLUSION
CHAPTER 5 DEVELOPING THE BAYESIAN NETWORK
CHAPTER 6 THE FINAL MODEL
CHAPTER 7 RHINO POACHING APPLICATION
CHAPTER 8 RESULTS, VALIDATION, AND INSIGHTS
CHAPTER 9 CONCLUSION
