Article
Simbarashe Nhokovedzo
Finland Futures Research Centre, Turku School of Economics, FI-20014 Turun yliopisto, Turku, Finland
Abstract
The article compared two methods of futures studies: the Delphi method and Horizon Scanning. Comparing the two foresight methods through literature review revealed more differences than similarities along dimensions such as origins, classification, characteristics, application, popularity in the foresight arena, strengths, and possible future progression of the methods. Conclusively, the two methods emerged as ideal for complimentary use, one after the other or simultaneously in a single foresight project with Horizon Scanning as the forerunner and/or parallel runner to Delphi studies.
Keywords
Delphi Method, Horizon Scanning, Futures Studies
Introduction
The Delphi Method and Horizon Scanning methods are commonly used in futures studies in both academic and consultancy settings. Selection of these two methods ahead of other foresight methods depends upon many factors, which include requirements of the task at hand, abilities of the methods to satisfy the objectives of a given task, nature of information needed and/or availability of expertise. Both methods have a fair share of peculiar capabilities that make them favourable to the users in particular circumstances. Comparison of these methods is imperative for foresight practitioners and futures researchers to deeply understand the characteristics, application contexts, strengths and weaknesses in order to apply them from an informed position (Puglisi, 2001). The purpose of this article is to compare the Delphi method and Horizon Scanning, giving contrasts and similarities. The article first delineates the two methods along definitions and origins, characteristics, common applications, general classification, dimensions, strengths, weaknesses, morphosis and possible futures. It then examines the similarities and ends with a conclusion on complementarity of the methods.
Definitions and origins of Delphi method and Horizon Scanning
There seems to be concordance among scholars on the definition of Delphi method. One of the earliest pioneers of Delphi method, Norman Dalkey, defined it as a technique of eliciting and refining group judgements to aid decision making (Dalkey, 1969). Linstone, et al, (2002) defined the Delphi method as a method of structuring group communication process in an effective manner that allows the group of individuals to deal with a complex problem. The Delphi method is often comprised of systematically selected experts to discuss a topical subject. Calleo & Pilla, (2023), therefore, viewed it as a process of knowledge generation that allows structuring the communication of a group of experts, congealing their subjective judgements regarding reality, and forecast of events for decision making. The scholars generally agree on the group communication aspect of Delphi method whilst they slightly differ on the intended outcomes. Dalkey (1969) and Calleo & Pilla (2023) linked it directly to policy formulation whilst Linstone et al, (2002) linked it to complex problem solving. Though the origins of Delphi method is traceable to the ancient Greek Delphi oracle, the modern futures studies version was developed by the RAND corporation in the 1950s to forecast the effect of technology on warfare. Among the earliest proponents were Norman Dalkey, Olaf Helmer and later Theodore Gordon among others (Gordon, 2009).The method builds on the traditional adage that “two heads are better than one” (Dalkey, 1969; Grime & Wright, 2016), which stresses the superiority of many experts opinions over one expert’s opinion. On the other hand, Horizon Scanning seems to have some ambiguities in its definition. Many scholars struggle to differentiate Horizon Scanning from environmental scanning, but they still maintain that there is a difference. To some, environmental scanning is a broad exercise of monitoring the broader environment often represented by the (Social, Technology, Economic, Ecology and Politics) STEEP acronym (Bishop, 2009), while Horizon Scanning is a much focused method of identifying trends, weak signal, wild cards, for a particular phenomenon (Hideg et al., 2021). Citing Sutherland et al 2010 (Palomino et al., 2012), offers one of the clearest definition of Horizon Scanning:
“Horizon scanning is a systematic search for incipient trends, opportunities and trends that might affect the probability of achieving set goals.”
Delaney, (2014) precisely defined Horizon Scanning as an attempt to systematically imagine the future in order to plan a response on detected signals, identified trends in order to capitalise on opportunities and mitigate risks. Cuhls, et al (2015:7) concurred with previous scholars, by highlighting Horizon Scanning as a systematic outlook to detect early signs of potentially important developments on a phenomenon of interest which flag out weak or early signals, trends, wild cards, persistent problems, risks, and threats. The recurring word in these definitions is “systematically’, which means that Horizon Scanning follows conventional procedures, at least, as opposed to general information searches.
Classification, typology, aims of Delphi method and Horizon Scanning
Futures research methods has been classified in various ways by scholars over the years. The classifications are based on method characteristics, application processes, aims and outputs among other dimensions. The Delphi method and Horizon Scanning appear differently in a classification shown by the Foresight diamond.
Fig 1: The Foresight diamond (Popper, 2008)
Interpreting from the foresight diamond, the Delphi method is classified as an expert-based method, in which selected experts apply their creativity, in a partially interactive process and based on their knowledge. More recent forms of Delphi versions such as the Real-Time Delphi (RTD) are highly interactive (Varndell et al., 2021). On the Millennium Project classification table, the Delphi method assumes qualitative, quantitative, exploratory and normative forms (Glenn, 2004). This shows how rich the method (and its variants) has become over the years. The aim of Delphi method is to generate expert opinions through an iterative process punctuated by controlled feedback. Some Delphi aims at consensus building among the experts while the dissensus type of Delphi does not seek agreement (Tapio et al., 2017). Dissensus Delphi are very common in policy related projects. There are several other versions of Delphi method, namely classical, conventional , Delphi conferences, Real-Time Delphi, policy Delphi, fuzzy Delphi, max-min Delphi, explorative, normative, quantitative, qualitative, historical, and Delphi forecasts among others (Withanaarachchi et al., 2015). On the contrary, Horizon Scanning is classified as an evidence-based method which is qualitative and exploratory in nature (Glenn, 2004). The evidence used in Horizon Scanning is obtained from people, databases and websites (Palomino et al., 2012). The aim of Horizon Scanning is to identify weak signal, trends, drivers, mega trends, continuities, discontinuities and wild cards (Saritas & Smith, 2011), and communicate them for decision making. Early detection of Horizon Scanning outputs is most ideal when the aim is to mitigate threats, inform innovation processes or seek competitive advantage.
Execution processes
The Delphi method and Horizon Scanning differs on their execution. Delphi studies rely on four important features namely anonymity, controlled feedback and iteration and formal group judgment (Dalkey, 2003). Anonymity is realized through data collection instrument administration in such a way that a group member’s contribution remain unnamed (Dalkey, 1969). In Real-Time Delphi, a group member can see aggregated response of others but not their identities. Iteration and controlled feedback occur when initial response becomes input for the next round data collection instrument, which is usually a questionnaire. Delphi data is collected in multiple rounds, allowing respondents to modify their response in the process (Withanaarachchi et al., 2015). Formal group judgment (also known as statistical group response) is the final aggregation of group response after the last round. However, group judgment must never be mistaken as objective truth or evidence-based truth but rather a subjective truth from a group of experts. Conversely, Horizon Scanning is a one shot process that can be repeated more frequently overtime to check if weak signals are maturing, fading out whether previously identified signals are degenerating into forceful trends or fading out (Wintle et al., 2020). The below table contrasts execution steps on Delphi method and Horizon Scanning.
Table 1: Comparing Delphi method and Horizon Scanning execution steps.
Delphi method typical process
|
Horizon Scanning process
|
Source: Informed by (Withanaarachchi et al., 2015 and Palomino et al., 2012)
Circumstances of use and timeframe
The Delphi method is often deployed under conditions of uncertainty for topics relating substantially to policy, governance and to long distant futures spanning 25 years to 50 years (Hsu & Sandford, 2007;Sablatzky, 2022). It is most appropriate when the problem at hand requires collective and subjective judgements and when group dynamics do not promote effective communication (Rowe & Wright, 2011) to an extent that renders focus group discussions impossible. Furthermore, the Delphi method is the best if available knowledge is incomplete or there is uncertainty that renders conventional methods ineffective (Galanis, 2018). Examples of circumstances where Delphi can be suitable are policy formulation to govern new technologies such as cultured meat, green energy innovations and higher versions of artificial intelligence. To these, there is limited or no historical data from which reliable extrapolations can be made thus one can rely on expert opinions in their consensus or dissensus forms. An example is the Delphi study on governance of Artificial General Intelligence (AGI) currently run by the Millennium Project with experts drawn from AI, politics, academia, quantum computing, futurists, business people, and the broader society opinion leaders. The duration of a Delphi study is usually longer when compared to Horizon Scanning. Though the duration of Delphi depends on the nature of the topic, level of technology use, approaches of the researcher, availability of experts among other factors, most Delphi studies run for a period between two to six months (Searchinger et al., 2018;Shang, 2023). Conversely, Horizon Scanning is relatively concerned with the detection of signals, trends, and disruptors in the present and progression of these identified issues over a period of time. Horizon Scanning is suitable where latent or major change in a phenomenon, industry or landscape is being monitored. For instance, monitoring changes in a particular organization’s competitive environment or market. Circumstances where decision makers are keen to detect early signs of threats or opportunities (Bonoff, 2007) prompts the use of Horizon Scanning. Depending on the source of scanning information (such as databases, websites, mass media and people), a Horizon Scanning exercise can take a few minutes to a few months. There are several emerging companies offering collaborative and user-friendly AI based platforms for digitized Horizon Scanning. One such company in Finland is called Futures Platform (https://www.futuresplatform.com/).
Popularity as a futures method
In its various types and forms, the Delphi method seems to be more popular in futures studies than Horizon Scanning. A query search in the Science Direct database including key words, ‘Delphi method’ and ‘futures studies’ yielded 500 outcomes whilst the same search including the words ‘Horizon Scanning’ and ‘futures studies’ comparably yielded 194 outcomes. Though reasons for the popularity of the Delphi method over Horizon Scanning could not be fully substantiated in the scope of this article, one can attribute it to the multiplicity of Delphi types and their ability to build scenarios. The scenario method is one of the most popularly used method to an extent that some people outside the foresight profession confuse scenarios and futures studies (Puglisi, 2001). Most foresight practitioners use different routes to build scenarios, one of which is the Delphi method. The use of Horizon Scanning to build scenarios is limited as it tends to promote ‘business as usual’ scenarios through extrapolation of observed trends.
Methods progression and potential future direction
With changes in the broader society, the use, form, application, and direction of foresight methods is changing accordingly. The Delphi method seems to be enjoying multiple, faster modifications than Horizon Scanning. From the earliest version of the Delphi method that was generated at the RAND corporation, the method has progressed significantly with a lot of modifications. Initially, the method was meant for reaching a consensus among experts (Hirschhorn, 2019), but today it can be applied to produce dissensus outcomes (Tapio et al., 2017). Though it retained most of its core founding principles such as anonymity and group thinking, the Delphi method has now disintegrated into so many types with relevance in different fields (Withanaarachchi et al., 2015). Interestingly, the Delphi method has heavily assimilated technology, thus now running on software which is continually upgraded. For example the Real-Time Delphi has now become ‘roundless’ (Cech & Tellioglu, 2019;Hofer et al., 2022), but allows experts to return to make changes to their previous response if they so wish. The Delphi method is already poised to make a new direction when it assimilates Artificial Intelligence. Proponents of smart foresight applications are working on a Delphi on the A.I (Zartha et al., 2020). On the other hand, Horizon Scanning has its own remarkable progression from the early work of Igor Ansoff to the exploratory scanning and issue centered scanning (Amanatidou et al., 2012) and the AI based scanning expert systems like the one offered by Futures Platform. It is likely that Horizon Scanning from data bases, online news articles and websites will become faster and more efficient with more advances in AI. For example, platforms such as Futures Platform are making significant progress to fully digitise Horizon Scanning and shorten the scanning time to one second or less through finger clicks.
Strengths and weaknesses of Delphi method and Horizon Scanning
The two-foresight methods can be demarcated along their strength and weakness. Knowing strengths and weakness of futures methods is a requirement when choosing combination of methods to use in a single foresight project (Puglisi, 2001). In mixing, foresight methods are selected partly on the basis of their complimentary combinations which maximise their strengths and minimise limitations. Table 2 shows the strengths and weaknesses of Delphi method and Horizon Scanning
Table 2: Comparing Delphi method and Horizon Scanning on strengths and weaknesses.
| Delphi Method | Horizon Scanning |
Strengths
|
Strengths
|
Weakness
|
Weaknesses
|
Potential combination of the methods in a single foresight project
As outlined in table 2, the Delphi method and Horizon Scanning possesses varying sets of strengths and weaknesses. Based on this knowledge, foresight practitioners can target these methods for combination as they offer relatively mutually exclusive benefits. In combining the methods, most practitioners start by Horizon Scanning to inform questions to include in the Delphi method (Wintle et al., 2020). However, it is also possible to have mini-Delphi embedded in Horizon Scanning work (Bonoff, 2007).
Similarities
Even though the Delphi method and Horizon Scanning are dissimilar on many aspects as discussed in above sections, they have some aspects in common. Firstly, both methods possess a penchant of multi-disciplinarity though this quality appear to be hidden particularly in Delphi. In selecting a Delphi panel, consideration is given to experts with knowledge of the subject (Chuenjitwongsa, 2017), but these experts are ideally drawn from different disciplines. For example in a Delphi study for energy futures, Tapio and others composed a Delphi panel with cognitive expertise spanning agriculture, natural resources, economy, society, climate, renewable energy and technology (Tapio et al., 2017). The multidisciplinary approach in panel selection breaks the silo mentality thinking by experts from a specific field. This improves quality of the Delphi outcome in decision making through cross pollination of opinions. Similarly, Horizon Scanning swipes through the breadth of the society on STEEP and pick the relevant signals (Bonoff, 2007;Hideg et al., 2021) from a wide spectrum of elements. The STEEP elements cuts across the whole society and professional/specialization silos hence embraces multidisciplinarity.
Secondly, both methods are iterative in practice despite their different approaches in iterating. The Delphi method relies on multiple rounds (Dalkey, 2003;Hsu & Sandford, 2007) where respondents can make changes to previously given opinions. Similarly, Horizon Scanning emphasise on repeating the scans to check if there are changes on the identified trends and signals (Palomino et al., 2012). Continuous repetition of the scans is akin to iterations in a way.
Thirdly, both methods are systematic as they follow some sequential and conventional procedures known in the foresight profession. As outlined in table 1, both methods are executed through series of structured steps (Grime & Wright, 2016;Delaney, 2014). However, it must be noted that these steps are not hard and fast cast stones or manuals, but they guide researchers. It is worth noting that some of these steps are becoming redundant as these methods are fast assimilating technology. A good example is the RTD which has repealed some of the monotonous steps associated with the traditional Delphi method.
Conclusion
As discussed, the Delphi method and Horizon Scanning are different across many dimensions which include their classification, timeframe, sources of data, outcomes, and applications. However, they resemble some similarities on embracing multidisciplinarity, iterativeness and being systematic. Though these methods are effectively used separately, I conclude that, with more differences and less similarities, researchers can benefit by combining these methods concurrently or sequentially in a single foresight project. With more differences the compared methods could have a higher complementarity power.
References
Afshari, A. R. (2019). Using the Delphi method for futures studies. International Conference on Industrial Engineering and Operations Management, 175–181. http://ieomsociety.org/pilsen2019/papers/69.pdf
Amanatidou, E., Butter, M., Carabias, V., Könnölä, T., Leis, M., Saritas, O., Schaper-Rinkel, P., & van Rij, V. (2012). On concepts and methods in Horizon Scanning: Lessons from initiating policy dialogues on emerging issues. Science and Public Policy, 39(2), 208–221. https://doi.org/10.1093/scipol/scs017
Bonoff, S. (2007). Environmental scanning. IPA Bulletin, 97(2), 6. https://doi.org/10.4324/9780203884713.ch28
Calleo, Y., & Pilla, F. (2023). Delphi-based future scenarios: A bibliometric analysis of climate change case studies. Futures, 149(March), 103143. https://doi.org/10.1016/j.futures.2023.103143
Cech, F., & Tellioglu, H. (2019). Impact of the Digital Transformation: An Online Real-Time Delphi Study. http://arxiv.org/abs/1904.11411
Chuenjitwongsa, S. (2017). How to conduct a Delphi study. Wales Deanary, 27(1173), 639–643. https://meded.walesdeanery.org/sites/default/files/how_to_conduct_a_delphistudy.pdf
Cuhls, K. (2015). Models of Horizon Scanning: how to integrato Horizon Scanning into European Research and Innovation Policies. May 2016, 10–12. https://doi.org/10.13140/RG.2.1.1938.7766
Dalkey, N. C. (1969). Dalkey, 1969.pdf.
Dalkey, N. C. (2003). The Delphi Methodology. Available from: Www. Fernuni-Hagen. de/ZIFF/v2-Ch45a. Htm [17 January 2005]. http://www.optimizationgroup.com/wp-content/uploads/2013/02/Idea-Generation-Turning-Questions-Into-Answers.pdf
Delaney, K. (2014). INNOVATION TOOL KIT A practical guide : Introduction to Horizon Scanning in the public sector. Innovation, 55(5), 9–10.
Fink-Hafner, D., Dagen, T., Dousak, M., Novak, M., & Hafner-Fink, M. (2019). Delphi Method: Strengths and Weaknesses. Metodoloski Zvezki, 16(2), 1–19. https://doi.org/10.51936/FCFM6982
Galanis, P. (2018). The Delphi method. Archives of Hellenic Medicine, 35(4), 564–570. https://doi.org/10.4324/9781315728513-10
Glenn, J. C. (2004). Introduction to the Futures Research Methods Series. Futures Research Methodology – Version 2.0, 1–61. http://mp.cim3.net/file/project/mp-sofi-sd/reference/01-Introduction.PDF
Grime, M. M., & Wright, G. (2016). D elphi Method . Wiley StatsRef: Statistics Reference Online, August, 1–6. https://doi.org/10.1002/9781118445112.stat07879
Heinonen, S., Karjalainen, J., Ruotsalainen, J., & Steinmüller, K. (2017). Surprise as the new normal – implications for energy security. European Journal of Futures Research, 5(1), 1–13. https://doi.org/10.1007/s40309-017-0117-5
Hideg, É., Mihók, B., Gáspár, J., Schmidt, P., Márton, A., & Báldi, A. (2021). Assessment in Horizon Scanning by various stakeholder groups using Osgood’s semantic differential scale – A methodological development. Futures, 126(October 2020), 1–14. https://doi.org/10.1016/j.futures.2020.102677
Hines, P., Hiu Yu, L., Guy, R. H., Brand, A., & Papaluca-Amati, M. (2019). Scanning the horizon: A systematic literature review of methodologies. BMJ Open, 9(5), 1–9. https://doi.org/10.1136/bmjopen-2018-026764
Hirschhorn, F. (2019). Reflections on the application of the Delphi method: lessons from a case in public transport research. International Journal of Social Research Methodology, 22(3), 309–322. https://doi.org/10.1080/13645579.2018.1543841
Hofer, J., Walter, D., & Knackstedt, R. (2022). Designing a Tool to conduct Real-Time Delphi Studies in Research Projects. (Online-)Proceedings of the 17th International Conference on Design Science Research in Information Systems and Technology (DESRIST), June, 3–9.
Hsu, C. C., & Sandford, B. A. (2007). The Delphi technique: Making sense of consensus. Practical Assessment, Research and Evaluation, 12(10), 1–8.
Linstone, Harold A., M. T. and O. H. (2002). The Delphi method: An efficient procedure to generate knowledge (Issue 8). https://doi.org/10.1007/s00256-011-1145-z
Mayer, J. H., Steinecke, N., Quick, R., & Weitzel, T. (2013). More applicable environmental scanning systems leveraging “modern” information systems. Information Systems and E-Business Management, 11(4), 507–540. https://doi.org/10.1007/s10257-012-0207-7
Palomino, M. A., Bardsley, S., Bown, K., de Lurio, J., Ellwood, P., HollandSmith, D., Huggins, B., Vincenti, A., Woodroof, H., & Owen, R. (2012). Webbased Horizon Scanning: Concepts and practice. Foresight, 14(5), 355–373. https://doi.org/10.1108/14636681211269851
Peter Bishop, B. (2009). Horizon Scanning Why is it so hard?
Popper, R. (2008). How are foresight methods selected? Foresight, 10(6), 62–89. https://doi.org/10.1108/14636680810918586
Puglisi, M. (2001). the Study of the Futures : an Overview of. 463, 439–463.
Rowe, G., & Wright, G. (2011). The Delphi technique: Past, present, and future prospects – Introduction to the special issue. Technological Forecasting and Social Change, 78(9), 1487–1490. https://doi.org/10.1016/j.techfore.2011.09.002
Ruotsalainen, J. (2018). Scanning the shape of journalism—Emerging trends, changing culture? Futures, 104(February), 14–24. https://doi.org/10.1016/j.futures.2018.06.011
Sablatzky, T. (2022). Peer Reviewed Article Methods Moment : The Delphi Method Peer Reviewed Article. Hypothesis, 34(1), 1–6.
Saritas, O., & Smith, J. E. (2011). The Big Picture – trends, drivers, wild cards, discontinuities and weak signals. Futures, 43(3), 292–312. https://doi.org/10.1016/j.futures.2010.11.007
Searchinger, T., Richard, W., Craig, H., & Janet, R. (2018). Creating a Sustainable Food Future; A Menu of Solutions to Feed Nearly 10 Billion People by 2050. Worlds Resources Institute, December, 96. https://research.wri.org/sites/default/files/2019-07/creating-sustainable-food-future_2_5.pdf
Shang, Z. (2023). Use of Delphi in health sciences research: A narrative review. Medicine (United States), 102(7), E32829. https://doi.org/10.1097/MD.0000000000032829
Tapio, P., Rintamäki, H., Rikkonen, P., & Ruotsalainen, J. (2017). Pump, boiler, cell or turbine? Six mixed scenarios of energy futures in farms. Futures, 88, 30–42. https://doi.org/10.1016/j.futures.2017.03.003
Varndell, W., Fry, M., & Elliott, D. (2021). MUITO DETALHADO Applying real-time Delphi methods: BMC Nursing, 20(1), 1–11.
Wintle, B. C., Kennicutt II, M. C., & Sutherland, W. J. (2020). Scanning horizons in research, policy and practice. Conservation Research, Policy and Practice, 29–47. https://doi.org/10.1017/9781108638210.003
Withanaarachchi, A. S., Nanayakkara, J., & Pushpakumara, C. (2015). Delphi as a Methodology for Eliciting Expert Opinion — Important Factors to Be Considered. SSRN Electronic Journal, July 2015. https://doi.org/10.2139/ssrn.2632765
Zartha, J. W., Manrique, J., & Palacio, J. C. (2020). Delphi Method Technologies in Emerging. November.