DOI:10.6531/JFS.201806.22(4).0004
Journal of Futures Studies, June 2018, 22(4): 45–60
A R T I C L E
An Analysis and Categorization of Scenario Planning Scholarship from 1995-2016
Thomas J. Chermack, Colorado State University USA
Abstract
This article presents an analysis and categorization of scenario planning scholarship over the last 20 years. The analysis reports the literature search criteria, followed by a general division of scenario planning scholarship into theoretical / conceptual work and data-driven inquiry. The data-driven scholarship is further classified as case study, qualitative research or quantitative research. Generally, over half of the published refereed articles over the last 20 years have been theoretical / conceptual work. Further, of the data-driven research, the vast majority is case study in nature. Interpretations of these results are provided, as well as suggestions for advancing scenario planning scholar- ship in the future.
Keywords: Scenario planning, Scholarship, Scenario planning research.
Introduction
Scenario planning has grown out of practice. As a result, research has lagged behind although publications focused on scenario planning have risen steadily over the last two decades (Amer, Diam, & Jetter, 2013; Ramirez, Selsky, & van der Heijden, 2010; Varum & Melo, 2010). However, the body of scenario planning scholarship has never been analyzed or categorized beyond several general literature reviews (Chermack, Lynham, & Ruona, 2001; Varum & Melo, 2010). The existing literature reviews, while comprehensive and thorough, primarily engage with the content of scenario planning articles. None of the scenario planning literature reviews attempt to categorize the type and nature of scenario planning scholarship or to map its scholarly landscape.
Pierre Wack, an influential pioneer of scenario planning was clear that scenario planning is a practice that requires a degree of discipline (1984). He stated: “Scenario analysis is a disciplined way to think about the future. It demands above all an understanding of the forces that drive the system, rather than a reliance on forecasts” (Wack, 1984, p.83). According to the Oxford English Dictionary, a discipline can be defined as “An activity that requires mental or physical training” (2017, p.278). There are many examples of Wack’s dedication to mental training as part of his scenario work (Burt, 2010; Chermack, 2017; Kleiner, 2008), and he recommended some form of meditative or contemplative practice to anyone working with scenarios: “I think this is a very personal element. You must find what suits you” (Wack, P. as cited in Chermack, 2017, p.183).
Wack also clarified his hope that scenario planning would become a discipline according to a different definition of the term: “A branch of knowledge, typically one studied in higher education” (2017, p.278). Based on a brief search, it does not appear that scenario planning has become the kind of discipline that can be characterized as a branch of knowledge in institutions of higher education. While the increased attention to scenario planning in the scholarly literature is an important signal regarding the growth of the potential discipline (Cohen & Lloyd, 2000; Glassick, Huber, Maeroff, & Boyer, 1997; Lindholm-Romantschuk, 1998), it is not the only basis on which to judge how much progress has been made (Harley, Acord, Earl-Novel, Lawrence, & King, 2010; Serenko & Bontis, 2013). A closer look at the types of contributions reveals another lens through which to consider the maturity of scenario planning and would allow scholars in the field to identify strategies for continuing to advance the discipline (Harley, Acord, Earl-Novel, Lawrence & King, 2010; Serenko & Bontis, 2013).
An additional important metric of any discipline is the type, nature, quality and rigor of the inquiry conducted by its scholars (Cohen & Lloyd, 2000; Glassick, Huber, Maeroff, & Boyer, 1997). That was the focus of the analysis. The purpose of the article is to present the status of scenario planning scholarship and to categorize the volume of work in the field over the last 20 years. The primary goal was to map the landscape of scenario planning scholarship in terms of its publication outlets, quantity and inquiry type. The purpose of this article was not to engage with the specific content of scenario planning scholarship or to analyze the various themes it may harbor as that task has been accomplished in several comprehensive literature reviews (Chermack, Lynham, & Ruona, 2001; Varum & Melo, 2010).
Research Questions
The following research questions framed the purpose of this analysis:
RQ1: What is the status of scenario planning scholarship according to its quantity and inquiry type, and
RQ2: What scholarship focus areas could advance scenario planning as a discipline in the future?
Literature Search Method
The general approach to answering the research questions involved database searches in order to gather the existing scholarship featuring scenario planning. Analyses were focused on understanding and categorizing the search results according to the frame of this review from 1995 – 2016. An initial search was performed on January 28, 2017 using the “Science Direct” database with search terms “scenario planning” in title, and / or “scenario” in abstract and in subject. The search was limited to peer reviewed articles and 223 results were found. An additional search was performed on January 28, 2017 using the “Business Source Complete” database, again using search terms “scenario planning” in title, and / or “scenario” in abstract and in subject. Again searchers were limited to peer reviewed articles. There were 124 results. Searches were bound from 1995 through the end of 2016 because the databases used were complete from 1995 to present. Articles that were in-press and available online, but not yet assigned to a journal issue number were not included. Results were compared and as there was some overlap between the databases searched. Duplicate results were combined for a total of 257 articles.
Methods
This section describes the strategies for searching, analyzing and categorizing scenario planning scholarship between 1995 and 2016. However, given the vast array of definitions of scenario planning, an important note must first be made about the context and initial selection criteria of the various articles included in the study.
Defining Scenario Planning
Before reporting the results of the analysis and categorization, it is important to acknowledge the lack of agreement among authors and scholars on what constitutes scenario planning. The lack of ability to define scenario planning has been covered by previous scholars (Ramirez, Mukherjee, Vezzoli, & Kramer, 2015) and has led to “methodological chaos” (Martelli, 2001, p.435). As one attempt to situate the results of this study, a list scenario planning definitions is provided in Table 1 (Chermack, Lynham, & Ruona, 2001). For clarity in this research, every article included in the results was screened for a definition or description of scenario planning that was judged to fit at least one of the definitions presented in Table 1. Even so, the problem remains that a clear, consistent and agreed upon definition of scenario planning is yet to be achieved, if it is even possible at all.
Analysis and Categorization Strategy
The first step of the analysis strategy was to generate simple descriptive data about the number of publications that fit the search criteria and arrange them by publication outlet. This was intended as an effort to identify the journals that publish scenario planning scholarship. The second step of the analysis strategy was to consider the type of inquiry. Results were separated into the following two categories: 1) data driven scholarship (quantitative or qualitative), and 2) theoretical or conceptual scholarship (including theorizing, model building and literature reviews). The intent was to generally consider what volume of scenario planning scholarship is based on data collected from participants or populations and what volume is not. These two steps are presented in Table 2.
The third step of the analysis strategy was to take a more granular view of the type of inquiry by further separating the results into the following three categories: 1) quantitative (survey research, industry data with reporting of statistics — post-positivist methodology), 2) qualitative (interview data collected from scenario planning participants with themes reported, constructivist — interpretivist and / or phenomenological methodology), and 3) case studies (reporting of specific applied instances of scenario planning — any relevant methodology).
Results
Table 2 presents a listing of journals and the corresponding number of articles published that met the search criteria in rank order. The table also categorizes scenario planning scholarship by dividing the published material into two categories; 1) data-driven scholarship and 2) theoretical / conceptual scholarship. In order to accomplish this, the abstracts of the 257 articles were reviewed and classified.
Table 3 breaks down the 110 data driven articles from Table 2 into three categories; 1) quantitative research articles, 2) qualitative research articles and 3) case study articles. Table 2 presents data in rank order by the number of data-driven articles. Note that Table 3 only includes journals with data-driven articles. In other words, journals with only theoretical or conceptual articles related to scenario planning were omitted as there is no further analysis required for this categorization. The importance of theoretical / conceptual scholarship is emphasized in the discussion section.
This analysis and categorization provided in Tables 2 and 3 shows that in the last 20 years, a total of 257 peer reviewed scenario planning articles were published that met the search criteria. Of the 257 articles, 147 were theoretical or conceptual articles, and 110 involved the collection of data in some form. Of the 110 peer reviewed articles that involved data collection, 84 were specific case study applications of scenario planning in particular contexts, 17 peer reviewed articles used survey research with the use of statistics in some form and nine peer reviewed articles were based on a qualitative approach featuring the analysis of participant interview data.
Discussion
As an initial attempt to analyze and categorize scenario planning scholarship, this article presents important information about the status and nature of publications on the topic. It is striking how varied the publication outlets are that publish scenario related content. 84 different journals have published at least one peer reviewed article related to scenario planning from 1995 and 2016. This finding demonstrates how widely the general scenario technique has been adopted and applied, suggesting that it is a truly inter- and trans- disciplinary approach. Certainly, scenario planning is modified to suit different content, cultures and contexts, and the utility of exploring multiple possible and plausible futures is clearly relevant to a wide set of problems and related fields.
Theoretical / Conceptual Scholarship on Scenario Planning
Over half of all peer reviewed articles from 1995-2016 (147 out of 257) that focused on scenario planning were theoretical / conceptual works. This finding suggests practitioners and scholars of the field continue to generate new ideas at a rapid rate. These articles included new models of scenario planning, proposed modifications for practicing scenario planning and articles exploring a variety of theoretical foundations. These works are important for advancing a variety possible approaches to the practice of scenario planning, but few are ever moved beyond the conceptual realm and assessed empirically. Theoretical and conceptual works are important contributions to the field that can lead to data-driven research, however they do not represent empirical inquiry into the applied phenomenon. The major issue with the proliferation of theoretical / conceptual works focused on scenario planning is that so few will eventually be investigated. The opportunity and challenge for scenario planning scholars is to advance these theoretical / conceptual works into the empirical domain. However, there are serious challenges in making this transition, which are discussed below.
Case Study Research on Scenario Planning
The majority of data-driven research studies on scenario planning from 1995 – 2016 were case studies. 84 out of 110 data driven research articles used the case study methodology or methods as the major frame. Yin (1981; 2011; 2013) and Stake (1978; 1995; 2005) were the most common structured approaches to case study research, although the majority of studies in this category did not identify a specific method or framework at all. It should be noted that Yin and Stake are the preeminent scholars of case study methods and have made no contributions to scenario planning content. The fact that scenario planning authors using a generic case study approach seem unfamiliar with Yin and Stake may signify that while well-versed in futures methods, many scenario authors may be lacking expertise in research methods. Because of the difficulties encountered in studying scenario planning, it makes logical sense that the case study method may be the best suited inquiry approach, however, cases, by definition are not generalizable to any practically or statistically significant degree (Thompson, 1989; 1995), and poorly conducted case studies do not achieve the level of rigor that will advance scenario planning. No authors discussed the alternative concept of transferability (Slevin & Sines, 2000) which can account for the concept of generalizability in studies with smaller samples that do not meet the criteria of random selection of participants and random assignment to the scenario planning treatment.
Of course, scenario planning carries many contextual factors (Ramirez, Oserman, & Gronquist, 2013; Volkery & Ribeiro, 2009) which can only be addressed by attending to the specifics of each case. Yet, there has been no meta case analysis that has summarized the insights that cut across a majority of cases. This is an additional significant opportunity for contributing to the scholarship of scenario planning. A study that examines the existing scenario planning cases and their conclusions with attention to what might be applicable under similar circumstances would be a major step in collecting the insights that have emerged from the case study approach. Many of the 84 cases used analysis of interview data as their primary feature, blurring the lines between case studies and qualitative studies, so for the purposes of this classification, the 84 case studies were classified as such because the authors defined them as case studies.
Qualitative Research on Scenario Planning
Nine studies of those reviewed were generally qualitative in nature, meaning that the primary research technique involved analyses of interview data, and authors identified these as qualitative studies. None of these nine studies reported a specific research framework or design (e.g., hermeneutic phenomenology, constructivism, social constructivism, etc…). Increased qualitative research provides the greatest opportunity to identify participant experiences in the scenario process and to identify relevant subjects of inquiry which might then be approached with other methods. Qualitative research is often used to gain an understanding of common experiences throughout the process, thereby opening up further lines of study that can be investigated through a variety of other methods and methodologies (Neuman, 2002).
An additional benefit of increased rigorous qualitative research on scenario planning would be a deeper understanding of how participants experience the process. For example, scenario planning is often claimed to be positioned as a tool for generating deep insights about the external environment and / or the organization. Qualitative inquiry would allow researchers to understand the points in the scenario process at which participants experience these insights and allow for further probing. Answers to questions like “what insights are gleaned?, when in the scenario process are participants experiencing insights?, how are they used? and in what context?” can be studied through qualitative research in a way that no quantitative study could ever approach. The opportunity and challenge to scenario planning scholars in this category is to identify specific qualitative approaches and describe their application and data analysis with greater rigor.
Quantitative Research on Scenario Planning
17 quantitative research studies in the pooled databases were published from 1995 – 2016. Nine of these studies were pretest – posttest, treatment and control group designs. None of the studies reviewed used random sampling or random assignment to the treatment or control groups, making all of these quasi-experimental research designs (Campbell & Stanley, 2015; Christensen, 2004; Keppel, 1991). For any scenario planning practitioner, the idea of randomly assigning participants to treatment and control groups is highly unlikely, (suggesting that scenario planning might not be rightly called a ‘science’ until such studies are achieved), however the conduct of such a study would represent a major step forward in terms of general research rigor and robustness of results.
Three of the quantitative studies were general surveys of companies and industries that have used scenario planning. General descriptive data about the frequency of scenario planning use, by how many companies, in which industries, according to which of the many methods with perceived levels of success is valuable information that will provide a better picture of widely used scenario planning practices. However, three articles are simply inadequate to provide a description of general scenario planning use and additional studies like these are desperately needed. The opportunity and challenge for scenario planning scholars in this domain involves the application of advanced research designs and statistics to identify predictor and outcome variables involved in the scenario planning process.
It must be acknowledged that scenario planning is often an iterative, time-consuming process in organizations and studying scenario planning quantitatively poses several challenges. Further, it is generally difficult to engage in intervention research when the intervention (scenario planning) is usually customized, methods are varied as well as the timespans and contexts of different scenario projects. Yet, other disciplines have made advances in areas equally complex and time consuming, such as leadership development, management training programs and various psychology interventions — all have a body of true experimental research studies to support changes in practice. The challenge for scenario planning researchers is to tackle the difficult research designs that may take years to complete, yet will advance the discipline and add to its credibility.
Overall Assessment
Generally, a high degree of theoretical / conceptual activity can indicate a relatively immature discipline (Harley, Acord, Earl-Novel, Lawrence, & King, 2010; Serenko & Bontis, 2013). In this case, a high degree of theoretical / conceptual activity, coupled with a low degree of empirical activity supports the conclusion that scenario planning is an underdeveloped field. 17 quantitative and nine qualitative studies in 20 years indicates a slow pace of empirical progress. When comparing scenario planning to other disciplines, (such as leadership, strategic management, or psychology, among others), the relatively low number of data-driven research over 20 years signal a major gap in the development of scenario planning. Quantitative research represents the greatest opportunity for bringing scientific rigor and credibility to scenario planning and its advancement, while qualitative research represents the greatest tool for describing participant experiences and identifying important research questions. Close and careful, rigorous study of the scenario planning process using a variety of methods has great potential for underscoring the benefits many practitioners may implicitly already know, but lack evidence to support. Still, it is a challenge to scholars of scenario planning to move beyond the substantial body of single case studies.
There is potential for the argument that scenario planning is simply not emphasized enough in any single domain to warrant status as a discipline itself. If there is a low level of interest in determining outcomes of scenario planning, it is unlikely the field will ever evolve (Harley, Acord, Earl-Novel, Lawrence, & King, 2010). In other words, it is possible that scenario planning is a strategic process best positioned as a tool within Strategic Management. However, top management journals (such as the Academy of Management Journal, Organization Science and Administrative Science Quarterly) do not tend to publish material related to scenario planning, suggesting that scenario planning is not strongly positioned there. This suggestion, however, rests on the assumption that scenario planning could be best positioned as a strategic business / organizational process, which certainly may not be true.
Returning to the vast number of journals that have published scholarship focused on scenario planning, there is strong evidence that scenario planning cuts across fields and can rightly be called an inter- or trans- disciplinary intevention. The wide set of application domains and industries serves to support this suggestion, reinforcing the view that the context of scenario planning is critically important.
Limitations
A Google search using the term “scenario planning” yields over 16,500,000 results, making it clear there is an overwhelming body of scenario planning content that is not refereed. While any interpretation of this fact would be purely speculative, it is clear that scenario planning remains largely based in practice with a wealth of perspectives based on practitioner experience, reflection and opinion. The analysis and categorization presented in this article is necessarily limited to peer- reviewed, academic sources given the research questions. The purpose of setting the boundary was specific to the intent of this inquiry — to examine the scholarly literature relevant to the topic with a focus on understanding the progress in gaining evidence to support claims based in practice.
A second limitation is necessarily in the databases reviewed. Additional databases exist that may provide access to other journals publishing content related to scenario planning. However, the databases chosen were the standards for the general content area of scenario planning and the degree of significant additional findings by searching a variety of additional databases was judged to be minimal. “Science Direct” lists their content areas as follows: “Science Direct provides bibliographic and full text content, including indexing and abstracts for scholarly business journals back as far as 1886 and full text journal articles in all disciplines of business, including marketing, management, MIS, POM, accounting, finance and economics” (http://www.sciencedirect.com). “Business Source Complete” lists their content areas as follows: “With premium full-text content and peer-reviewed business journals, this database is an essential tool for business students. It covers all disciplines of business, including marketing, management, accounting, banking, finance and more” (https://www.ebscohost.com/academic/business-source-complete). The search results, however, indicate that these databases reach far beyond solely business publications.
Recommendations for Advancing Scenario Planning
The future of scenario planning rests in the hands and minds of its scholars and practitioners. As an applied discipline, scenario planning cannot exist without practice — and in order for it to mature and advance, scholarship is a necessary component (Swanson & Chermack, 2013; Weber & McCall, 1978). The greatest opportunity for validating practitioner claims (and advancing scenario planning towards a discipline in its own rite) lies in increased quantitative research studies. Many have claimed that scenario planning is more art than science (Godet, 2000; Godet & Roubelat, 2000; O’Brien, 2004; Ogilvy, 2005), and there is minimal science to support the artistic claims. Thus, the evolution of scenario planning requires increased study through more rigorous research methods in order to offer evidence to support claims of scenario planning outcomes. While post-positivism has its limitations, the power of this approach has not yet been adequately leveraged to the extent that outcomes of scenario practice can be anticipated and the associated rigor has not yet been applied.
The optimal approach will continue to see advancements in the application of all forms of inquiry, including quantitative, qualitative and case study research, though the scholarship on the topic of scenario planning is skewed in favor of case studies to date. Again, there are legitimate reasons for the emphasis on case study research, and the alternative methods are under-utilized when thinking about advancing scenario planning as a discipline.
| Author | Date | Definition | Dependent Variables |
| Porter | 1985 | “An internally consistent view of what the future might turn out to be – not a forecast, but one possible future outcome” (Porter, 1985, p.63). | A view of one possible future outcome |
| Schwartz | 1991 | “A tool for ordering one’s perceptions about alternative future environments in which one’s decisions might be played out” (Schwartz, 1991, p.45). | Ordered perceptions about alternative future decision-making envi- ronments |
| Simpson | 1992 | “The process of constructing alternate futures of a business’ external environment” (Simpson, 1992, p.10). | Constructed alternate futures |
| Bloom & Menefee | 1994 | “A description of a possible or probable future” (Bloom & Menefee, 1994, p.223). | A described possible or probable future |
| Collyns | 1994 | “An imaginative leap into the future” (Collyns, 1994, p.275, in 5th discipline field book). | An imagined future |
| Thomas | 1994 | “Scenario planning is inherently a learning process that challenges the comfortable conventional wisdoms of the organization by focusing attention on how the future may be different from the present” (Thomas, 1994, p.6) | Challenged comfortable conventional wisdoms about the future |
| Schoemaker | 1995 | “a disciplined methodology for imagining possible futures in which organizational decisions may be played out” Schoemaker, 1995, p.25) | Imagined possible deci- sion-making futures |
| Van der Heijden | 1997 | External scenarios are “internally consistent and challenging descriptions of possible futures”
An internal scenario is “a causal line of argument, linking an action option with a goal”, or “one path through a person’s cognitive map” (van der Heijden, 1997, p.5) |
Descriptions of possible futures
Explicit cognitive maps |
| De Gues | 1997 | “Tools for foresight-discussions and documents whose purpose is not a prediction or a plan, but a change in the mindset of the people who use them” (De Gues, 1997, p.46) | Changed mindsets |
| Ringland | 1998 | “That part of strategic planning which relates to the tools and technologies for managing the
uncertainties of the future” (Ringland, 1998, p.83). |
Managed future uncer- tainties |
| Bawden | 1998 | “Scenario planning is one of a number of foresighting techniques used in the strategic development of organizations, which exploit the remarkable capacity of humans to both imagine and to learn from what is imagined”. (University of Western Australia, GBN) | Human imagination and learning made explicit |
Table 1. Definitions of scenario planning (from Chermack & Lynham, 2002).
| Author | Date | Definition | Dependent Variables |
| Fahey & Randall | 1998 | “Scenarios are descriptive narratives of plausible alternative projections of a specific part of the future” (Fahey & Randall, 1998, p.6) | Plausible alternative projections of a specific part of the future |
| Alexander & Serfass | 1998 | “Scenario planning is an effective futuring tool that enables planners to examine what is likely and what is unlikely to happen, knowing well that unlikely elements in an organization are those that can determine its relative success” (Alexander & Serfass, 1998, p.35) | Examined future likeli- hoods and unlikelihoods |
| Tucker | 1999 | “Creating stories of equally plausible futures and planning as though any one could move forward” (Tucker, 1999, p.70). | Stories of equally plau- sible futures that inform planning |
| Kahane | 1999 | “A series of imaginative but plausible and well- focused stories of the future” (Kahane, 1999, p.511 in 5th discipline field book). | Plausible stories of the future |
| Kloss | 1999 | “Scenarios are literally stories about the future that are plausible and based on analysis of the interaction of a number of environmental variables” (Kloss, 1999, p.73) | Informed, plausible sto- ries about the future |
| Wilson | 2000 | “Scenarios are a management tool used to improve the quality of executive decision making and help executives make better, more resilient strategic decisions” (Wilson, 2000, p.24) | Improved executive stra- tegic decision-making |
| Godet | 2001 | “A scenario is simply a means to represent a future reality in order to shed light on current action in view of possible and desirable futures” (Godet, 2001, p.63) | A represented future reality |
Table 2. Number of scenario planning articles by publication with categorization into data driven and theo- retical / conceptual articles.
| Publication Title | Total Number of Articles | Data-Driven | Theoretical / Conceptual | |
| 1 | Futures | 38 | 15 | 23 |
| 2 | Technological Forecasting and Social Change | 28 | 7 | 21 |
| 3 | Energy | 23 | 8 | 15 |
| 4 | Transportation Research | 11 | 2 | 9 |
| 5 | Landscape and Urban Planning | 8 | 7 | 1 |
| 6 | Long Range Planning | 8 | 3 | 5 |
| 7 | Applied Energy | 7 | 3 | 4 |
| 8 | Energy Policy | 7 | 3 | 5 |
| 9 | European Journal of Operational Research | 7 | 3 | 4 |
| Publication Title | Total Number of Articles | Data-Driven | Theoretical / Conceptual | |
| 10 | Environmental Modelling & Software | 4 | 2 | 2 |
| 11 | Harvard Business Review | 4 | 3 | 2 |
| 12 | International Journal of Production Economics | 4 | 1 | 3 |
| 13 | Journal of Environmental Management | 4 | 4 | 0 |
| 14 | Omega | 4 | 2 | 2 |
| 15 | Expert Systems with Applications | 3 | 1 | 2 |
| 16 | European Journal of Training and Development | 3 | 3 | 0 |
| 17 | Human Resource Development Quarterly | 3 | 3 | 0 |
| 18 | International Journal of Forecasting | 3 | 1 | 2 |
| 19 | Marine Policy | 3 | 3 | 0 |
| 20 | Procedia Engineering | 3 | 2 | 1 |
| 21 | Resources, Conservation and Recycling | 3 | 1 | 2 |
| 22 | Accident Analysis & Prevention | 2 | 3 | 0 |
| 23 | Applied Soft Computing | 2 | 2 | 1 |
| 24 | Biological Conservation | 2 | 2 | 0 |
| 25 | Climate Risk Management | 2 | 0 | 2 |
| 26 | Computers & Industrial Engineering | 2 | 0 | 2 |
| 27 | Computers & Operations Research | 2 | 0 | 2 |
| 28 | Energy for Sustainable Development | 2 | 2 | 0 |
| 29 | International Journal of Electrical Power & Energy Systems | 2 | 0 | 2 |
| 30 | Journal of Hydrology | 2 | 0 | 2 |
| 31 | Journal of the American College of Radiology | 2 | 2 | 0 |
| 32 | Land Use Policy | 2 | 2 | 0 |
| 33 | Ocean & Coastal Management | 2 | 2 | 0 |
| 34 | Oceanologia | 2 | 2 | 0 |
| 35 | Sloan Management Review | 2 | 0 | 2 |
| 36 | Water Research | 2 | 1 | 1 |
| 37 | Agricultural Water Management | 1 | 0 | 1 |
| 38 | Applied Acoustics | 1 | 0 | 1 |
| 39 | Applied Geography | 1 | 1 | 0 |
| 40 | Applied Mathematical Modeling | 1 | 0 | 1 |
| 41 | Business Horizons | 1 | 0 | 1 |
| Publication Title | Total Number of Articles | Data-Driven | Theoretical / Conceptual | |
| 42 | California Management Review | 1 | 1 | 0 |
| 43 | Case Studies on Transport Policy | 1 | 1 | 0 |
| 44 | Computer Communications | 1 | 0 | 1 |
| 45 | Computers & Education | 1 | 0 | 1 |
| 46 | Computers in Industry | 1 | 1 | 0 |
| 47 | Computers, Environment & Urban Systems | 1 | 1 | 0 |
| 48 | Conservation Biology | 1 | 1 | 0 |
| 49 | Ecological Economics | 1 | 1 | 0 |
| 50 | Ecological Engineering | 1 | 0 | 1 |
| 51 | Ecological Indicators | 1 | 1 | 0 |
| 52 | Ecosystem Services | 1 | 1 | 0 |
| 53 | Electric Power Systems Research | 1 | 0 | 1 |
| 54 | Energy Procedia | 1 | 1 | 0 |
| 55 | Environmental Impact Assessment Review | 1 | 0 | 1 |
| 56 | Environmental Science & Policy | 1 | 1 | 0 |
| 57 | Forest Policy and Economics | 1 | 0 | 1 |
| 58 | Global Environmental Change | 1 | 0 | 1 |
| 59 | Habitat International | 1 | 0 | 1 |
| 60 | Health Policy | 1 | 0 | 1 |
| 61 | International Journal of Naval Architecture and Ocean Engineering | 1 | 0 | 1 |
| 62 | International Journal of Sustainable Built Environment | 1 | 0 | 1 |
| 63 | Journal for Nature Conservation | 1 | 0 | 1 |
| 64 | Journal of Business Research | 1 | 1 | 0 |
| 65 | Journal of Great Lakes Research | 1 | 1 | 0 |
| 66 | Journal of Loss Prevention in the Process Industries | 1 | 0 | 1 |
| 67 | Measurement | 1 | 0 | 1 |
| 68 | Policy Focus Report | 1 | 0 | 1 |
| 69 | Political Geography Quarterly | 1 | 0 | 1 |
| 70 | Preventative Veterinary Medicine | 1 | 1 | 0 |
| 71 | Procedia – Social and Behavioral Sciences | 1 | 1 | 0 |
| 72 | Procedia Economics and Finance | 1 | 1 | 0 |
| Publication Title | Total Number of Articles | Data-Driven | Theoretical / Conceptual | |
| 73 | Reliability Engineering & System Safety | 1 | 0 | 1 |
| 74 | Robotics and Autonomous Systems | 1 | 0 | 1 |
| 75 | Science of the Total Environment | 1 | 0 | 1 |
| 76 | Simulation Modelling Practice and Theory | 1 | 0 | 1 |
| 77 | Socio-Economic Planning Sciences | 1 | 0 | 1 |
| 78 | Space Policy | 1 | 0 | 1 |
| 79 | Speech Communication | 1 | 0 | 1 |
| 80 | Systems Engineering – Theory & Practice | 1 | 0 | 1 |
| 81 | Tsinghua Science & Technology | 1 | 0 | 1 |
| 82 | Tunnelling and Underground Space Technology | 1 | 0 | 1 |
| 83 | Water Policy | 1 | 1 | 0 |
| 84 | World Futures | 1 | 0 | 1 |
| Total | 257 | 110 | 147 |
Table 3. Categorization of data-driven articles into quantitative, qualitative and case study approaches.
| Publication Title | Number of Data-Driven Articles | Quantitative Studies | Qualitative Studies | Case Studies | |
| 1 | Futures | 15 | 3 | 3 | 9 |
| 2 | Energy | 8 | 2 | 2 | 4 |
| 3 | Landscape and Urban Planning | 7 | 0 | 2 | 5 |
| 4 | Technological Forecasting and Social Change | 7 | 2 | 1 | 4 |
| 5 | Journal of Environmental Management | 4 | 0 | 0 | 4 |
| 6 | Accident Analysis & Prevention | 3 | 0 | 0 | 3 |
| 7 | Applied Energy | 3 | 0 | 0 | 3 |
| 8 | Energy Policy | 3 | 0 | 0 | 3 |
| 9 | European Journal of Operational Research | 3 | 2 | 0 | 1 |
| 10 | European Journal of Training and Development | 3 | 3 | 0 | 0 |
| 11 | Harvard Business Review | 3 | 0 | 0 | 3 |
| 12 | Human Resource Development Quarterly | 3 | 3 | 0 | 0 |
| 13 | Long Range Planning | 3 | 1 | 0 | 2 |
| 14 | Marine Policy | 3 | 0 | 0 | 3 |
| Publication Title | Number of Data-Driven Articles | Quantitative Studies | Qualitative Studies | Case Studies | |
| 15 | Applied Soft Computing | 2 | 0 | 0 | 2 |
| 16 | Biological Conservation | 2 | 0 | 0 | 2 |
| 17 | Energy for Sustainable Development | 2 | 0 | 0 | 2 |
| 18 | Environmental Modelling & Software | 2 | 0 | 0 | 2 |
| 19 | Journal of the American College of Radiology | 2 | 0 | 0 | 2 |
| 20 | Land Use Policy | 2 | 0 | 0 | 2 |
| 21 | Ocean & Coastal Management | 2 | 0 | 0 | 2 |
| 22 | Oceanologia | 2 | 0 | 0 | 2 |
| 23 | Omega | 2 | 0 | 1 | 1 |
| 24 | Procedia Engineering | 2 | 0 | 0 | 2 |
| 25 | Transportation Research | 2 | 0 | 0 | 2 |
| 26 | Water Research | 1 | 0 | 0 | 1 |
| 27 | Applied Geography | 1 | 0 | 0 | 1 |
| 28 | California Management Review | 1 | 0 | 0 | 1 |
| 29 | Case Studies on Transport Policy | 1 | 0 | 0 | 1 |
| 30 | Computers in Industry | 1 | 0 | 0 | 1 |
| 31 | Computers, Environment & Urban Systems | 1 | 0 | 0 | 1 |
| 32 | Conservation Biology | 1 | 0 | 0 | 1 |
| 33 | Ecological Economics | 1 | 0 | 0 | 1 |
| 34 | Ecological Indicators | 1 | 0 | 0 | 1 |
| 35 | Ecosystem Services | 1 | 0 | 0 | 1 |
| 36 | Energy Procedia | 1 | 0 | 0 | 1 |
| 37 | Environmental Science & Policy | 1 | 0 | 0 | 1 |
| 38 | Expert Systems with Applications | 1 | 0 | 0 | 1 |
| 39 | International Journal of Forecasting | 1 | 1 | 0 | 0 |
| 40 | International Journal of Production Economics | 1 | 0 | 0 | 1 |
| 41 | Journal of Business Research | 1 | 0 | 0 | 1 |
| 42 | Journal of Great Lakes Research | 1 | 0 | 0 | 1 |
| 43 | Preventative Veterinary Medicine | 1 | 0 | 0 | 1 |
| 44 | Procedia – Social and Behavioral Sciences | 1 | 0 | 0 | 1 |
| 45 | Procedia Economics and Finance | 1 | 0 | 0 | 1 |
| Publication Title | Number of Data-Driven Articles | Quantitative Studies | Qualitative Studies | Case Studies | |
| 46 | Resources, Conservation and Recycling | 1 | 0 | 0 | 1 |
| 47 | Water Policy | 1 | 0 | 0 | 1 |
| Total | 110 | 17 | 9 | 84 |
Correspondence
Thomas J. Chermack is a Professor of Organizational Learning, Performance and Change, and Director of the Scenario Planning Institute at Colorado State University. He can be contacted at: Chermack@colostate.edu
References
Amer, M., Daim, T. U., & Jetter, A. (2013). A review of scenario planning. Futures, 46, 23-40.
Burt, G. (2010). Revisiting and extending our understanding of Pierre Wack’s the gentle art of re-perceiving. Technological Forecasting and Social Change, 77(9), 1476-1484.
Campbell, D. T., & Stanley, J. C. (2015). Experimental and quasi-experimental designs for research. Ravenio Books.
Chermack, T. J., (2017). Foundations of scenario planning: The story of Pierre Wack. New York, NY: Routledge.
Chermack, T. J., & Lynham, S. A. (2002). Definitions and outcome variables of scenario planning.
Human Resource Development Review, 1(3), 366-383.
Chermack, T. J., Lynham, S. A., & Ruona, W. E. A. (2001). A review of scenario planning literature.
Futures Research Quarterly, 17(2), 7-31.
Christensen, L. B. (2004). Experimental methodology. Allyn & Bacon.
Cohen, E., & Lloyd, S. “Disciplinary Evolution and the Rise of Transdiscipline” (PDF). Informing Science: the International Journal of an Emerging Transdiscipline.
Glassick, C. E., Huber, M. T., Maeroff, G. I., & Boyer, E. L. (1997). Scholarship assessed (Vol. 9).
San Francisco: Jossey-Bass.
Godet, M. (2000). The art of scenarios and strategic planning: Tools and pitfalls. Technological fore- casting and social change, 65(1), 3-22.
Godet, M., & Roubelat, F. (2000). Scenario planning: An open future. Technological Forecasting and Social Change, 65(1), 1-2.
Harley, D., Acord, S. K., Earl-Novell, S., Lawrence, S., & King, C. J. (2010). Assessing the future landscape of scholarly communication: An exploration of faculty values and needs in seven disciplines. Center for Studies in Higher Education.
Keppel, G. (1991). Design and analysis: A researcher’s handbook. Prentice-Hall, Inc.
Lindholm-Romantschuk, Y. (1998). Scholarly book reviewing in the social sciences and humanities.
The flow of ideas within and among disciplines. Westport, Connecticut: Greenwood Press.
Neuman, L. W. (2002). Social research methods: Qualitative and quantitative approaches.
O’Brien, F. A. (2004). Scenario planning––lessons for practice from teaching and learning. Europe- an Journal of Operational Research, 152(3), 709-722.
Ogilvy, J. (2005). Scenario planning, art or science?. World Futures, 61(5), 331-346.
Ramirez, R., Mukherjee, M., Vezzoli, S., & Kramer, A. M. (2015). Scenarios as a scholarly method- ology to produce “interesting research”. Futures, 71, 70-87.
Ramírez, R., Österman, R., & Grönquist, D. (2013). Scenarios and early warnings as dynamic capa- bilities to frame managerial attention. Technological Forecasting and Social Change, 80(4), 825-838.
Ramírez, R., Selsky, J. W., & Van Der Heijden, K. (Eds.). (2010). Business planning for turbulent times: new methods for applying scenarios. London: Earthscan.
Serenko, A. & Bontis, N. (2013). The intellectual core and impact of the knowledge management academic discipline. Journal of Knowledge Management, 17(1), 137-155.
Slevin, E., & Sines, D. (2000). Enhancing the truthfulness, consistency and transferability of a qual- itative study: utilising a manifold of approaches. Nurse Researcher, 7(2), 79-98.
Stake, R. E. (1978). The Case Study Method in Social Inquiry 1. Educational Researcher, 7(2), 5-8. Stake, R. E. (1995). The art of case study research. Sage.
Stake, R. E. (2005). Qualitative case studies.
Swanson, R. A., & Chermack, T. J. (2013). Theory building in applied disciplines. Berrett-Koehler Publishers.
Thompson, B. (1989). Meta-Analysis of Factor Structure Studies: A Case Study Example with Bern’s Androgyny Measure. The Journal of Experimental Education, 57(2), 187-197.
Thompson, B. (1995). Exploring the replicability of a study’s results: Bootstrap statistics for the multivariate case. Educational and Psychological Measurement, 55(1), 84-94.
Varum, C. A., & Melo, C. (2010). Directions in scenario planning literature–A review of the past decades. Futures, 42(4), 355-369.
Volkery, A., & Ribeiro, T. (2009). Scenario planning in public policy: Understanding use, impacts and the role of institutional context factors. Technological forecasting and social change, 76(9), 1198-1207.
Wack, P. (1984d). Scenarios: The Gentle Art of Re-perceiving, Harvard Business School Working Paper 9-785-042, December 1984.
Weber, G. H., & McCall, G. J. (Eds.). (1978). Social scientists as advocates: Views from the applied disciplines (Vol. 4). Sage Publications, Inc.
Yin, R. K. (1981). The case study crisis: Some answers. Administrative Science Quarterly, 26(1), 58-65.
Yin, R. K. (2011). Applications of case study research. Sage.
Yin, R. K. (2013). Case study research: Design and methods. Sage publications.