Introduction

The purpose of this document is to provide an overview of the methods and sources that have been utilized to synthesize the Annual Report. The Annual Report of the Strategic Monitor 2018-2019 is oriented around the themes of international peace & security and societal stability. To ensure that analysis remains consistent over different reports, and to facilitate the identification of trends over time, the research design for the Annual Report is codified within a conceptual framework which will remain consistent across future iterations.

The framework applied within the Annual Report contains four recurring sections; namely: (1) Geodynamics, which appraises system-level trends, (2) Threat Environment, which details (external) developments that have arisen and which pertain to the Netherlands’ vital national interests; (3) Developments in the International Order, which evaluates the context within which threats and opportunities manifest; and (4) the Position of the Netherlands and Partnerships, which operationalizes the Netherlands’ position within the international order, developments within this dimension, and which countries it partners with.

We employ a multimethod approach that draws on a wide-ranging analytical toolkit, including horizon scanning, multi-dimensional index construction, econometric analysis, structured focused comparison, trend analysis, risk assessment, and trend impact analysis (see Table 1). This document consists of an overview of the methodological design and rationale behind each of the previously outlined components of the Strategic Monitor: Geodynamics, Threat Environment, Developments in the International Order, and Position of the Netherlands & Partnerships. The subsequent annexes provide further insight into the methods and data sources utilized to operationalize the research designs corresponding to the aforementioned components of the Strategic Monitor.

Table 1
Overview of employed methodologies

Method

Description

Horizon Scanning

Horizon scanning methodology relies on the manual monitoring of hundreds of validated foresight resources, the structured coding of signals, and expert assessment. The horizon scanning exercise is geared largely towards establishing likelihood of manifestation and likely impact.[1]

Multi-dimensional Index Building

Multi-dimensional index building provides a quantitative overview of phenomena on the basis of a predefined array of indicators.[2]

Structured Focused Comparison

The structured focused comparison approach employs a structurally consistent evaluation of phenomena with systematic scope boundaries.[3]

Econometric Analysis

Econometric analysis applies statistical and mathematical models. Within this framework, quantitative data is utilized to develop theories, test existing hypotheses, and to forecast future trends on the basis of historical data.[4]

Trend Impact Analysis

Trend impact analysis is characterized by structured multi-year tracking of emerging and manifesting phenomena.[5]

Geodynamics

This section focuses on what previous Strategic Monitors have dubbed geodynamics,[6] as a more neutral and broader term than the more familiar – but also more value-laden – geopolitics. The geodynamics component of the Strategic Monitor measures system-level trends as they present within the economic, military, political, societal, and judicial domains.[7] While metrics which capture dynamics within the global economy are relatively commonplace – with the majority of yearly overviews of economic trends being heavily based on datasets such as GDP growth, debt to GPD ratio, and consumer confidence – precious few analogous metrics exist for monitoring the overall state of the international political system. Data availability within this field continues to be limited to structural datasets such Freedom House’s Press Freedom Index, Systemic Peace’s Polity IV Index (both of which operationalize various aspects of democracy), as well as datasets such as those made available by Amnesty International and the World Bank Group (geared towards operationalizing human rights and, the rule of law – among others) and the International Institute for Strategic Studies (which are geared towards operationalizing military capabilities).

While the contemporary international environment is characterized by fundamental changes presenting and/or developing at a breakneck pace, structural datasets can be universally associated with the drawback of staggered update rates. This means that reported variable values within these datasets typically lag one (and in many cases, two) years behind the present-day. Given that – under ideal conditions – the geodynamics component of the Strategic Monitor should facilitate up-to-date trend analysis, this also means that structural datasets require augmentation to yield (within the context of this study) sufficiently valuable insights. This is achieved through the methodology’s adherence to a tri-pronged approach to evaluating structural data, which can be summarized as follows:

1.
Evaluation of structural components. By drawing on 10 years of historical (structural) data, the Strategic Monitor offers insights into trends pertaining to a number of key (read: geo-economic, geo-political, geo-societal, geo-judicial, and geo-identitarian) aspects of geodynamics;
2.
Dynamic components. Dynamic components are evaluated on the basis of a.) a 10-year (backwards-looking) perspective to the more recent ‘present’, and b.) trends observed over the course of two years preceding the present. This includes event data (derived from ICEWS, GDELT, etc.), which tracks global news coverage in real-time;
3.
Dyadic risk. The Strategic Monitor’s dyadic risk analysis focuses on gauging the risk of interstate conflict on the basis of various datasets. The employed methodology utilizes event data derived from the GDELT and ICEWS datasets (among others) to train a machine learning algorithm on a.) CAMEO codes, and b.) the HIHOST variable contained within the Militarized Interstate Conflict datasets, with the goal of forecasting conflict risk on a dyad-by-dyad basis.

The combination of these three components facilitates the process of tracking the state of geodynamics based on data insights. Insights derived from the geodynamics component of the Strategic Monitor further inform the qualitatively-based observations presented within the development of the international order component.

Threat Environment

The threat environment component of the Strategic Monitor aims to monitor threats and opportunities. The overarching goal is to present a holistic view of the Netherlands’ security environment by emphasizing (among others) external developments, the threats they give rise to, and their impact on nationally vital interests and values, each of which are evaluated on the basis of a.) their potential impact, and b.) the likelihood of specific threats that may emanate from their development.

Trend Tables

The first step in such an evaluation is to assess the development of phenomena that pertain to the threat environment through a trend table. Trend tables constitute a recurring feature of the Strategic Monitor reports (see Figure 1), and are geared towards providing a holistic, per-theme overview of relevant trends and indicators. Trend tables are constructed through a conceptually hierarchical process, with a top-level phenomenon (hereafter referred to as the concept) being divided into various dimensions. Dimensions serve to split the high-level phenomenon into distinct sub-phenomena. As an example, the interstate military cooperation trend table incorporates dimensions relating to ‘intentions’, ‘capability, and ‘activity’. Every dimension is measured through several contributing dynamics, each of which is operationalized by quantifiable sub-variables.[8] The concepts, dimensions, contributing dynamics presented within trend tables are formulated on the basis of a.) expert opinion and, and b.) a horizon scan.[9] Initial versions of trend tables are subjected to expert critique during expert sessions, after which they are revised on the basis of additional desk research.

To allow for an overall trend evaluation, trends are evaluated against a baseline measurement (a ‘zero measurement’ which captures the status quo on the first date of measurement) at each level. Observed trends are communicated by means of simple trichotomies, which are universally geared towards indicating whether a phenomenon is ‘improving’, ‘stable’, or ‘deteriorating’, and which are defined on a phenomenon-by-phenomenon basis. As a general rule, threat analyses are performed on the basis of foresight analysis in which projections x years into the future are derived from observations spanning 2x years into the past. While the time frame varies per theme, the general goal is to evaluate five years into the future, meaning that each query consults at least ten years of historic data.

Figure 1
Example of trend table, showcasing concept: interstate military competition; dynamics: intentions, capability and activity; and associated sub-dimensions
Example of trend table, showcasing concept: interstate military competition; dynamics: intentions, capability and activity; and associated sub-dimensions

The analysis incorporated within the Strategic Monitor offers a structured, systematic approach to appraising the global threat landscape. Threats may be identified and subjected to further research on the basis of the aforementioned threat analysis. Threats are analyzed on the basis of two metrics; namely: the probability of a threat materializing within the next five years (operationalized through chance of occurrence) and the impact of such a threat materializing in the Netherlands (operationalized through an impact score).

Estimative Probability

As previously outlined, threats are analyzed on the basis of a.) their probability of occurrence, and b.) their potential and/or projected impact. The first of these dimensions of the Strategic Monitor’s threat analysis is operationalized through the application of two distinct criteria; namely:

1.
Causal reasoning underpinning the threat’s manifestation. This variable is derived from a combination of a.) historically supporting evidence, and b.) a reasoned reflection outlining contextual factors which have the potential of impacting the threat’s present-day probability of manifestation, and;
2.
Evaluation of the threat’s probability trend. This exercise is geared towards establishing whether the threat’s probability of manifestation has increased or decreased over time on the basis of a 5-year time evolution. This approach builds on the analysis completed towards the finalization of the trend table by (where possible) incorporating specific trend ‘breaks’ that are of relevance to the threat probability.

The aforementioned assessment culminates in the assigning of threat probability on the basis of the rubric presented in Table 1.

Table 2
Threat probability assessment scheme

Threat Probability

Numerical likelihood

Very Likely

75% chance of occurrence

Likely

37.5% chance of occurrence

Possible

5% chance of occurrence

Unlikely

1% chance of occurrence

Very Unlikely

0.1% chance of occurrence

Impact Score Assessment

The threat probability assessment is supplemented by an evaluation of a threat’s likely impact (should it materialize) on the Netherlands’ vital national interests. Vital interests are defined on the basis of the priorities outlined in official Dutch policy briefs, including the Dutch MFA’s Notitie Geïntegreerde Buitenland- en Veiligheidsstrategie (GBVS) and the 2018 version of the Dutch MoD’s Defensienota,[10] which explicitly identify the Netherlands’ territorial integrity, the health of the international order, and economic stability as vital interests.[11] In-line with these documents, threat impact is evaluated on the basis of three distinct criteria; namely:

1.
Impact on public safety and rule of law (this includes the threat’s projected societal, ecological, physical and digital impact within the Netherlands);
2.
International rule of law (the degree to which the threat’s manifestation violates and/or infringes on a.) the Netherlands’ territorial sovereignty, and b.) international rules and norms), and;
3.
Economic prosperity (the degree to which the threat’s manifestation can be associated with significant costs and/or disruptions to the Dutch economy).

Each of these three criteria are evaluated on the basis of the magnitude of damage, with a distinction being made between limited, significant, or severe. Magnitude of damage is established through the application of two criteria; namely:[12]

1.
Causal reasoning underpinning the threat’s manifestation. This variable is derived from a combination of a.) historically supporting evidence, and b.) a reasoned reflection outlining contextual factors which have the potential of impacting the threat’s present-day magnitude of damage, and;
2.
Evaluation of the threat’s magnitude of damage trend. This exercise is geared towards establishing whether the threat’s magnitude of damage has increased or decreased over time on the basis of a 5-year time evolution. This approach builds on the analysis completed towards the finalization of the trend table by (where possible) incorporating specific trend ‘breaks’ that are of relevance to the threat’s magnitude of damage.

Developments in the International Order

Given the fact that the threats evaluated in the previous section occur within an international environment, the degree to which a threat may have spillover effects and/or require a multilateral solution to manage and mitigate requires an evaluation of the state of the international order. In addition to incorporating analyses pertaining to geodynamics and the Netherlands’ threat environment, the Strategic Monitor features an analysis of developments in the international order. In concrete terms, this component of the Strategic Monitor is geared towards establishing the degree to which the international order exhibits signs of convergence (or diversion), whether – on the basis of observed instances of cooperation or confrontation – the international order is equipped to tackle overarching threats.

The methodology featured within the Strategic Monitor views the international order through the lenses of separate policy domains (hereafter defined as international regimes). It takes the form of a structured focused comparison. A structured focused comparison is ‘structured’ because a systematic set of questions is evaluated on a regime-by-regime basis. It can be considered ‘focused’ because only important elements are assessed (degree of institutionalization, development of and compliance with norms and rules).[13] The methodology utilized within the Strategic Monitor adheres to Stephen Krasner’s definition of international regimes (‘a set of implicit and explicit principles, norms, rules and decision-making procedures around which the actors converge in a particular area of international relations’),[14] and thus affords the concepts of norms and rules central positions within the overall analysis of the international order. Within this context, norms are defined as the standard of appropriate behavior. Rules are defined as concrete actions (such as strategies or information sharing). These concepts are evaluated on the basis of a set of three criteria; namely:

1.
Actors and Institutions (which actors or institutions are most important with respect to the norm or rule, and how is this trend likely to develop?);
2.
Norms and Rules (how has this norm or rule been historically understood and how does it reflect concerns for human rights and western interests?), and;
3.
Adherence and compliance (how has adherence to and/or compliance with the existing set of norms and rules developed over the past 10 years, and is there an overall trend towards conflicting or rather cooperation?).

The application of this methodology results in the development of a trend table which features distinct sections for norms and rules (Figure 2). Much like those utilized in the threat landscape section of the report, the trend tables break observations down into sub-components, each of which are evaluated within the context described above in order to gauge the overall increase (or decrease) in threat.

Figure 2
Trend table for developments in the international order in the European periphery
Trend table for developments in the international order in the European periphery

Following the plotting of the regime into the matrix, Trends identified within the trend tables are further evaluated on the basis of desk research, qualitative reasoning, and judgements rendered during an expert meeting. These evaluations – which contribute to an overall assessment of developments within the international order – are further updated on the basis of an overall assessment. The overall assessment incorporates an expert evaluation of several aspects of the analyzed regime as they have developed over the course of the last 10 years; namely: 1.) the regime’s three most important norms, 2.) the regime’s three most important rules, 3.) the seven most important (whether positive or negative) formal and/or informal debates which have taken place between members of the regime, 4.) the outcomes of the aforementioned meetings (and the implications thereof), and 5.) the seven most important (whether positive or negative) formal and/or informal actions the regime has undertaken. The results of this analysis are subsequently translated into a measure of the state of an international regime and transposed onto an ‘assenkruis’ which features axes that showcase ‘the types of players’ and ‘the degree of cooperation and non-cooperation’ (Figure 3).[15]

Figure 3
Cooperation matrix as conceptualized in the 2010 report Verkenningen; Houvast voor de Krijgsmacht van de Toekomst
Cooperation matrix as conceptualized in the 2010 report Verkenningen; Houvast voor de Krijgsmacht van de Toekomst

Using international regimes that are specific to issue areas represents a disaggregated, grounded approach in which the international order is analyzed on the basis of the subsystems which shape it. The assenkruis facilitates visual analysis of the international order. In concrete terms, the assenkruis can be applied towards the identification of institutionalization within the system, the identification of ‘value clusters’,[16] the projection of likely development paths vis-a-vis norm and rule development, and gauging of state adherence and/or compliance with existing norms and rules on a regime-by-regime basis.

Position of the Netherlands and Partnerships

The final component of the Strategic Monitor concerns itself with the Netherlands’ position in the international order. In concrete terms, this component of the Strategic Monitor is geared towards a.) evaluating the Netherlands on the basis of its position within transnational organizations such as the European Union and NATO, b.) identifying ‘middle powers’, and c.) identifying countries which (on the basis of, among others, shared values) can either be considered (close) partners of the Netherlands or fulfill similar roles as it within the international community. The NATO and EU analysis utilizes a case-study-based approach which is geared towards evaluating interplay between the aforementioned institutions’ various member states. Observations within this component are predominantly qualitative in nature. The adhered-to methodology draws extensively on the analytical framework utilized in the development of the international order section, with the differentiating factor being that it places a strong emphasis on these developments’ (potential) impact on the Netherlands. Analyses concerning partnership and middle powers base themselves primarily on qualitative observations presented within the middle power and partnerships indices. Taken together, these facilitate the evaluation of a country’s importance to the Netherlands. The respective indices are described in further detail below. A comprehensive list of datasets used for this study is provided in Annex 2.

Middle Powers

The purpose of the middle powers study is to identify middle powers on the basis of a set of predefined criteria which operationalize the notion that middle powers are countries which can play an important role across different fields and/or within different international regimes. The adhered-to methodology also facilitates the process of distinguishing between ‘established’ middle powers and ‘emerging’ middle powers.[17] As such, the paper identifies specific middle power partners for the Netherlands across a number of relevant policy domains in the areas of peacekeeping, transnational crime prevention, energy security, and defense (among others).

Middle power status is evaluated on the basis of a set of key characteristics that can be used to identify contemporary middle powers; namely:

1.
Power (the degree to which a state has capacity to project authority, bring weight to the negotiation table, and shape international solutions);
2.
Influence (the degree to which a state is embedded in the international community), and;
3.
Identity (the degree to which a state portrays an identity that signifies a capacity and willingness to act towards diplomatic solutions, is specifically geared towards promoting good international citizenship, and has a reputation for upholding civil and political rights domestically).

Each of these characteristics is evaluated on the basis of a predefined set of exclusion criteria. The resulting method iteratively excludes states from the pool of countries that are considered middle powers. As an example, middle powers are differentiated from great powers through the application of a 5% method, which removes all states that command a ‘dominating’ (5% or higher) share of the world’s total population, economic output, or military power from the pool of potential middle powers. While a full description of the exclusion criteria is beyond the scope of this paper, an overview of the utilized indicators is presented in Table 1.[18] This holistic evaluation results not only in a list of countries that currently meet all the criteria for middle power status, but also in a list of those countries that may be considered ‘emerging middle powers.’ Emerging middle powers are conceptualized as those countries that meet all but one criterion (see Table 3) for middle power status.

Table 3
Middle Power criteria

Characteristic

Criterion

Indicator

Source

Power

Economic power

Gross Domestic Product (GDP) (2017 figures, in absolute terms).

The World Bank.[19]

Power

People power

Population size (2017, in absolute terms).

The World Bank.[20]

Power

Military power

Military expenditure in absolute terms (average figure across five years: 2013-2017).[21]

Stockholm International Peace Research Institute (SIPRI).[22]

Influence

Size of the diplomatic network

Number of embassies and permanent missions worldwide.

Based on counts from Embassy Pages.[23]

Influence

Membership in key international bodies

UNSC (temporary) membership and/or membership of the Human Rights Council.

Based on records from the UNSC Membership Database[24] and UNHRC documents: “List of past members of the Human Rights Council​​”;[25] “Current Membership of the Human Rights Council.”[26]

Identity

Spending on development aid

Contributions to UN Development Program from 2012-2017.

UNDP Transparency Portal.[27]

Identity

Human rights upheld at home

Freedom status according to Freedom House Index 2018. Survey edition (year under review: 2017).

Freedom House.[28]

Partnerships

In a volatile world, significant value can be associated with seeking partners that can help in upholding the international order in its current state. The data-driven approach utilized within this Strategic Monitor yields a global, per-country overview of the degree to which the Netherlands has pursued and/or exhibits signs of partnership with other states between 2008-2017. The adhered-to methodology allows for a wide variety of analyses, ranging from the evaluation of which countries are (currently) closest to the Netherlands, to an overview of which countries exhibit the largest change in partnership scores over time and a per-dimension breakdown of which countries can be considered partners (i.e.: differentiation between economic and diplomatic partners).

The Partnership study draws on qualitative datasets to establish the strength of Netherlands’ mutual ties with other countries. Potential partners are established on the basis of their importance for the Netherlands, as captured in the below-listed indicator array:[29]

1.
Economic relations. Included because trade between states allows for the exchange of goods and services to the benefit of both parties. High trade volumes typically require significant levels of coordination and generate shared interests;
2.
Military relations. Included because military cooperation is based on (temporary) alignment in terms of strategic objectives between two states, and is based on institutional ties and exchange of military goods;
3.
Diplomatic relations. Included because formalized diplomatic ties allow nations to communicate, coordinate actions, and/or settle disputes through peaceful means, and;
4.
Value-based relations. Included because shared values are an expression of ideological affinity, and may facilitate the pursuit of mutual interests.

Because these categories do not necessarily capture all the aspects of partnership, the index also incorporates data relating to official state visits. While this metric differs somewhat from the other indicators utilized, it offers an indication of the Netherlands’ political affinity with the country in question, as it constitutes a public expression of bilateral support.

Each of the operationalized dimensions are ordinally scored from 0-4 (based on a dimension-specific criteria).[30] An overview of the number of points available per category is given below, with a perfect score of 17 constituting the highest degree of partnership and 0 indicating no partnership at all (Table 4). The time period for the aforementioned analysis is 2008-2017.

Table 4
Dimensions and Indicators used to evaluate Dutch partnership

Dimension

Indicator

Calculation

Points available

Economic

Bilateral trade data

% of total value of trade relation with the Netherlands, summing exports and imports together.

4

Military

Dutch arms trade & DFRI data

Presence of arms trade, see detailed description for DFRI data.[31]

4

Diplomatic

Dutch diplomatic presence abroad

Level of diplomatic representation per country

4

Values

Convergence with Dutch values

Freedom House Index divided into 5 categories based on human rights compliance.

4

Special interests

State or Royal visits

Countries that have received or sent a state/royal visit to or from the Netherlands are awarded an extra point.

1

Conclusion

This document has aimed to provide an overview of the methodologies and/or datasets which were used to operationalize the studies featured in this year’s Strategic Monitor, with the specific goal of ensuring that a.) these methodologies are replicable going forward, and b.) the observations presented within the Strategic Monitor are transparent in nature.

The presented Strategic Monitor framework outlines methodologies which are intended to operationalize geodynamics, the threat landscape, the development of the international order, and the position of the Netherlands within that order. Each of these components interacts with the others and it is at the intersection of these topics that a holistic and complete analysis occurs. While each component in the strategic monitor can stand on its own merits when it comes to offering insight, the adhered-to research design facilitates the collective and individual tracking of each component, resulting in a method that is reproducible, and which allows for tracking of trends over multiple years.

As new data and methods become available, the utilized framework will be updated and modified to in order to better capture a world that is dynamic and complex. An overview of all data utilized within the scope of this research is presented in the Annexes 1 and 2. The authors appreciate any comments and/or questions which may arise from this paper.

Bibliography

Bruijne, Kars de. “Vitale Belangen.” The Hague: Clingendael Institute of International Relations, 2018. link.

———. “Vitale Belangen.” The Hague: Clingendael Institute of International Relations, 2018. link.

“Contribution to UNHCR - 2017.” United Nations Human Rights Council, 2018.

De Spiegeleire, Stephan, Yevhen Sapolovych, and Khrystyna Holynska. Things May Not Be as They Seem: Geo-Dynamic Trends in the International System. HCSS Strategic Monitor 2018. The Hague, The Netherlands: The Hague Centre For Strategic Studies, Forthcoming.

“Defensienota 2018: Investeren in Onze Mense, Slagkracht En Zichtbaarheid.” The Hague: Ministerie van Defensie, 2018. link.

“Embassies and Consulates around the World.” EmbassyPages.com, 2019. link.

“Freedom in the World 2018.” Freedom House, January 13, 2018. link.

“GDP (Current US$) | Data.” The World Bank, 2019. link.

George, Alexander L., Andrew Bennett, Sean M. Lynn-Jones, and Steven E. Miller. Case Studies and Theory Development in the Social Sciences. mit Press, 2005.

Glenn, Jerome C., and Theodore J. Gordon. Futures Research Methodology: Version 3.0. Editorial desconocida, 2009.

Gordon, Theodore Jay. “The Methods of Futures Research.” The Annals of the American Academy of Political and Social Science 522, no. 1 (1992): 25–35. link.

Greene, William H. Econometric Analysis. 5th edition. Upper Saddle River, N.J: Prentice Hall, 2002.

Krasner, Stephen D. “Structural Causes and Regime Consequences: Regimes as Intervening Variables.” International Organization 36, no. 2 (1982): 185–205. link.

OECD, and EU DG JRC. “Handbook on Constructing Composite Indicators: Methodology and User Guide.” OECD, 2008. link.

“OHCHR | HRC List of Past Members of the HRC.” United Nations Human Rights Council, 2019. link.

“OHCHR | HRC Membership of the Human Rights Council, 1 January - 31 December 2019.” United Nations Human Rights Council, 2019. link.

Oosterveld, Willem, and Bianca Torossian. “A Balancing Act: The Role of Middle Powers in Contemporary Diplomacy.” The Hague: The Hague Centre For Strategic Studies, 2018. link.

“Overview of Methodologies.” OECD, 2019. link.

“Population, Total | Data.” The World Bank, 2019. link.

“SIPRI Military Expenditure Database | SIPRI.” Stockholm International Peace Research Institute. Accessed January 4, 2019. link.

“Sources and Methods | SIPRI.” Stockholm International Peace Research Institute. Accessed January 4, 2019. link.

Sweijs, Tim, Hugo van Manen, and Paul Verhagen. “The Dutch Foreign Relations Index: Methodological Note (v0.1).” The Hague: The Hague Centre For Strategic Studies, 2017. link.

“United Nations Security Council.” United Nations, 2019. link.

“Verkenningen: Houvast Voor de Krijgsmacht van de Toekomst.” The Hague: Ministerie van Defensie, 2010. link.

“Wereldwijd Voor Een Veilig Nederland: Geïntegreerde Buitenland- En Veiligheidsstrategie 2018-2022.” The Hague: Ministerie van Buitenlandse Zaken, 2018. link.

Annex 1: Data Types

An overview with examples of the different types of data that have been utilized within the Strategic Monitor is provided here. This list is not exhaustive and merely represent examples of the different categories and their respective use cases.

Type

Example

Use case

Structural data

World Bank

Used to measure over time development of properties

Event data

ICEWS (Integrated Crisis Early Warning Unit), GDELT (Global Database on Events, Language and Tone), ACLED

Used to analyze specific instances of types of events

Ordinal Data

World Freedom House Index

Used to compare actors to each other and determine growth/regression

Nominal Data

World Values Survey

Used to identify political regime type

Annex 2: Data Overview

A comprehensive overview of all data used within the Strategic Monitor is provided here. This overview is divided into each separate study and contains sources description and the name of the dataset utilized.

Study

Data Source

Indicator

Dataset

Political Warfare

Council on Foreign Relations (CFR)

State-sponsored cyber-attacks

Cyber operations tracker

European External Action Service (EEAS)

Spreading of disinformation

Disinformation cases database

European Commission

Economic Coercion

EU-Russia trade statistics

Stockholm International Peace Research Institute

Paramilitary activity

SIPRI Arms Industry Database

Association of Former Intelligence Officers (AFIO)

Covert (in)direct action by Russian Intelligence Services

Significant murders or attacks related to Putin’s regime

Vertical tensions: Order

Eurostat

Debt load of national government

Government debt (% GDP)

Eurostat

Deficit of national government

Government deficit (% GDP)

Middle Powers

World Bank

Degree of economic power

Gross Domestic Product (absolute)

World Bank

Size of population

Population (absolute)

Stockholm International Peace Research Institute

Level of military power

Military Expenditure (absolute)

Embassy Pages

Scope of diplomatic presence

Embassies Around the World

United Nations Security Council

Scope of diplomatic influence

Security Council Membership Database

United Nations Human Rights Council

Scope of diplomatic Influence

Current Membership of the Human Rights Council & List of past members of the Human Rights Council​​

United Nations

Development Program

Amount of aid expenditure

Donor Database

Freedom House

Degree of political liberties

Freedom House Index

Political Violence Abroad

Uppsala Conflict Data Program

Number of deaths due to state-based, non-state and one-sided violence.

UCDP Georeferenced Event Dataset (GED)

Global Database of Events, Language and Tone

Number of Protests

GDELT Event Database

The Legatum Institute

Degree of governance

The Legatum Prosperity Index

The World Bank

Presence of Youth Unemployment in populations

Unemployment, youth total.

The Food and Agriculture Organization of the United Nations

Monetary cost of basic food

FAO Food Price Index

The World Bank

Presence of Youth Bulges in a population

Youth bulges, population total.

Interstate Military Competition

Global Database of Events, Language, and Tone

Negative rhetorical military assertiveness (own indicator, based on GDELT codes)

GDELT 2.0

The North Atlantic Treaty Organization

Military spending on equipment (procurement and R&D) as a percentage of GDP

Defence Expenditure of NATO Countries

Heidelberg Institute for International Conflict Research

Instances of violent crises

Heidelberg Conflict Barometer

Stockholm International Peace Research Institute

Amount of Military spending

SIPRI military expenditure database

Uppsala Conflict Data Program and The Peace Research Institute Oslo

Instances of internationalized intrastate conflict

UCDP/PRIO armed conflict dataset

Vertical Tensions

European Commission

Dutch citizens thoughts on Future Economic Situation within the Netherlands

Eurobarometer Interactive

European Commission

EU citizens thoughts on Future Economic Situation of EU

Eurobarometer Interactive

European Commission

Dutch optimism on the future of the EU

Eurobarometer Interactive

European Commission

European Optimism about future of EU

Eurobarometer Interactive

European Commission

EU citizen satisfaction with EU democracy

Eurobarometer Interactive

European Commission

EU citizen satisfaction with national democracy

Eurobarometer Interactive

European Commission

Dutch satisfaction with EU democracy

Eurobarometer Interactive

European Commission

EU Level of trust in the EU institutions

Eurobarometer Interactive

European Commission

Level of trust in national governments- European.

Eurobarometer Interactive

European Commission

Dutch Trust in national Government

Eurobarometer Interactive

Geodynamics

World Bank

Geo-economic indicators

Total Factor Productivity, Trade Volume, Global GDP, global Inequality

Pardee Global Influence

Geo-political indicators

Influence and bandwidth metrics

World Bank

Geo-military indicators

Armed forces personal, military expenditure

SIPRI

Global arms trade volume

SIPRI military expenditure database

UCDP

Number of interstate conflicts

UCDP/PRIO armed conflict database

World Values Survey

Geo-societal indicators

World Values Survey Wave 6

World Bank

Geo-judicial

World Governance Indicators

Google

Search volume for specific terms

Google Trends

GDELT

Event data

GDELT (Trans-lingual)

ICEWS

Event data

ICEWS

Landfill Strategy Group

Economic performance small vs large advances economies

Real % GDP growth dataset

Nexus Terrorism-Organized Crime

United Nations Office on Drugs and Crime (UNODC)

Interaction between terrorists and criminals in prisons

UNODC data portal

National Consortium for the Study of Terrorism and Responses to Terrorism (START)

Successful reintegration of former extremists in society

Global Terrorism Database (GTD)

Migration

Eurostat

Refugee flows to Europe

Resettled persons annual data

FRONTEX

Migrant flows to Europe

Migrants at EU borders annual data

United Nations High Commissioner for Refugees (UNHCR)

Refugee flows worldwide

UNHCR data portal

International Organization for Migration (IOM)

Migrant flows worldwide

Migration data portal

Partnerships

International Monetary Fund

Bilateral trade data

Direction of Trade Statistics

Freedom House

Degree of political liberties

Freedom House Index

The Hague Centre for Strategic Studies & Stockholm International Peace Research Institute

Degree of arms trade

Dutch Foreign Relations Index and SIPRI Military Expenditure Database

Pardee Centre for International Futures & Government of the Netherlands

Dutch diplomatic presence abroad

Diplometrics Project (Diplomatic Representation dataset) & Official Dutch National Government Documentation

Notes

See “Overview of Methodologies,” OECD, 2019, link.
See OECD and EU DG JRC, “Handbook on Constructing Composite Indicators: Methodology and User Guide” (OECD, 2008), link.
See Jerome C. Glenn and Theodore J. Gordon, Futures Research Methodology: Version 3.0 (Editorial desconocida, 2009). See also Alexander L. George et al., Case Studies and Theory Development in the Social Sciences (MIT Press, 2005).
See William H. Greene, Econometric Analysis, 5th edition (Upper Saddle River, N.J: Prentice Hall, 2002).
See Glenn and Gordon, Futures Research Methodology.; see also Theodore Jay Gordon, “The Methods of Futures Research,” The Annals of the American Academy of Political and Social Science 522, no. 1 (1992): 25–35, link.
Geodynamics refers to ongoing dynamic patterns and trends in various key aspects of the international system.
For an overview of the indicators employed to operationalize the domains utilized within the Strategic Monitor’s geodynamics component, see Stephan De Spiegeleire, Yevhen Sapolovych, and Khrystyna Holynska, Things May Not Be as They Seem: Geo-Dynamic Trends in the International System, HCSS Strategic Monitor 2018 (The Hague, The Netherlands: The Hague Centre For Strategic Studies, Forthcoming).
Contributing dynamics are geared towards operationalizing in which ‘direction’ a dimension of the studied phenomenon is moving. As an example, the ‘intentions’ dimension of interstate military cooperation incorporates contributing dynamics relating to ‘negative rhetoric’.
A horizon scan constitutes an in-depth literature review. Horizon scans are geared towards identifying as wide a range of ongoing discourses as possible, and derive insights from academic publications, newspaper articles and/or online publications, reports made available by international organizations, and ‘influencer’ profiles twitter and/or other relevant social media platforms.
See “Wereldwijd Voor Een Veilig Nederland: Geïntegreerde Buitenland- En Veiligheidsstrategie 2018-2022” (The Hague: Ministerie van Buitenlandse Zaken, 2018), link.; see also “Defensienota 2018: Investeren in Onze Mense, Slagkracht En Zichtbaarheid” (The Hague: Ministerie van Defensie, 2018), link.
For a full overview of incorporated policy documents, see Kars de Bruijne, “Vitale Belangen” (The Hague: Clingendael Institute of International Relations, 2018), link.
The adhered-to criteria largely align with the methodology outlined on page 10 of Kars de Bruijne, “Vitale Belangen” (The Hague: Clingendael Institute of International Relations, 2018), link. It should be noted that the currently adhered-to methodology serves only to provide an initial (rough-cut) appraisal of the magnitude of impact, and typically requires further (contextually-based) elaboration.
For more information, see George et al., Case Studies and Theory Development in the Social Sciences.
See Stephen D. Krasner, “Structural Causes and Regime Consequences: Regimes as Intervening Variables,” International Organization 36, no. 2 (1982): 185–205, link.
See “Verkenningen: Houvast Voor de Krijgsmacht van de Toekomst” (The Hague: Ministerie van Defensie, 2010), link.” The matrix was developed for this study, which was commissioned by the Netherlands Ministry of Defense in 2010.
In this context, value clusters refer to groups of countries which can be grouped on the basis of shared norms and rules.
Established middle powers are typically defined as countries which share the Netherlands’ liberal-democratic outlook.
See Willem Oosterveld and Bianca Torossian, “A Balancing Act: The Role of Middle Powers in Contemporary Diplomacy” (The Hague: The Hague Centre For Strategic Studies, 2018), link. for a complete overview of the method document.
See “GDP (Current US$) | Data,” The World Bank, 2019, link.
“Population, Total | Data,” The World Bank, 2019, link.
See “Sources and Methods | SIPRI,” Stockholm International Peace Research Institute, accessed January 4, 2019, link. for an overview of the adhered-to methodology. Note that some figures are estimates.
“SIPRI Military Expenditure Database | SIPRI,” Stockholm International Peace Research Institute, accessed January 4, 2019, link.
“Embassies and Consulates around the World,” EmbassyPages.com, 2019, link.
“United Nations Security Council,” United Nations, 2019, link.
“OHCHR | HRC List of Past Members of the HRC,” United Nations Human Rights Council, 2019, link.
“OHCHR | HRC Membership of the Human Rights Council, 1 January - 31 December 2019,” United Nations Human Rights Council, 2019, link.
“Contribution to UNHCR - 2017” (United Nations Human Rights Council, 2018).
“Freedom in the World 2018,” Freedom House, January 13, 2018, link.
Our list of countries considered only includes those with 500,000 inhabitants or more (as per World Bank population data of 2017). This is a design choice made because of data availability, and is in line with the standard data approaches.
A full overview of the methodology is beyond the scope of this paper. For a detailed method document, please refer to this document.
For more information, please refer to the ‘Military Utility’ section of the DFRI Methodology. See Tim Sweijs, Hugo van Manen, and Paul Verhagen, “The Dutch Foreign Relations Index: Methodological Note (v0.1)” (The Hague: The Hague Centre For Strategic Studies, 2017), link.