Coherence assessment framework
Four-phased approach
Below you will find a practical guidance for the organization of the coherence assessments. The guidance breaks down the coherence assessment into four phases, to ensure a shared focus on the selected EGD strategies and objectives as well as selected EU and (sub)national policies.
A) Structuring the evaluation
Select the policies that will be part of the assessment
Policies selected in CrossGov
- EGD Strategies
2030 Target Plan
Climate Change Adaptation Strategy
Biodiversity Strategy to 2030
Zero Pollution Action Plan
Sustainable Blue Economy Strategy
- EU policies meant to implement the selected EGD strategies
- Water Framework Directive (WFD)
- Marine Strategy Framework Directive (MSFD)
- Maritime Spatial Planning Directive (MSPD)
- Habitats Directive
- Birds Directive
- Strategic Environmental Assessment Directive (SEA Directive)
- Environmental Impact Assessment Directive (EIA Directive)
- Renewable Energy Directive
- Common Agricultural Policy (CAP, parts that are relevant for the selected EGD objectives in the marine sphere)
- Common Fisheries Policy (CFP, parts that are relevant for the selected EGD objectives)
- National policies transposing the relevant EU policies at the national level
- (Sub)national delivery mechanisms relevant for implementing the selected EGD strategies
- Setting the time frame
Determining the time frame of the assessment is important to clarify which versions of policies are being evaluated and which policies are included or excluded. For assessing cross compliance, it is also important to specify the time period within which progress/success of policy implementation is evaluated.
- Determining the geographical scope
This is particularly important when evaluating regional frameworks like EU policies that are
implemented in spec ific national and sub national contexts.
- Defining the evaluation questions
For example: How does the degree of coherence across environmental and sectoral policies and governance levels affect progress towards achieving cross compliance with the selected EU EGD goals and target ?
To answer this overarching evaluation question, several sub-questions can be defined, for example:
- To what extent are EU policies coherent with the selected EGD goals and targets related to/relevant for European seas?
- To what extent are EU policies coherent with each other; How do specific horizontal coherence challenges across EU level policies affect a single policy’s support towards the EGD goals and targets?
- How do transposition processes (from EU level to national level) affect the level of vertical policy coherence of the national level policy framework towards the EGD goals and targets , and the level of horizontal coherence across these national policies?
- How well aligned are the strategic plan s according to transposed versions o f the WFD, MSFD and MSPD , and how do they influence selected sectors in contributing to achieve EGD goals and targets?
- How far do/can mainstreaming processes of environmental/biodiversity related aspects into sectoral decision making affect coherent and effective policy implementation towards multiple E GD goals and targets?
- In how far do regional seas commissions/policies affect policy coherence within the North Sea, Baltic Sea and Mediterranean Sea policy landscapes?
B) Data collection
Data collection methods
- Legislative and policy document text analysis
Data is collected from the actual policy document text, as well as related documents including preparatory works such as evaluations and impact assessments, guidelines, explanatory memorandums and other possible documents that aim to explain the intention or anticipated effects of the policy, as well as case law.
- Analysis of academic and policy literature
This includes impact assessments, fitness checks, strategic environmental assessments, mid-term and ex-post evaluations studies. It also includes academic literature on scholarly analyses of specific policies. - Surveys (interviews and questionnaires)
Survey participants can be experts, officials, stakeholders or anyone who has relevant knowledge of, or a concrete interest in, the subject under investigation. - Stakeholder workshops
Data is collected from a group of stakeholders or experts through workshops, expert panels or focus groups. - Case studies
Can be an important approach to better understand the causal pathways between policy design, implementation and impacts. It is possible to study single cases, though more insight can be gained by comparing findings across different case studies.
C) Data analysis
- Vertical coherence with the EGD
Specifically, the assessment will look at two elements of the coherence evaluation framework:
– Comparison of policy objectives against objectives of five key EGD strategies [e.g.
using a screening matrix]
– Mainstreaming : Do the policies explicitly state an intent to contribute to addressing
biodiversity, climate change and pollution (beyond their specific objectives)?
- Coherence in policy design
The objective is to assess whether policy design is likely to support horizontal coherence between the policies intended to deliver the EGD. Based on the assumption that coherence between policies supports effective, cross compliant implementation , horizontal incoherence can negatively affect policies’ vertical coherence with the EGD.
- Horizontal coherence at EU level
Where are key coherence issues? What is causing (in)coherence? What are the implications for the individual policies’ contributions to delivering the EGD?
The assessment can involve:
- A desk-top study of the policies, preparatory works, case law, impact assessments, evaluations etc. to help answer the guiding questions and conduct the scoring.
- Interviews/focus groups with stakeholders (policymakers, DGs, EC) to verify the findings. Also consider asking stakeholders about explanations for the choices made in the policy formulation process, and/or viewpoints on how they perceive certain policies to contribute to the various ambitions of the EGD.
- Coherence in national transposition
Coherence in national transposition can be assessed in two steps:
- Identifying how selected countries have chosen to transpose key policies and assessing using the coherence attributes as relevanta.
- How do approaches compare across countries?
- How likely are design related choices to contribute to (in)coherence?
- Assessing coherence horizontally at the national level using the coherence attributes framework and guiding questions
- Identifying how selected countries have chosen to transpose key policies and assessing using the coherence attributes as relevanta.
- Coherence and Cross compliance in planning processes at the local level
The objective is to understand how strategic plan s prepared according to national transposition of the WFD, MSFD and MSPD affect coherent and effective policy implementation towards multiple EGD goals and targets.
- Coherence and Cross compliance in sectoral decision making
The objective is to understand how mainstreaming of environmental/biodiversity related aspectsinto sectoral decision making does/can affect coherent and effective policy implementation towards multiple EGD objectives.
D) Synthesis and conclusions
How does the degree of coherence across environmental and sectoral policies and governance levels affect progress towards achieving cross-compliance with the selected
EGD goals and targets?
The objective is to understand how coherence across environmental and sectoral policies and governance levels affects progress towards achieving cross-compliance with the selected EU EGD goals and targets. To understand which elements of policies are causing barriers to coherence and/or cross-compliance. To understand at which governance level(s) and phases of the policies’ life cycles, coherence and cross-compliance challenges mostly emerge and where they can be best mitigated.
SPSI analysis
4 steps to implement the methodology
Step 1) Inception phase
The methodology can be applied in a flexible way to different context. For this, an inception phase is needed, to define the scope of the analysis and the following steps.
Defining:
- The geographical scope of the analysis
- The policies under analysis and which stages of the policy process need consideration
- The time frame of the analysis
- The research questions to be addressed
- The most important sources of information
Step 2) Defining the Science-Policy-Society system under analysis
Step 3) Characterising the Building Blocks of SPSI to answer the research questions
At the core of the analytical framework and the operational procedure to analyse SPSIs are the 6 Building Blocks introduced earlier.
The analysis of Building Blocks and their attributes shall consider spatial scales and phases of the policy cycle, as also defined in step 1.
Spatial elements can be relevant in the analysis from different perspectives. On one side, local scale data is necessary to understand and manage larger scale processes, pressures and impacts; on the other, policy-makers often need to be able to understand local impacts of larger scale issues, and how mitigative and adaptive strategies enacted at one scale can constrain or benefit actions at other scales.
Challenges for policy-makers might relate to the need to match (1) the scales of bio-geophysical systems and management systems, (2) the scales of assessments and management systems, and (3) understanding the linkages between scales and their consequences for decision-making (Cash, 2000). Moreover, the analysis can reveal significant differences in how SPSIs work in different environments, countries or sea basins, which can be of real added value to the analysis.
All key phases of the policy cycle are informed by SPSIs. Such phases are not hermetically isolated from one another: relevant processes are usually not linear but highly iterative, with phases continuously shaping each other (Van den Hove, 2007). Nevertheless, it is important to consider at which phase and how an SPSI intervenes.
In the policy design and formulation phase, issues requiring action are first detected. This can depend, for example, on newly available scientific input, impactful events, or societal transformations.
Then, it is defined how the policy is being structured, what its objectives and expected effects are, the resources that need to be used. Stakeholders may be involved at this stage, and evidence-based solutions are required.
The Implementation phase requires responsible entities and available resources to turn decisions into actions while ensuring monitoring provisions are met.
Evaluation and Adaptation (or Maintenance) refer respectively to the assessments of the degree of success of the policy and of its implementation and to the confirmation, revision, or dismissal policy phase (Cairney, 2016).
SPSIs can have a role in all these steps and it is important to understand where and how, also to identify possible improvements.
Step 4) Synthesis: answering the research questions
- What type of science, knowledge, and interactions do we need for coherent and cross-compliant formulation and implementation of policies?
2. What type of knowledge is currently provided and used in the different formulation and implementation stages?
3. Which role does science play in planning, policy-making and decision-making?
4. How is knowledge contributing to horizontal and vertical coherence and cross-compliance of policies?
The 2 Frameworks in practice
For a practical example of application of the Policy Coherence framework and the SPSI framework, you can read the following report from the Mediterranean Case Study. The study uses both frameworks to evaluate the current state of play in
coherence and cross-compliance within the relevant case study area.
The process is designed to be flexible, allowing for the selection of attributes most relevant to the specifics of the case study, while a set of guiding questions and a standardized template help structure the analysis.
References
Cash, D., 2000. Linking Global and Local Scales: Designing Dynamic Assessment and Management Processes. Global Environmental Change 10, no. 2: 109–20. https://doi.org/10.1016/S0959-3780(00)00017-0.
Cairney, P, 2016. The Politics of Evidence-Based Policy Making. London: Palgrave Macmillan UK. https://doi.org/10.1057/978-1-137-51781-4.
Van Den Hove, S., 2007. A Rationale for Science–Policy Interfaces’. Futures 39, no. 7: 807–26. https://doi.org/10.1016/j.futures.2006.12.004.