Chapter 7: Multi-Criteria Analysis

7.1 Introduction to Multi-Criteria Analysis (MCA)

While Cost-Benefit Analysis (CBA) is a powerful tool for economic policy evaluation, it has limitations when dealing with complex decisions involving multiple, often conflicting objectives that can’t all be monetized. This is where Multi-Criteria Analysis (MCA) comes into play, offering a structured approach to decision-making that can incorporate both quantitative and qualitative criteria.

MCA is particularly valuable when:

• Not all impacts can be readily expressed in monetary terms
• There’s a need to consider a wide range of criteria simultaneously
• Stakeholder perspectives and preferences need to be explicitly incorporated
• Transparency in the decision-making process is crucial

The development of MCA techniques gained momentum in the 1960s and 1970s, with methods like the Analytic Hierarchy Process (AHP) introduced by Thomas Saaty in 1970. However, it was in the 1980s and 1990s that MCA gained widespread recognition as a valuable tool for policy analysis.

In 2000, the UK government published its “Multi-criteria analysis: a manual,” providing comprehensive guidance on MCA use in government decision-making. Similarly, in Australia, the Department of Finance and Administration’s “Introduction to Cost-Benefit Analysis and Alternative Evaluation Methodologies” (2006) and subsequent guidelines have recognised MCA as a valuable complement to CBA, particularly for complex policy decisions involving multiple objectives.

Key Principles and Concepts of MCA:

1. Multiple Criteria: MCA explicitly recognises that decisions often involve multiple, sometimes conflicting objectives that can’t be reduced to a single metric.
2. Weighting and Scoring: MCA typically involves assigning weights to different criteria and scoring options against these criteria, allowing for a structured comparison.
3. Stakeholder Involvement: Many MCA techniques incorporate stakeholder perspectives in defining criteria and assigning weights, enhancing the legitimacy of the decision-making process.
4. Transparency: MCA provides a clear, documented process for decision-making, making the rationale behind decisions more transparent and defensible.
5. Handling Qualitative Data: Unlike CBA, MCA can readily incorporate qualitative criteria alongside quantitative measures.
6. Sensitivity Analysis: MCA often includes sensitivity analysis to understand how changes in weights or scores affect the final ranking of options.
7. Non-Compensatory Methods: Some MCA techniques allow for non-compensatory decision rules, where a poor performance on one criterion can’t be offset by good performance on others.
8. Scalability: MCA can be applied to a wide range of decision problems, from simple personal choices to complex policy decisions affecting entire populations.

As we explore MCA further, we’ll examine how these principles are applied in practice, the various MCA techniques available, and how MCA complements other economic policy analysis tools like CBA. We’ll also touch on related approaches such as Cost-Effectiveness Analysis (CEA) and Cost-Utility Analysis (CUA), which share some similarities with MCA but have their own specific applications. By understanding when and how to apply MCA, policy analysts can enhance their ability to tackle complex, multi-faceted decisions that resist simple monetization, providing decision-makers with a more comprehensive view of policy options and their potential impacts.

Charting the Course: How MCA Shaped Australia’s Inland Rail Project

In 2010, the Australian government faced a complex decision regarding the future of freight transport between Melbourne and Brisbane. The existing coastal rail line was nearing capacity, and projections showed a significant increase in freight demand over the coming decades. The Australian Rail Track Corporation (ARTC) was tasked with evaluating options for a new inland rail route, a project that would become one of the largest infrastructure investments in Australia’s history.

As the ARTC team delved into the challenge, they quickly realized that traditional cost-benefit analysis alone wouldn’t suffice. The sheer scale and complexity of the project demanded a more nuanced approach. This is where Multi-Criteria Analysis (MCA) came into play, offering a framework to navigate the myriad of factors at stake.

The team began by embracing the principle of multiple criteria. They identified 11 key factors across four categories: technical feasibility, safety, environmental impact, and community and property impacts. This comprehensive approach allowed them to consider everything from construction costs and travel time to flood immunity and effects on agricultural land.

But how to weigh these diverse factors? The ARTC knew that the success of the project hinged on more than just technical considerations. They needed to understand the perspectives of those who would be most affected by the new rail line. This led them to one of MCA’s core strengths: stakeholder involvement.

Over the course of months, the team engaged in extensive consultations with local communities, industry groups, and government agencies. These discussions not only helped refine the criteria but also brought to light concerns and opportunities that might have otherwise been overlooked. For instance, conversations with farmers highlighted the importance of minimizing the fragmentation of agricultural land, while discussions with regional development agencies emphasized the potential for the rail line to stimulate economic growth in rural areas.

As the project progressed, the ARTC team remained committed to transparency. They meticulously documented each step of the MCA process, from the initial criteria selection to the final route recommendations. This openness not only allowed for public scrutiny but also built trust with stakeholders, many of whom had initially been sceptical of the project.

One of the biggest challenges the team faced was how to handle qualitative data. While some factors, like construction costs, could be easily quantified, others, such as community acceptance or visual impact on landscapes, were more subjective. The flexibility of MCA allowed them to incorporate both types of data into their analysis, providing a more holistic view of each route option.

As the team evaluated the various route options, they found that MCA’s structured approach helped them navigate trade-offs that might have seemed impossible to reconcile. For example, while some routes offered lower upfront costs, they scored poorly on long-term efficiency and community impact criteria. The MCA framework allowed the team to clearly see how these factors balanced out across different options.

In the end, the preferred route that emerged from the MCA process wasn’t the cheapest or the fastest to construct. However, it offered the best overall performance across the range of criteria, promising better long-term outcomes for freight efficiency, community well-being, and environmental sustainability.

The use of MCA in the Inland Rail project showcased its power in tackling complex, multi-faceted decisions. It provided a structured yet flexible framework for comparing vastly different options, incorporated diverse stakeholder perspectives, and offered a clear, defensible rationale for the chosen route.

Approved in 2017 and with construction beginning in 2018, the Inland Rail project stands as a testament to the value of MCA in shaping major infrastructure decisions. As the 1,600 km line progresses towards its prioritised completion of the Beveridge–Parkes section by 2027 (ARTC, 2025), it carries with it the imprint of a decision-making process that strove to balance diverse needs and perspectives for the benefit of Australia’s future.

1. ARTC. (2010). Melbourne–Brisbane Inland Rail Alignment Study.
2. Department of Infrastructure, Regional Development and Cities. (2015). Inland Rail Programme Business Case.

This real-world example illustrates the practical application of MCA principles in a major infrastructure project, demonstrating its value in complex decision-making processes.

7.2 Practical Guide to Applying MCA

In today’s complex policy landscape, Multi-Criteria Analysis (MCA) serves as your navigational tool for making decisions that involve multiple, often conflicting objectives. Let’s explore how MCA can guide us through a challenging policy decision, using environmental conservation as our example. We’ll see how MCA helps us determine our goals, assess our options, and choose the best path forward.

Step 1: Define the Decision Context and Objectives

Defining the decision context sets the stage for your entire analysis. It involves clearly articulating the problem you’re addressing and the objectives you’re trying to achieve. This step is crucial because it ensures that your analysis remains focused and relevant to the real-world issue at hand.

Practitioner’s Tip: As you define your decision context and objectives, take a moment to reflect on the clarity and comprehensiveness of your problem statement. Have you articulated the issue in a way that captures its full complexity? Consider whether your objectives are specific and measurable, and how well they align with broader policy goals. It’s crucial at this stage to ensure you’ve considered the perspectives of all relevant stakeholders. Their input can often reveal aspects of the problem you might have overlooked.

Case Study: A state environmental agency sets its objectives: “To protect biodiversity, reduce carbon emissions, and promote sustainable land use in a 100,000-hectare forest area over the next 20 years.”

Step 2: Identify Alternatives

Identifying alternatives involves brainstorming and researching possible solutions to your problem. This step is important because it ensures you’re considering a full range of options, not just the obvious ones. As in CBA, always include a “do nothing” option as your baseline.

Practitioner’s Tip: When identifying alternatives, cast your net wide. Challenge yourself to think beyond the obvious solutions. Have you considered innovative approaches that might not be immediately apparent? Consultation is key here – stakeholders can often provide valuable insights into potential alternatives. Remember, the goal is to create a diverse range of options that truly explores the solution space. Don’t forget to include a ‘do nothing’ option as your baseline, just as you would in a CBA.

Case Study: The agency identifies four options:

Option A: Maintain current forest management practices (do nothing)

Option B: Establish a strict nature reserve with no human activity allowed

Option C: Implement sustainable forestry practices

Option D: Create a mixed-use area with zones for conservation, sustainable forestry, and ecotourism

Step 3: Determine Evaluation Criteria

This step involves identifying the factors you’ll use to judge the performance of each alternative. It’s a critical step where MCA diverges from CBA, as it allows for the inclusion of criteria that can’t be easily monetized.

Practitioner’s Tip: As you develop your evaluation criteria, strive for comprehensiveness and clarity. Your criteria should cover all aspects of your objectives, but be careful to avoid overlap or redundancy. Each criterion should be distinct and independently assessable. Consider how you’ll measure or assess each criterion – will it be quantitative or qualitative? Consistency in measurement across alternatives is crucial for a robust analysis.

Case Study: The criteria include:

1. Biodiversity preservation
2. Carbon sequestration potential
3. Economic impact on local communities
4. Recreational value
5. Cost of implementation and management
6. Resilience to climate change impacts

Step 4: Assign Weights to Criteria

Weighting involves determining the relative importance of each criterion. This step is crucial because it reflects the priorities of decision-makers and stakeholders, and can significantly influence the final outcome.

Practitioner’s Tip: Weighting criteria is often one of the most challenging and contentious parts of an MCA. It’s essential to involve relevant stakeholders in this process to ensure buy-in and capture diverse perspectives. Consider both short-term and long-term priorities in your weighting. You might find structured techniques like the Analytic Hierarchy Process helpful in navigating this step. Remember, transparency in how weights are determined is crucial for the credibility of your analysis.

Case Study: After stakeholder consultations, the highest weights are assigned to “Biodiversity preservation” and “Carbon sequestration potential”, reflecting the primary environmental goals.

Step 5: Score Alternatives Against Criteria

This step involves evaluating how well each alternative performs against each criterion. It’s a critical step that often involves both quantitative data and qualitative judgments.

Practitioner’s Tip: When scoring alternatives, consistency is key. Ensure you’re using a uniform scoring system across all criteria. Consider both immediate and long-term impacts in your assessments. Base your scores on the best available evidence, and don’t hesitate to seek expert judgment where needed. If you’re using qualitative assessments, try to define what each score means as clearly as possible to maintain consistency across different scorers.

Case Study: Each option is scored against the criteria. For example, Option B (strict nature reserve) scores high on biodiversity preservation but lower on economic impact on local communities.

Step 6: Calculate Overall Scores

This step involves combining the weights and scores to get an overall value for each alternative. While similar in concept to calculating Net Present Value in CBA, MCA allows for more flexibility in how these calculations are performed.

Practitioner’s Tip: As you calculate overall scores, consider whether your chosen method is appropriate for your decision context. There are various MCA methods available, from simple weighted sums to more complex outranking approaches. The choice depends on your specific situation and the level of complexity your stakeholders can engage with. Whichever method you choose, ensure it’s transparent and understandable to all involved. This transparency is crucial for the credibility and acceptance of your results.

Case Study: After calculations, Option D (mixed-use area) emerges with the highest overall score, followed closely by Option C.

Step 7: Conduct Sensitivity Analysis

Sensitivity analysis involves testing how changes in your inputs affect your results. This step is crucial for understanding the robustness of your analysis and identifying which factors have the most influence on the outcome.

Practitioner’s Tip: In your sensitivity analysis, be thorough in testing how changes in weights and scores affect your results. Consider best-case and worst-case scenarios to understand the range of possible outcomes. Pay particular attention to any ‘tipping points’ where small changes in inputs lead to significant shifts in results. This analysis not only tests the robustness of your results but can also highlight which factors are most crucial in driving the outcome.

Case Study: The agency tests how the results change if more weight is given to “Economic impact on local communities” and less to “Recreational value”.

Step 8: Consider Qualitative Factors

While MCA is designed to incorporate qualitative factors better than CBA, there may still be important considerations that didn’t fit neatly into your criteria. This step allows for a final “sense check” of your results.

Practitioner’s Tip: As you reflect on qualitative factors, ask yourself if there are any important considerations that your formal analysis might have missed. Does the analytical result align with intuition and expert judgment? If not, explore why. Consider any ethical or political dimensions that might not have been captured in your criteria. This step is your opportunity to ensure that your analysis truly reflects the full complexity of the decision at hand.

Case Study: The agency considers factors like the cultural significance of the forest to indigenous communities and potential future changes in climate policy.

Step 9: Present Findings and Recommendations

The final step involves communicating your results and recommendations to decision-makers and stakeholders. Clear, transparent communication is crucial for the credibility and acceptance of your analysis.

Practitioner’s Tip: When presenting your findings, clarity and transparency are paramount. Explain your methodology and assumptions in a way that’s accessible to your audience. Visual aids can be incredibly helpful in communicating complex results. Be upfront about any uncertainties or limitations in your analysis – this honesty enhances the credibility of your work. Remember, your role is to inform decision-making, not to make the decision itself.

Case Study: The agency prepares a report recommending Option D (mixed-use area), supported by the MCA results. They also suggest that elements of Option C (sustainable forestry) could be emphasized within the mixed-use framework to address concerns about economic impact.

Remember, while MCA provides a structured approach to complex decisions, it’s not infallible. Like CBA, it’s a tool to inform decisions, not make them for you. By following these steps and adapting them to your context, you’ll be well-equipped to tackle multi-faceted policy challenges, balancing diverse objectives and stakeholder perspectives in a way that purely economic analyses often struggle to achieve.

7.3 Navigating the Complexities: Limitations, Criticisms, and Effective Communication in MCA

As we continue our journey through the landscape of Multi-Criteria Analysis, we find ourselves in a terrain that, while rich with possibilities, is also fraught with challenges. Much like a seasoned hiker navigating a complex trail system, we must be aware of the limitations and criticisms of MCA to use it effectively.

Throughout our exploration of MCA, you might have wondered about the subjectivity in assigning weights to criteria or the potential for stakeholder bias. These are common concerns that analysts and policymakers grapple with when employing MCA.

Let’s consider a cautionary tale from urban planning. In the early 2000s, many cities used MCA to decide on sustainable transportation strategies. While these analyses often highlighted the benefits of cycling infrastructure and public transit, they sometimes underestimated the challenges of changing deeply ingrained car-centric behaviours. It’s akin to planning a new hiking trail without considering the habits of existing hikers – the benefits may be clear on paper, but the real-world implementation can be far more complex.

This example underscores one of the key challenges in MCA: the difficulty in accurately predicting how different stakeholders will respond to policy changes, especially when dealing with complex social systems. It’s like trying to forecast how a diverse group of hikers will react to a new trail system – their preferences and behaviours can be as varied as the landscape itself.

But the challenges don’t end there. Consider the application of MCA in healthcare resource allocation. While MCA can help balance factors like cost-effectiveness, equity, and public health impact, it may struggle to capture less tangible aspects like the psychological impact on healthcare workers or long-term societal effects. This reveals another limitation of MCA: its effectiveness is only as good as the criteria we choose to include.

Now, let’s look at two related approaches that can complement MCA in certain situations: Cost-Effectiveness Analysis (CEA) and Cost-Utility Analysis (CUA). CEA is particularly useful when comparing alternatives that have a common effect but differ in magnitude and cost. For instance, in healthcare, CEA might compare different treatments for a specific condition based on their cost per life-year saved. CUA, on the other hand, goes a step further by incorporating quality of life measures, often using metrics like Quality-Adjusted Life Years (QALYs). These tools can be particularly valuable when dealing with interventions where the primary outcomes are health-related and can be standardized.

However, both CEA and CUA have their own limitations. They focus primarily on a single outcome measure, which may not capture the full range of impacts that MCA can consider. Additionally, they can struggle with equity considerations and may not fully reflect societal preferences.

So, how do we navigate these complex terrains? The key is to recognise that MCA, like any analytical tool, is a powerful aid in our decision-making toolkit, but it’s not infallible. It’s more like a compass that helps us understand the direction we’re heading, but doesn’t determine the entire journey for us.

Here are some tips for your MCA journey:

1. Be transparent about the process of criteria selection and weight assignment. Clearly document and communicate how these crucial decisions were made.
2. Consider complementary analyses. Use sensitivity analysis to understand how changes in weights or scores affect the outcomes. Employ techniques like stakeholder analysis to ensure all relevant perspectives are captured.
3. Engage with a diverse range of stakeholders throughout the process. Their insights can help refine criteria, provide reality checks on scores, and increase buy-in for the final decision.
4. Communicate clearly and visually. Use charts, diagrams, and plain language to explain your analysis to decision-makers and the public. Be prepared to explain not just the results, but the reasoning behind each step of the MCA process.
5. Remember that MCA is a tool to inform decisions, not to make them automatically. The final decision should consider the MCA results alongside other factors, including political feasibility and ethical considerations.

As we reach this checkpoint in our journey through economic policy analysis, it’s crucial to understand MCA’s role within the larger toolkit available to policymakers and analysts. While MCA is a versatile instrument, it’s most effective when used in conjunction with other analytical approaches.

Think of economic policy analysis as a Swiss Army knife, with MCA being one of its most adaptable tools. Other implements in this kit might include CBA for monetizable impacts, CEA or CUA for standardized outcome comparisons, or input-output analysis for understanding economic interdependencies. Each tool has its strengths and limitations, and the art of good policy analysis lies in knowing when and how to use each one.

In the complex landscape of modern policy challenges, from climate change adaptation to healthcare reform, relying on any single tool can be limiting. MCA shines when we need to balance multiple, often conflicting objectives, but it might need to be complemented by other approaches to provide a comprehensive analysis.

The key is to approach economic policy analysis with a flexible, multi-faceted strategy. By understanding the strengths and limitations of MCA, and knowing how to complement it with other analytical approaches, we can develop more robust, nuanced policy recommendations. This holistic approach allows us to not only evaluate alternatives across multiple criteria but also to consider broader societal impacts, navigate uncertainties, and align policies with overarching social and economic goals.

In essence, while MCA helps us structure and analyse complex decisions, the full suite of economic policy analysis tools enables us to contextualize, predict, and strategize more comprehensively. As we continue to face intricate challenges in our ever-changing economic and social landscape, this comprehensive approach to policy analysis will be more important than ever. It allows us to craft policies that are not just analytically sound, but also socially equitable, environmentally sustainable, and adaptable to future changes.

Tying to Economic Policy Analysis

Sometimes the most honest thing an analyst can say is: “not everything that matters can be converted to dollars.” That’s not a failure of analysis — it’s the starting point for MCA. The tool’s strength isn’t that it avoids value judgements, it’s that it makes them visible and debatable. When using MCA:

• Use it when the decision genuinely involves criteria that can’t be meaningfully monetised, or when different stakeholders hold fundamentally different views about what matters most.
• Involve stakeholders in defining criteria and weights — not just in reviewing your results. The process of working through weights together is often as valuable as the final ranking.
• Run sensitivity analysis on weights and scores. If small changes flip the ranking, that instability is important information, not an embarrassment.
• Remember that MCA structures a decision — it doesn’t make one. Your job is to present the results clearly and let decision-makers exercise informed judgement.

Chapter 7: Further Reading & References (Multi-Criteria Analysis)

Further Reading

MCA Methodology and Applications

Greco, S., Ehrgott, M., & Figueira, J. R. (Eds.). (2016). Multiple Criteria Decision Analysis: State of the Art Surveys. (2nd edition). Springer.

Belton, V., & Stewart, T. (2002). Multiple Criteria Decision Analysis: An Integrated Approach Kluwer Academic Publishers.

MCA in Environmental Policy

Geneletti, D. (2019). Multi-Criteria Analysis for Environmental Decision-Making. Anthem Press.

Linkov, I. (2021). Multi-Criteria Decision Analysis: Environmental Applications and Case Studies. (2nd edition). CRC Press.

Gregory, R., Failing, L., Harstone, M., Long, G., McDaniels, T., & Ohlson, D. (2012). Structured Decision Making: A Practical Guide to Environmental Management Choices. Wiley-Blackwell.

References

Beinat, E., & Nijkamp, P. (Eds.). (1998). Multicriteria Analysis for Land-Use Management. Kluwer Academic Publishers.

Department of Finance and Administration. (2006). Introduction to Cost-Benefit Analysis and Alternative Evaluation Methodologies. Commonwealth of Australia.

Dodgson, J., Spackman, M., Pearman, A., & Phillips, L. (2009). Multi-criteria analysis: a manual. Department for Communities and Local Government, London.

Hajkowicz, S. A., & Collins, K. (2007). A review of multiple criteria analysis for water resource planning and management. Water Resources Management, 21(9), 1553-1566.

Herath, G., & Prato, T. (2006). Using Multi-Criteria Decision Analysis in Natural Resource Management. Ashgate.

Huang, I. B., Keisler, J., & Linkov, I. (2011). Multi-criteria decision analysis in environmental sciences: Ten years of applications and trends. Science of the Total Environment, 409(19), 3578-3594.

Kind, J. M. (2014). Economically efficient flood protection standards for the Netherlands. Journal of Flood Risk Management, 7(2), 103-117.

Kiker, G. A., Bridges, T. S., Varghese, A., Seager, T. P., & Linkov, I. (2005). Application of multicriteria decision analysis in environmental decision making. Integrated Environmental Assessment and Management, 1(2), 95-108.

Saaty, T. L. (1980). The Analytic Hierarchy Process: Planning, Priority Setting, Resource Allocation. McGraw-Hill.

Zeleny, M. (1982). Multiple Criteria Decision Making. McGraw-Hill.