Policy Evaluation and Feedback

The Question Every Policy Must Answer

After a policy is implemented, the most important question is deceptively simple: is it working? Policy evaluation is the systematic assessment of a policy's design, implementation, and outcomes. It asks not just whether goals were achieved but why or why not — and what should be done differently.

Evaluation is where policy analysis comes full circle. The problem was defined, alternatives were generated, a recommendation was made, and the policy was implemented. Now we find out whether the analysis was right. Good evaluation requires intellectual honesty: the same team that designed a policy is often reluctant to find that it is not working. This is why many governments separate evaluation functions from policy design — the UK's National Audit Office and the US Government Accountability Office exist precisely to provide independent assessment.

Types of Policy Evaluation

Process evaluation examines how the policy is being implemented. Is it reaching the target population? Are services being delivered as designed? Are timelines being met? Process evaluation catches implementation problems early — before outcome data even becomes available. If a job training program is only reaching 20% of eligible participants, outcome evaluation will be misleading regardless of the program's effectiveness.

Outcome evaluation measures whether the policy achieved its intended results. Did the education reform improve test scores? Did the pollution regulation reduce emissions? Outcome evaluation requires clear, measurable indicators defined before the policy launches — not invented after the fact to fit the data.

Impact evaluation goes further, attempting to establish a causal link between the policy and observed outcomes. This requires a counterfactual: what would have happened without the policy? Techniques include randomized controlled trials, difference-in-differences analysis, and regression discontinuity designs. Impact evaluation is the most rigorous but also the most expensive and time-consuming.

Economic evaluation assesses whether the policy's benefits justify its costs. Cost-effectiveness analysis compares outcomes per dollar spent across alternatives. Cost-benefit analysis monetizes all outcomes and compares total benefits to total costs.

Feeding Evaluation Back Into Policy

Evaluation is only valuable if its findings actually influence future decisions. This sounds obvious but rarely happens automatically. Research by the OECD found that fewer than half of government evaluations led to any identifiable policy change.

Effective feedback loops require several conditions:

Timeliness. An evaluation that arrives two years after the policy decision window has closed is academically interesting but practically useless. Rapid-cycle evaluation — using administrative data to produce findings within months rather than years — is increasingly used by governments that want actionable feedback.

Accessibility. A 200-page evaluation report will not be read by decision-makers. Evaluation findings must be translated into brief, actionable summaries with clear recommendations. The What Works centres in the UK excel at this: they produce one-page 'evidence summaries' alongside the full technical reports.

Institutional channels. There must be a formal mechanism for evaluation findings to reach decision-makers. In Australia, the Productivity Commission's reports are formally tabled in Parliament, ensuring they enter the policy debate. Without such channels, even excellent evaluations gather dust.