1 Measuring influence on outcomes
This part considers the particular issue that regulators face in measuring whether they are achieving their intended outcomes through the influence they exert on regulated entities. It considers the challenges for regulators in measuring their performance and steps that regulators can take to help them to:
- Understand and measure outcomes and associated activities and costs.
- Understand and measure the linkages between those costs, inputs, outputs and outcomes.
- Identify how much influence the regulator has over intended outcomes.
- Draw together information on the use of influence to inform performance measurement and improvement.
It also considers the role of risks to regulatory objectives, and how regulators can assess these risks and measure their performance in mitigating such risks.
2 Performance measurement frameworks
This part summarises existing guidance on good practice criteria for developing performance measurement frameworks and measures, drawing out some examples of existing good practice among regulators. It considers:
- the importance of clear goals in measuring performance;
- the establishment of effective performance frameworks and information; and
- important considerations for regulators in how they report and use performance information.
Background
Performance measurement is important in helping regulatory organisations to make sure that they are achieving their objectives, and making the best possible use of their resources. It is also important for accountability, since regulators are accountable for their activities to Parliament, to government departments, and ultimately to the public who depend on them. Stakeholders - particularly those who fund the activities of regulators - have an interest in whether the resources they provide to regulators are being used efficiently and effectively.
Performance measurement by regulators is particularly complex, because their intended outcomes (for example protection of consumers and improved service quality) are generally delivered by the organisations that they regulate. There are also many external factors outside regulators' control, and outcomes often do not become evident for several or even many years. Regulatory performance depends on the effectiveness of influence that they exert over regulated providers, as well as the outcomes that those providers deliver. Past NAO reports on value for money achieved by regulators have acknowledged this complexity, while also pointing to weaknesses in performance measurement and making recommendations for improvement.
Objectives
In this guide we aim to help promote improvement in performance measurement and reporting by regulators. It draws together relevant material from existing guidance on performance measurement, together with the NAO's experience from our past work and examples of good practice undertaken by regulators. Part One outlines the elements that promote a good performance measurement framework. Part Two sets out ways in which regulators can understand more clearly the influence that they have over their intended outcomes, and how effectively they use that influence. The report includes examples of how some of the challenges outlined have been approached; these are intended to inform regulators' thinking about these challenges rather than being prescriptive.