Greater Manchester Centre for Voluntary Organisation

When a measured impact isn't an impact - economically at least

 I recently was in the audience of a presentation on impact measurement which caused me some concern. The consultant presenting was trying to make the point that by understanding the costs of public services, we can understand the impact we generate by reducing demand. This isn’t always true for two reasons: the nature of fixed costs and social trends. I’ll explain each in turn

The example given by the consultant was in the reduction of reoffending having an impact on criminal justice costs. The argument goes that since it costs £40k/year to imprison an offender then by reducing reoffending rates these costs can be saved. The problem is that it’s really hard to save money in this area. In order to reduce prison costs you need to reduce offending significantly. Stop 10 individuals reoffending and a prison’s costs are untouched. You may need to stop many hundreds and potentially thousands from reoffending before you can start reducing the prison estate. A small voluntary organisation trying to measure its impact just won’t be able to deliver that much change

The main problem is that much of the cost of public service delivery in some areas is fixed. The prison will have a similar cost to the state whether it’s operating on full capacity or 10% below capacity. Even if there was enough of a drop in demand to close a prison wing then many fixed costs would remain. Shutting down an entire prison wouldn’t see savings initially – redundancy and demolition costs would reduce gains in the first year of closure although savings would be seen in the future.

Clearly, reducing demand and preventing problems from occurring is plainly a good thing. Economically though there isn’t an immediate return and unless there are significant levels of prevention there may be no direct savings at all. What may happen is that rather than a year of prison costing £40k/year the fixed costs are spread over fewer prisoners and the cost per year increases.

The use of the fixed costs of prisons might seem a little extreme but similar examples can be found across the public sector, especially in health and social care where the growth in demand will potentially fall on hospital provision and through care homes. The ability (and will) to decommission services will influence the ability to save money through prevention.

Even if significant prevention occurs then the second issue influencing impact can remove the chance of any savings occurring. This is when the trends within society are causing an increase in demand greater than the impact of prevention. If there’s an increase in crime within society or if sentencers are directed to punish more harshly, then no matter the performance delivered through preventative working there will be no cashable savings. What will have occurred is that the prevention will have stopped costs from increasing, rather than saving money.

Clearly this makes any Payment By Results scheme quite tricky. If the commissioner is expecting to fund the PBR reward out of savings they need to be sure that they will be able to cut spending once demand drops and that there’s actually a fighting chance to reduce demand in the first place. This makes some areas of delivery less suitable than others in linking impact to cashable savings. In particular, in the field of health and social care we’re seeing predictions of significant increases in demand. Commissioners trying to reduce demand *and* costs at the same time will often be asking the impossible of suppliers. I think much of the drive to measure impact in terms of savings to spending come from desperate public sector commissioners who hope to be able to avoid incredibly difficult spending decisions by finding ways to make problems go away before they occur.

Of course not all methods of measuring impact are so crude as the example given in my first paragraph. We do need to develop ways of understanding the changes we’re able to make if we’re able to explain to funders from any sector why they need to invest in us. It’s also important to be able to explain to those we work with why our practices will help them. However we have to be wary of simplistic economic arguments being employed by commissioners more out of hope than expectation.