We are both interested in the setting of priorities in health care, and in particular using economic analysis to help with that. In the United Kingdom, an official organization called NICE (National Institute of Health and Clinical Excellence) was set up to tackle difficult decisions about new health-care technologies, and that was interesting to us as an internationally ground-breaking initiative.

“ ...if public sector decisions are based on cost-effectiveness ratios, then the decision-maker must be invoking some sort of judgment about what they consider to be acceptable value for money—and if they are doing this consistently, it can be inferred from the decisions actually made.”

But there was some controversy about the way NICE was using evidence on the cost-effectiveness of technologies: how was it judging whether any given cost-effectiveness ratio (usually expressed as a cost per Quality Adjusted Life Year [QALY] gained) represented good value for money or not? There had been a lot of speculation about this and we were involved in a workshop at which it was debated.

At the same time, we had both been interested in a set of statistical techniques—binary choice models—used in a quite different context. All NICE’s decisions and documents are published on its website—and we realized that we had all of the data we needed to pinpoint the decision-rule implicit from the choices NICE had made, and a useful statistical tool to analyze it.

A binary choice model attempts to explain the factors that have influenced a choice between two alternatives. The alternatives in this case are NICE saying either "yes" or "no" to a particular health care treatment technology: If they say "yes," that drug or procedure is then funded by the UK health-care system.

Regression analysis is used, with the dependent variable being the decision and explanatory variables being those which were hypothesized to influence it, such as the cost-effectiveness ratio for that treatment, its overall costliness to the NHS budget, the burden of disease experienced by those who have the condition for which the treatment is given, whether there were any special considerations of fairness to patient groups, and so on.

The key difference from an ordinary regression analysis is that the resulting regression equation does not predict a value for the dependent variable, but a probability—in this case the probability of a "no" decision. This is done by transforming the binary dependent variable, which is 1 or 0, to a continuous variable.

That the probability of NICE saying "no" to a drug was higher the less cost effective the drug, but that decisions were also influenced by the uncertainty about cost-effectiveness evidence and the burden of disease. Overall, the results suggested NICE decisions are consistent with its using a cost-effectiveness threshold range, although the results suggested this was a bit higher than the £20-30k per QALY gained range sometimes mentioned.

The most basic implication is that if public sector decisions are based on cost-effectiveness ratios, then the decision-maker must be invoking some sort of judgment about what they consider to be acceptable value for money—and if they are doing this consistently, it can be inferred from the decisions actually made. If things other than cost effectiveness matter to decision-makers, this will be observable as trade-offs against value for money. The challenge is for decision-makers to be explicit about these judgments, because any given threshold value or range is going to be difficult to defend to the general public.