Research on Research
Unfortunately the residual effects of some significant surgery this summer prevented me from attending the recent launch of the Research on Research Institute (RoRi). This initiative has the laudable aim of better informing funders and policymakers about “how research is funded, practiced and evaluated, and… how research cultures and systems can be made more efficient, open, inclusive and impactful.” It is receiving significant support from the Wellcome Trust and Digital Science “to build an international consortium of funders, academics and technologists committed to transformative & translational RoR.”
A great deal of research funding and policy is indeed poorly evidenced, driven by outdated theories and prejudices about the virtues of ‘great men,’ and neglectful of the significant body of social science research on leadership and teamwork. Senior figures claim they can ‘smell’ talent as if it were a pheromone, mostly given off by young white men. Principal investigators think standards can be driven up by randomly shouting at graduate students or postdocs. Fraud and fabrication are the result of ‘bad apples’ rather than systemic incentives that reward corrupt behavior. Ideologies about Open Science, Open Data and Open Access substitute for empirical research on the evolution of the present system of scientific and scholarly production.
Nevertheless, RoRi cannot help reminding me of one of my favorite papers: ‘The Informers’ Tale’ by Lee and Alexandra Benham. The authors have made serious contributions to the field of institutional economics, from their backgrounds in economics and political science. This paper is, however, owes more to Jonathan Swift. It was presented at a conference on the regulation of professions in the late 1970s.
The paper opens by discussing an essay by the fictional Professor Zeitgeist. This considered the recurrent crises in the US about health care costs and quality as a problem in informational asymmetry. The closure of health care markets by professional licencing failed to solve the problem. Service users had to take the providers’ word for it that their treatment was appropriate, cost-effective and of acceptable quality. Zeitgeist proposed to remedy this by creating a new profession of ‘informers.’ They would be barred from providing health care services but would be experts in evaluating service user’s needs and informing them about how they would be best met. The expected benefits were that high quality care would be rewarded, providers would compete to provide this, and demand for costly and unnecessary services would plummet.
In this alternative universe, Professor Zeitgeist’s plan was adopted with enthusiasm. Resistance from established health professions led to a compromise where anyone desiring services covered by a national program would first have to consult an informer, and would only be reimbursed for what had been recommended, but remained free to choose a specific provider and treatment plan.
The program’s success attracted interest from other sectors of the economy where similar informational asymmetries could be found. The informers’ jurisdiction progressively expanded across a range of consumer goods and services. Consumers were forbidden from purchasing these goods and services without an advice certificate from a licensed informer. Determining the needs of consumers became a new academic discipline: needology.
With experience, however, it became clear that the impact on the consumption of professional services had not been matched by an impact on the consumption of consumer goods. People were still buying the same old rubbish. Clearly, they were not seeing informers often enough, the standards for licensing were too low, and some direct control of the market was needed to overcome the bad habits of consumers. Needs would be given priority over wants or desires. Informer prescribing would raise the real real incomes of poor people by ensuring that they only bought worthy products. Of course determining the objective worthiness of a product could not be done in a hurry so the rate of innovation slowed drastically.
This new legislation dramatically boosted the demand for informers. Some responded by hiring assistants to do the work or substituting computerized consultations for personal assessment. Some even pandered to the whims of their clients, although the professional ethics committee of the Informers Society soon came down hard on them. Nevertheless, public compliance continued to be a problem. People would insist on trying to access gaudy trinkets rather than worthy handicrafts. Evidence that people in other countries were happier and healthier was obviously the work of crackpots and troublemakers, who were not applying the correct evaluation methodology. However, new tracer technologies were at hand that would eliminate the problem of the mendacious and deceitful client by compiling detailed and intimate records of their purchasing and consumption behaviour.
Except that a graduate student attending her first convention of the Informers’ Society observed that not all informers were equally competent. Why was there no information that allowed individuals to compare informers? Why had there been no evaluations of the outcomes of different practitioners or different institutes? Shouldn’t the informers recognize the growing demand for ‘mentors’ who helped consumers and service users to choose which informer they should go to for advice….
This summary does scant justice to the wit of the original. However, it brings out a substantial point about the nature of professions and information. I recall a piece of doggerel from my school days:
Big fleas have little fleas upon their backs to bite’em
Little fleas have smaller fleas, and so ad infinitum.
Research on research fills a gap in the world of knowledge. However, it is important not to confuse it with the research enterprise itself or to assume that this will benefit from being made so planned, rational and evidence-based that the result is to squeeze innovation out of the system. Funders need to assess and manage risks – but they also need to take them. It would be a serious mistake to imagine that RoR should hustle us towards a world of rationally planned and organized research funding. While better information is certainly desirable, we need more diversity not less if researchers are to innovate rather than replicate.
