Is Maximizing the Impact of Research Desirable?


Impact crash
Sometimes the desire to have impact can come at a price.
Currently, there are considerable demands made upon social science to demonstrate its ‘impact.’ These are institutionalized in the requirements laid down by funding bodies and in attempts strategically to manage research within universities. Moreover, as part of campaigns designed to protect the funding of social science there have been efforts to provide evidence of its impact and thereby of its value. Underpinning all this is the generally accepted assumption that it is desirable to maximize the impact of research on policymaking and practice. But is this true?

Of course, we must recognize the practical and ideological contexts in which arguments about research impact take place. Governments want to be able to justify their funding of social science by pointing to practical ‘returns’ on the ‘investment’ made. Universities have been required to demonstrate the impact of the research carried out by their members, not least for the purposes of the Research Excellence Framework. Social scientists, and the organizations that represent them, wish to highlight the impact of their work in order to defend its funding in an age of austerity.

The Limits of Social Science book cover
This article by Martyn Hammersley is drawn from material in his new book, The Limits of Social Science.

Nevertheless, there are questionable assumptions involved in the idea that the impact of social research should be maximized. First, it seems to assume that the effects of sound research on policymaking and practice are always desirable. This is held almost as an article of faith by many researchers: very often people become researchers precisely because they believe that through research they can bring about social improvement of one kind or another, whether piecemeal or transformative. While, of course, they recognize that such change does not always result, perhaps even that it happens infrequently, they nevertheless assume that, if it had the influence it deserved, good research would always improve policymaking or practice, and that when this does not happen it amounts to a distortion of the relationship.

However, this assumption is of doubtful validity. It implies that research findings carry within them instructions about what ought to be done, whereas in fact any set of empirical findings can generate conflicting evaluations and prescriptions, depending upon the particular set of values adopted to interpret them. This clearly raises the question of what counts as ‘improvement’. And there is rarely general consensus about the priority that should be given to different values. Aside from this, no empirical findings about the nature of some problem, or even about the effects of previous policies or practices, can tell us with great certainty what the effects of future policy or practice will be: there is too much contingency in social processes. For both these reasons, the relationship between particular empirical findings and the production of social improvement (however defined) is highly mediated, to say the least. Indeed, the publication of research findings may sometimes even have consequences that researchers and others regard as undesirable.

The second assumption involved in the commitment to maximize the impact of social research is the belief that it is desirable to increase the contribution of research at the expense of the various other factors that shape policy and practice. For example, in the promotion of evidence-based policymaking and practice around the turn of the last century it was common for these other factors to be caricatured as irrational, for example as reflecting the influence of ‘tradition, prejudice, dogma, and ideology’ (Hargreaves 2007:12). From this point of view, making policy or practice more research-based necessarily results in it becoming more rational. And many researchers outside of the evidence-based practice movement have shared this view.


References

Dunne, J. (1997) Back to the Rough Ground: Phronesis and Techne in Modern Philosophy and in Aristotle, Notre Dame IN, University of Notre Dame Press.

Hammersley, M. (2002) Educational Research, Policymaking and Practice, London, Paul Chapman/Sage.

Hammersley, M. (2013) The Myth of Research-Based Policy and Practice, London, Sage.

Hargreaves, D.H. (2007) ‘Teaching as a research-based profession: possibilities and prospects’, in Hammersley, M. (ed.) Educational Research and Evidence-based Practice, London, Sage. (Annual Teacher Training Agency, given in 1996.)

MacIntyre, A. (1971) ‘The end of ideology and the end of the end of ideology’, in Against the Self-Images of the Age, London, Duckworth.

Turner, S. (2003) Liberal Democracy 3.0, London, Sage.

Weiss, C. (1979) ‘The many meanings of research utilization’, Public Administration Review, 39, 5, pp426-31.


However, this too is questionable. For one thing, research cannot provide all of the information that is required by policymakers and practitioners – they must necessarily rely upon other sources of knowledge (Hammersley 2002 and 2013): for information about local conditions not studied by researchers or about very recent events (research can never be entirely up to date). Furthermore, like any other source of knowledge claims, research is fallible. For this reason, policymakers and practitioners must engage in evaluation of the relevant evidence available to them, rather than accepting it at face value.

In addition, we must recognize that policymaking relies upon background experience and judgment about priorities, about what would and would not be ethically and politically acceptable policies, as well as about which ones are likely to mobilize sufficient support to get off the ground. Moreover, what is demanded here is close in character to the sort of practical, rather than theoretical, judgment that Aristotle calls phronesis (Dunne 1997).

Against this background, it is perhaps worth giving attention to one particular influence on policymakers and practitioners that is frequently caricatured as a negative factor: ideology. This is generally treated as consisting of false views that people adopt so as to serve their own interests, or into which they have been socialized because these views function to preserve some larger form of social organization. However, what the term ‘ideology’ points to is more complex than this, as Alasdair C. MacIntyre (1971:5-6) highlighted in his critique of the claim that the 1950s had witnessed ‘the end of ideology.’ Ideology offers a worldview within which particular events, actions, etc – and information about these – can be understood. As MacIntyre emphasizes, it is impossible to escape reliance upon ideology in this sense, especially if one is engaged in practical action: even a thoroughly pragmatic or expedient orientation expresses a distinctive ideology. Moreover, a key component of ideologies is a particular prioritization among value principles, this providing the basis for judgments about the relative significance of potential benefits or costs associated with particular policies or practices.

This should lead us to ask: Do researchers also necessarily rely upon ideological assumptions? Probably few social scientists today would deny this. It is certainly true that to the extent that researchers seek to present practical evaluations and recommendations as legitimated by their research, they cannot avoid reliance upon ideology in this sense of the term. And, if this is true, then any insistence that the impact of research should be maximized at the expense of ideology is disingenuous. What may be involved here is, in effect, a contest between ideologies.

What this suggests is that ‘impact’ is most likely to occur where there is a consonance between the ideological framework within which research operates, and that relied upon by policymakers and practitioners – or where research offers a new framework that seems appealing or productive. An interesting case from this point of view is the way in which, over the course of the twentieth century, a mode of economic thinking operating within a neoliberal ideological framework effectively colonized public discussion and decision-making. Existing and proposed policies came to be assessed almost entirely according to their costs and benefits in these economic terms. We might reasonably ask whether this was a desirable development.

In conclusion, then, we need to adopt a rather more sophisticated view of the relationship between research and policymaking/practice than that assumed by the injunction to maximize the impact of research. Carol Weiss’s words in 1979 remain true today:

There has been much glib rhetoric about the vast benefits that social science can offer if only policy makers paid attention. Perhaps it is time for social scientists to pay attention to the imperatives of policymaking systems and to consider soberly what they can do, not necessarily to increase the use of research, but to improve the contribution that research makes to the wisdom of social policy.


Martyn Hammersley

Martyn Hammersley is professor of educational and social research at The Open University, UK. He has carried out research in the sociology of education and the sociology of the media. However, much of his work has been concerned with the methodological issues surrounding social enquiry. He has written several books, including: (with Paul Atkinson) Ethnography: Principles in Practice (Third edition Routledge 2007); Reading Ethnographic Research (Longman 1991); What's Wrong with Ethnography? (Routledge 1992); The Politics of Social Research (Sage 1995); Taking Sides in Social Research (Routledge, 1999); Educational Research, Policymaking and Practice (Paul Chapman, 2002), Media Bias in Reporting Social Research? The case of reviewing ethnic inequalities in education (Routledge, 2006); Questioning Qualitative Inquiry (Sage 2008), Methodology, Who Needs It? (Sage, 2011), The Myth of Research-Based Policy and Practice (Sage, 2013), and The Limits of Social Science (Sage, 2014).

2
Leave a Reply

avatar
2 Comment threads
0 Thread replies
0 Followers
 
Most reacted comment
Hottest comment thread
0 Comment authors
Joe Kotarba on ‘Symbolic Interaction and Applied Social Research’ | Symbolic Interaction (Journal) BlogIs Maximizing the Impact of Research Desirable? | Learning Research & Change Methods Recent comment authors

This site uses Akismet to reduce spam. Learn how your comment data is processed.

  Subscribe  
newest oldest most voted
Notify of
trackback

[…] the “impact” of sociology and related social research. Over on socialscinecespace.com Martyn Hammersley has challenged the unreflected demands for impact and the often naive calls for evidence-based […]

Skip to toolbar