A new white paper from SAGE Business examines existing bibliometrics and institutional reward structures at play within business schools. We aim to move the dial toward ways in which societal impact could become central to the assessment of business and management research.

In a pioneering survey of its membership in 2017, the Academy of Management asked its worldwide membership what constituted scholarly impact. (With about 20,000 members worldwide, the AOM is the preeminent scholarly association in management. The subsequent membership survey had a response rate of 19 percent, or 700 respondents.)

Despite the ubiquity of journal impact factors as measures of scholarly impact, the majority of respondents (60 percent) indicated that journal rankings and lists, including impact factors, probably or definitely did not or might or might not reflect scholarly impact. Conversely, the top five indicators of scholarly impact were: Scholarly articles in top-tier journals, scholarly citations to research, scholarly books, competitive research grants, and articles in practitioner-oriented/industry publications.

Indicating the historical focus on internal audiences for academic research, respondents to the AOM survey saw the top five research audiences as other academics in management, top management and decision-makers in companies, government and policymakers, other academics in the social sciences, and students.

On the influence of the field, generally, respondents thought that management research had been somewhat influential, but the greatest influence had been on other management academics, including what they currently research and will research and teach. Yet, about 54 percent of the survey’s respondents considered impact on practice as either strongly or intensely important. Similarly, about 46 percent considered impact on government policy as either strongly or intensely important. Though more difficult to publish, about 59 percent viewed interdisciplinary research as probably more or definitely more impactful than research that draws on one discipline. Respondents overwhelmingly saw institutional support as very strong for publications in top-tier journals, with other activities receiving far less, if any, support. Still, only 38 percent said their own institutions supported research with external impact.

Similarly, a global survey undertaken by SAGE in 2021 of social science academics (the first author was involved in drafting this survey and analyzing results), found that of the 373 global respondents in business and management (most from the United States, United Kingdom and India), 81 percent said that it was either important or highly important that their research have value outside academics, and 62 percent stated that their peers felt the same way about their own research. 

Ninety-four percent of the respondents either agreed or strongly agreed that ultimately the goal of their research is to make a positive impact on society. Yet, when asked if their institutions rewarded these efforts to apply research outside academia, only about a third thought they did so. Only 36 percent said that having external impact mattered for tenure, 34 percent said for awards, 32 percent said for funding further research, and, 30 percent for other research resources. Thirty-one percent of the respondents said their institutions provided no rewards or acknowledgments for having external impact through their research. Unsurprisingly, 63 percent either agreed or strongly agreed that ultimately the goal of their research is career advancement. Seventy-seven percent of the respondents stated that publishing their research in a highly cited journal with a high impact factor was either important or very important.

A 2022 poll of 9,609 academics by Times Higher Education found similar results. Aside from personal interactions with the researchers, the perceived quality of the journals where the researchers published their findings constituted the most influential factor when forming opinions of academic standing, according to 49 percent of the respondents. Citation metrics had lower support. Only 24 percent of respondents said a scholar’s h-index and other similar measures were important, and only 5 percent said they constituted the most crucial factor. These findings regarding citations could arise because of increasing awareness of gaming the system through self-citations, citations that speak negatively of the articles, forced citations by journal editors and reviewers, and citations made by researchers who haven’t even read the articles (see Haley, 2021). Additionally, citation counts vary by source—for instance, those in Google Scholar differ from those in Clarivate.

Few measures have exerted greater influence than the journal impact factor, or JIF. Initially developed to help librarians to purchase journals, Clarivate’s JIF has transmogrified into an evaluation of the quality of individual publications and of individual researchers.

The measure has come under extensive scrutiny and criticism: as an inaccurate estimate of citations of any article within a specific journal, as easy to manipulate, and with no associations to objective measures of quality. A decade ago, the San Francisco Declaration on Research Assessment (DORA), critiqued using impact factors as a surrogate measure of quality for individual articles and researchers. DORA currently has 20,000 individual and 2,600 institutional signatories worldwide.

The AOM survey and FT awards (see Jack, 2022) have identified interdisciplinary research as more impactful than single discipline research as it incorporates diverse perspectives and experiences. Yet, interdisciplinary research is also more difficult to publish. Ricardo Fini, Julien Jourdan, Markus Perkmann, and Laura Toschi argued last year in Organization Science that interdisciplinary researchers and research may threaten disciplines’ and evaluators’ distinctiveness and knowledge domains, and hence, evaluators may penalize them. High-performing, interdisciplinary researchers appeared to suffer the greatest penalties in small and distinctive academic discipline where evaluators appeared as representative members of their disciplines. Fini et al. argued that attempts to maintain social boundaries contributed to the relative lack of interdisciplinary research.

Upcoming Articles in This Series

“A Quick Examination of Existing Academic Impact Metrics and Concerns in Business Education” | Usha Haley and Andrew Jack

“How Might Societal Impact be Recognized within an FT Top 50 Journal?” | Renate E. Meyer

“A Decades-Long Journey of Marketing and Public Policy Research to Support the Greater Good” | Maura L. Scott

“Why Don’t Business Schools Publish More Impactful Research?” | Ben McLeish and Mike Taylor

“Some Opportunities for Future Business & Management Research: Employee Health and Well-Being” | Sir Cary L. Cooper

“Efforts to Turn the Tide” | Usha Haley and Andrew Jack

“Medium And Short-term Recommendations to Move Forward” | Usha Haley and Andrew Jack