Enhancing the Role of Local Input for Measuring Value

In this third response to Ziyad Marar’s thought piece “On Measuring Social Science Impact” from Organization Studies, professor Laura Rovelli, one of the advisory board members for the Declaration on Research Assessment, or DORA, discusses some of the components of impact beyond citation count and how we can harness those components.

The relevance or lifespan of social and behavioral science encompasses, from an instrumental approach, at least two aspects.

The first reflects their contribution to the design and enforcement of diverse public policies that align development with equity, diversity, inclusion, and sustainability. These are broadly expressed in some of the 17 Sustainable Development Goals of the United Nations 2030 Agenda.

The second aspect, and one inherent to these disciplines, relates to the production of critical knowledge and its inputs to reflexivity in science. This can be seen, for example, through the identification of scientific commons, the relationship between new heritages and new technologies, alongside analytical and methodological perspectives, theories, and concepts. This type of knowledge is oriented towards conviviality and the different collective forms of living together. It involves key dimensions of social interaction and organization, ways of dealing with social demands and conflicts, and it nourishes public debate in democratic societies.

One example is the role of historians opening the hidden archives and processing the memories of 20th century dictatorships in many Latin American and Caribbean countries. A more recent example is the actions of social scientists, feminists and diversity movements to implement same-sex marriage and voluntary, legal abortion; these resulted in little recognition and reward in academic and scientific research assessment ecosystems, but had a wide and transformative impact on the landscape for our societies.

International diagnoses, in the form of declarations of principles, statements, recommendations, and studies, have warned about reliance on journal impact factors or citation-based indicators as a proxy measure to assess the quality of individual trajectories and performance in academia and research articles’ contributions in all disciplines (DORA, 2013; Hicks, et. al., 2015). Particularly in social and behavioral sciences, the extended misuse of these measures limits the local autonomy and discourages research from interacting with society (Ráfols, 2019). It also restricts different forms of knowledge creation and communications, which can include peer-reviewed journals but more extensively affects reports, lectures and books (UNESCO, 2021).

In this scenario, the Latin American Forum for Research Assessment (FOLEC, in Spanish) from the Latin American Council of Social Sciences (CLACSO) encourages knowledge circulation indicators and methodologies that assess and value different scales, varied registers, linguistic diversity and different audiences, based on the profile of the researcher or institution under evaluation (CLACSO, 2021). Therefore, it promotes bibliodiversity and advocates multilingualism, and favors development of socially relevant research that helps sustain cultural diversity (CLACSO, 2022). It also complements the notion of impact in research assessment with the notion of collaboration and participation in research processes. This can be seen in relevance and social interaction indicators, social intervention and creation for social purposes indicators, and other measures drawn from the tradition of the social and behavioral sciences.

To better incentivize impact that goes beyond citation count, funding agencies and academic institutions should advance regional collaboration for adequate federated infrastructures, scientific information systems and interoperable databases, particularly in Latin America and the Caribbean. The scientific information and data contained in the region’s institutional repositories, and evaluated by peers, could contribute to the design of new indicators more in line with the diversity of existing knowledge in social and behavioral sciences and its rootedness in societies. At the same time it can strengthen the components of open and reproducible research, open research assessment, open access and open data, as part of a global and new social pact for science, all pivoting on research assessment systems reform.