Research

Making the Invisible Visible

June 2, 2022 1589

In this post, Jessica Villiger examines the process of coding and how it can be used in meta-analytical research projects. The reflection draws from the article, Making the Invisible Visible: Guidelines for the Coding Process in Meta-analyses, by Villiger, Simone A. Schweiger and Artur Baldauf, all of the University of Bern, published in Organizational Research Methods.

Meta-analyses are seen as the lighthouses that help entire research communities in shedding light on a plethora of future academic research avenues. However, despite their pertinence for academia, we experienced that there was little methodological guidance on one of ‘the’ key and most time-intensive steps in meta-analytic research projects – Coding.

With our article, “Making the Invisible Visible: Guidelines for the Coding Process in Meta-analyses,” published in the special issue of Organizational Research Methods, we address this research gap by developing guidelines for the process of coding. Our article empowers novice as well as experienced meta-analysts with practical advice on ‘how’ to plan, conduct and report their coding process.

We believe that the existing void in methodological literature is also partially responsible for the common misconception that coding is a purely technical task (i.e., the extraction of numeric data from studies to run meta-analytic calculations). Although we know and learned that coding encompasses much more. In fact, coding is the process of translating independent studies into a common language. Hence, researchers performing the coding task aim to make sense of the variables represented in these studies by examining and extracting their labels, definitions and measurements. This, in turn, allows a meaningful comparison and syntheses of empirical findings.

Intrigued by the question of how authors report their own coding process, we analyzed 124 meta-analyses published in top-tier journals. Our analysis revealed that only a small number of authors transparently described their coding process. This is problematic because it limits readers to fully comprehend the authors’ decision-making process, for instance, on which basis the authors developed their coding categories, and consequently to assess a meta-analysis’ quality. As such, the coding process remains the invisible architecture behind meta-analyses.

Backed with these insights we contacted the authors of the 124 meta-analyses to find out whether coding has been an issue addressed by editors and journal reviewers during their review process of these top-tier journals. Our survey findings indicated that authors have been indeed requested to describe their coding, but mostly only on a superficial level by reporting the number of coders involved and their intercoder reliability value (i.e., the extent to which coders agreed on their coding). Hence, their decision-making process on how to build coding categories or the depth of data-extraction was seldom assessed.

Moreover, the majority of authors participating in our survey stated that they were not aware of any guidelines for coding and thus, would be very grateful if such a coding manual were to exist. This information supported us in pursuing our research idea of thoroughly developing guidelines for the coding process.

Jessica Villiger, left, Simone A. Schweiger and Artur Baldauf.

Besides the content-analysis and survey, our article also offers a state-of-the-art assessment of prior advice given on the topic of coding. We added to this extant knowledge our four-step guidelines, enriched with practical examples from our own meta-analytic experiences. When referring to our guidelines, authors of meta-analyses and their readers should be able to get answers to certain questions: What do meaningful coding practices entail? What are main pitfalls? Or to what extent should the coding process be reported?

Finally, our research also aims to aid editors and reviewers on how to support authors in improving the quality of their coding and its subsequent reporting. For that reason, our article seeks to contribute to the ongoing paradigm shift of more transparent reporting of coding practices. We believe making the invisible architecture of meta-analyses visible is important because it helps researchers to better use insights from prior meta-analyses in their own work, which consequently enables them to contribute more pertinently to the advancement of science overall.

Are you curious to find out more? Then check out now “Making the Invisible Visible: Guidelines for the Coding Process in Meta-analyses.”

Jessica Villiger is a Post Doc/Senior Researcher for the Department of Management & Entrepreneurship at the University of Bern, a public university in Switzerland.

View all posts by Jessica Villiger

Related Articles

The End of Meaningful CSR?
Business and Management INK
November 22, 2024

The End of Meaningful CSR?

Read Now
Boards and Internationalization Speed
Business and Management INK
November 18, 2024

Boards and Internationalization Speed

Read Now
How Managers Can Enhance Trust
Business and Management INK
November 11, 2024

How Managers Can Enhance Trust

Read Now
Exploring the Citation Nexus of Life Sciences and Social Sciences
Industry
November 6, 2024

Exploring the Citation Nexus of Life Sciences and Social Sciences

Read Now
The Role of Place in Sustainability

The Role of Place in Sustainability

In this article, co-authors Arno Kourula, Panikos Georgallis, Irene Henriques, and Johanna Mair reflect on the inspiration behind their research article, “Introduction to the Special Issue […]

Read Now
Turning to Glitter in Management Studies – Why We Should Take ‘Unserious’ Glitter Serious to Understand New Management Practices

Turning to Glitter in Management Studies – Why We Should Take ‘Unserious’ Glitter Serious to Understand New Management Practices

In this article, author Jette Sandager reflects on the inspiration behind her research article, “The sensuous governmentality of glitter: Educating managing women scientists […]

Read Now
Exploring the ‘Publish or Perish’ Mentality and its Impact on Research Paper Retractions

Exploring the ‘Publish or Perish’ Mentality and its Impact on Research Paper Retractions

When scientists make important discoveries, both big and small, they typically publish their findings in scientific journals for others to read. This […]

Read Now
3 2 votes
Article Rating
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

0 Comments
Newest
Oldest Most Voted
Inline Feedbacks
View all comments