Does Your Data Suffer from Common Method Variance?

In this post, authors Brian K. Miller and Marcia J. Simmering reflect on their recent research article, Attitude Toward the Color Blue:  An Ideal Marker Variable,” published in Organizational Research Methods.

We wanted to help researchers resolve the age-old question of determining if their data suffered from common method variance (CMV) which is the tendency for the relationship between variables collected at one time, from one source, via one method to have artificially inflated correlations. 

This paper is based off of a conference paper by Brian that was presented in 2008. In that paper, a scale designed to be unrelated to anything else in the data set was developed called Attitude Toward the Color Blue (ATCB). Unbeknownst to Brian, he had created an ideal marker variable that others started to find useful. Since then, Brian has had many requests to use the items, which was confirmation that this variable had real value to researchers. Several years ago, Marcia co-authored a paper in Organizational Research Methods that found that of all the ideal marker variables being used, that ATCB likely performed well. Brian reached out to Marcia, and together we planned a full-blown multi-sample scale development paper to refine the ATCB scale. 

The biggest challenge for Brian was cracking the secret Mplus code used to test the very complicated models as part of the CFA Marker Technique and to not lose track of the steps in the process for each sample in the paper. For Marcia the challenge was helping reviewers and peers understand that most other so-called ideal marker variables were far from ideal and that the world truly needed a real ideal marker variable. 

We painstakingly demonstrated the best practices in scale development procedures and we verified many useful features of the scale. We hope that any researcher who collects cross-sectional same-source data uses our items, downloads our thoroughly annotated Mplus code, and tests whether their data suffers from CMV. Having this ideal marker variable and the code to run the CFA Marker Technique should go a long way in helping other researchers verify that their data is free from CMV.

We think that everything a researcher would want to know about the scale is in the paper, even why we chose blue over other colors! We sincerely thank Dr. Louis Tay, the action editor, and the anonymous reviewers at ORM who helped us along the way and pushed us to explore ideas that we wouldn’t have addressed if not for their insight.

In a research world where it seems that all of the good ideas have been examined already, new scholars should bounce their ideas off of lay persons, not just other academics. Oh, and find good co-authors too!  Being able to collaborate with someone whose skillset complements yours, who you trust, and who you enjoy working with, makes it easier to publish.

The most influential paper that Brian read was: Simmering et al. (2015).  “Marker variable choice, reporting, and interpretation in the detection of common method variance:  A review and demonstration” in Organizational Research Methods.

The most influential paper that Marcia read was: Aguinis et al. (2020). “Best-practice recommendations for producers, evaluators, and users of methodological literature reviews.” Organizational Research Methods.