In Strategic Management Studies, ‘It’s the Measurement, Stupid!’
Editor’s note: We are pleased to welcome Dan R. Dalton and Herman Aguinis, both of Indiana University, whose article “Measurement Malaise in Strategic Management Studies: The Case of Corporate Governance Research” was published in the January 2013 issue of Organizational Research Methods.
Our article describes a surprising commonality in empirical results relying on agency theory in general and in the particular domain of corporate governance research: Mean estimates of relationships between measures of boards of directors’ independence, CEO duality, equity holdings, and corporate financial performance are disappointingly low. Typically, measures of board independence expl
ain only about 1% of variance in relevant outcomes. We use meta-analytic evidence to document that this disappointing result is highly consistent across very diverse bodies of research. Our article was motivated by our desire to understand the reason(s) for such small effect sizes, because this knowledge would allow us to design and execute better studies in the future which will hopefully lead to an improved understanding (i.e., larger effect sizes) of critical outcomes such as firm performance.
Our conclusion can be summarized in a simple phrase: “It’s the measurement, stupid!” Threats to construct validity such as less ideal operationalization of constructs and confounding constructs and levels of constructs seem to be the culprits, at least in part, for why observed effect sizes are so small. How did we arrive at this conclusion? We implemented a five-step protocol through which we (1) established the base rate for the phenomenon in question, (2) evaluated the extent to which the dependent variables are germane, (3) evaluated the extent to which the independent variables are germane, (4) determined whether explanatory power is improved as a consequence of improved measurement, and (5) concluded whether previously established estimates should be revised. Due to the implementation of our proposed five-step protocol, we were able to improve variance explained in outcome variables many times over—in some cases, a tenfold increase.
So, what are the main take-aways? First, remember the motto “It’s the measurement, stupid!” In most empirical research in macro studies, small effect sizes can be expected even before data are collected due to construct validity problems. Second, implementing our proposed protocol in other research domains is likely to lead to more accurate, and larger, effect estimates of relationships among constructs.
Click here to read “Measurement Malaise in Strategic Management Studies: The Case of Corporate Governance Research” in Organizational Research Methods.
Dan R. Dalton is the founding director of the Institute for Corporate Governance, Dean Emeritus, and the Harold A. Poling Chair of Strategic Management in the Kelley School of Business, Indiana University. A Fellow of the Academy of Management, his research focuses on corporate governance and research methods and analysis. He is the recipient of the 2011 Academy of Management Research Methods Division Distinguished Career Award and a former editor-in-chief of the Journal of Management.
Herman Aguinis is the Dean’s Research Professor, a professor of organizational behavior and human resources, and the founding director of the Institute for Global Organizational Effectiveness in the Kelley School of Business, Indiana University. His research interests span several human resource management, organizational behavior, and research methods and analysis topics. He has published five books and more than 100 articles in refereed journals. He is the recipient of the 2012 Academy of Management Research Methods Division Distinguished Career Award and a former editor-in-chief of Organizational Research Methods.
