Looking for Impact? Seek Feedback — Especially From Friends in High Places
Researchers in many fields present their findings at research seminars in universities and at conferences long before their paper is sent to a journal. These researchers receive feedback from colleagues, too. This type of informal intellectual collaboration is a vital part of modern science. Editors of some top journals go so far as to urge authors to receive feedback before submitting their papers to increase the quality of submitted manuscripts:
“So how should authors maximize the value of the journal review process? They should circulate their papers and give seminars to colleagues to receive constructive criticism before submitting to a journal.”
In our new research, we study the importance of informal intellectual collaboration in our own field, financial economics. Estimates of our study suggest that every additional commenter or seminar does indeed increase scientific impact (as measured by number of citations over an article’s lifespan). It also tends to increase the probability of publishing in a top journal.
This article by Co-Pierre Georg and Michael E. Rose originally appeared on the LSE Impact of Social Sciences blog as “Feedback helps increase the impact of academic research, even more so when coming from well-connected colleagues” and is reposted under the Creative Commons license (CC BY 3.0).
However, and this is the novelty of our paper, it matters with whom authors discuss their work. Feedback from researchers connected to other well-connected researchers has a much larger impact on the publication success of an article. Interestingly, we show that these well-connected academics are not necessarily particularly well-known, very senior, or affiliated to prestigious universities. Instead, it is their structural connectedness that matters. Highly connected commenters may point authors to relevant new topics or the most rewarding avenues for future research. This is especially important in an environment where publication of results takes time, but unpublished results already matter for new research. This has led others to argue that “a researcher who is close to more productive researchers may have early access to new ideas.”
Our measure of connectedness is the eigenvector centrality. The idea behind it is simple: if you know important people, you are probably important as well. Technically, it is the weighted sum of neighbors, where the weights correspond to your neighbor’s eigenvector centrality. For this reason, eigenvector-central individuals are clustered together. Think of highly eigenvector-central individuals as being the opinion leaders in the social network of a research field; those academics who are highly influential without necessarily being especially senior or prolific. Google’s PageRank algorithm builds on eigenvector centrality, too.
We manually built a new database of formal and informal collaborations from published papers and constructed a social network to assess each individual’s network position. In economics, authors commonly acknowledge where they have presented their work (i.e. in seminars or at conferences) and from which colleagues they have received helpful feedback. We collected the acknowledgement sections of more than 5,800 research papers from six major finance journals. More than 90 percent of these acknowledge at least one form of informal collaboration, highlighting the importance of informal collaboration in financial economics.
After consolidation, we created a social network from all the commenters and authors that somehow contributed to financial economics research in our six sample journals between 1998 and 2011. A link is formed when they have co-authored an article or one acknowledges the other for exchange of comments, feedback or ideas. This network connects nearly 7,500 researchers and indicates information flows between them. Such a social network analysis helps uncover patterns and structures that remain hidden when looking at individual researchers only. To make the analysis feasible we constructed 12 networks from 2000 to 2011, each spanning three years. The most recent network for 2009-2011 is shown in the figure below.
A visualization of the largest connected component of the network of intellectual collaboration in financial economics. A node is a researcher that either published a paper in one of the six journals between 2009 and 2011, or has been acknowledged in this period, or both. Red links indicate a joint paper, blue links indicate an informal collaboration (i.e. one acknowledges the other), while rare purple links indicate both a formal and an informal collaboration. Nodes are coloured according to their Eigenvector centrality rank, with darker colors corresponding to higher ranks. Nodes are positioned according to an adapted Fruchterman-Reingold algorithm. For further details about this study please read the authors’ article.
In a companion paper we also list the most eigenvector-central financial economists in different networks from 1997 to 2011 and contrast their rank with other metrics. We find that common author metrics such as citation count (according to Scopus) or number of published articles only weakly correlate with rank positions according to eigenvector centrality. Part of the answer is that author metrics poorly capture academic strength. It might thus be that academics themselves are better able to identify key figures, which is why they turn to them for advice. This view is backed by the weak correlation between eigenvector centrality and other centrality measures, such as degree (the number of neighbors) or betweenness centrality, which proxies the importance of a single researcher in the flow of knowledge. Researchers can trace themselves on our website, where we also provide a view of the network at different times.
Since the causal links and mechanisms are not fully understood, more research on the topic is warranted. Understanding knowledge flows helps understanding of productivity differentials among scientists. These insights can help to improve the innovative process of science; guiding researchers to find appropriate collaboration partners, helping shape better incentives at the university level or even improving the publication process as a whole. Our findings might also help forge better funding schemes, given the crucial role of academic conferences as a venue to network and disseminate results for scientific progress.
If there is any advice for young scholars that can be offered at this stage, it is this: young scholars must not be afraid of receiving feedback, even, and especially, if negative; and they must learn early on to present their work well and as often as possible.
***
This blog post is based on the authors’ article, ‘The importance of informal intellectual collaboration with central colleagues’, available on the Social Sciences Research Network (DOI: 10.2139/ssrn.2877586).
