7  Publish

A clear empirical methodology forms the foundations of any empirical research work. However, to ensure the wide dissemination of a study, two aspects are crucial: accessibility and dissemination. Nowadays, public accessibility of a paper is relatively easy to achieve because anyone can upload their work online to ensure its availability. Dissemination, however, is more complex. While there are various strategies to increase the popularity of a paper, the traditional and arguably most effective method is to publish the paper in a high-impact journal or a popular journal with a large readership. The successful placement of a paper in such journals is almost an art form. The paper must fulfill several criteria as well as possible: a sound methodology, a clear contribution, thematic relevance and a compelling text that appeals to the journal’s readership are just some of the key benchmarks.

How can you learn to publish a paper successfully? I think that it is crucial to study papers that have gained recognition and make an effort to recognize the reasons for their success. Doing so sharpens your writing skills, improves your perceptiveness as a researcher, and helps you to select important research topics and to design research projects more efficiently.

Prestigious journals like the American Economic Review and the Journal of Economic Literature, published by the American Economic Association (AEA), have refined and enhanced their guidelines over time. These rules are often regarded as a benchmark in academic publishing, not necessarily because they set the highest possible standards, but because they strike a balance that takes into account the practical realities faced by researchers, reviewers, and editors. Moreover, readers often prefer a swift review process, too, because they want papers that connect with current events. In that respect, it’s important to acknowledge the inherent trade-off journals must navigate: stringent standards can slow the pace of research and publication. This is particularly critical in fields like economics and business, where timely insights can significantly impact politics and society, addressing urgent needs.

This section looks at the AEA’s publishing standards. The path to publication with the AEA, particularly for empirical research, has changed considerably and is now governed by strict criteria. It is imperative that authors familiarize themselves with the key considerations and guidelines for submitting empirical research to an AEA journal. Although standards may vary across disciplines, the AEA’s exacting standards have influenced a wide range of social science journals. Engagement with these standards is invaluable as it provides guidance in designing and planning research that meets these rigorous requirements.

While each AEA journal has its specific focus and requirements, all aim for integrity, clarity, and replicability of empirical research. Authors should start by carefully reviewing the author guidelines for their targeted journal, paying close attention to any specific mandates regarding empirical work. For Randomized Controlled Trials (RCTs), for example, a registration is required for all applicable submissions prior to submitting, see RCT Registry Policy.

Tip 7.1: The AEA’s registry for randomized controlled trials

Visit www.socialscienceregistry.org and inform yourself about some registered RCTs.

Figure 7.1: The spectrum of reproducibility

The AEA has placed a significant emphasis on the transparency and replicability of empirical research with the objective of ensuring that the empirical results can be fully replicated (see Figure 7.1). But why is replicability so crucial? Well, trust is good, control is better. Or, as Peng (2011, p. 1226) put it:

Peng, R. D. (2011). Reproducible research in computational science. Science, 334(6060), 1226–1227.

“Replication is the ultimate standard by which scientific claims are judged. With replication, independent investigators address a scientific hypothesis and build up evidence for or against it. The scientific community’s “culture of replication” has served to quickly weed out spurious claims and enforce on the community a disciplined approach to scientific discovery.” the Authors are required to ensure that their data and methodologies are openly available and clearly described, allowing other researchers to replicate their results. This commitment to transparency extends to the publication of data sets, code, and detailed methodological appendices, which must accompany the submitted manuscript.

Table 7.1 is taken from Nosek et al. (2015) and it breaks down how scientific journals ask researchers to follow more strict rules, from Level 0 to Level 3, across eight different important categories. All these standards follow the objective to come closer to the gold standard shown in Figure 7.1.

Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015). Promoting an open research culture. Science, 348(6242), 1422–1425.
Table 7.1: Summary of the eight standards and three levels of the TOP guidelines
Level 0 Level 1 Level 2 Level 3
Citation Standards
Journal encourages citation of data, code, and materials—or says nothing. Journal describes citation of data in guidelines to authors with clear rules and examples. Article provides appropriate citation for data and materials used, consistent with journal’s author guidelines. Article is not published until appropriate citation for data and materials is provided that follows journal’s author guidelines.
Data Transparency
Journal encourages data sharing—or says nothing. Article states whether data are available and, if so, where to access them. Data must be posted to a trusted repository. Exceptions must be identified at article submission. Data must be posted to a trusted repository, and reported analyses will be reproduced independently before publication.
Method Transparency (Code)
Journal encourages code sharing—or says nothing. Article states whether code is available and, if so, where to access them. Code must be posted to a trusted repository. Exceptions must be identified at article submission. Code must be posted to a trusted repository, and reported analyses will be reproduced independently before publication.
Material Transparency
Journal encourages materials sharing—or says nothing. Article states whether materials are available and, if so, where to access them. Materials must be posted to a trusted repository. Exceptions must be identified at article submission. Materials must be posted to a trusted repository, and reported analyses will be reproduced independently before publication.
Design Transparency
Journal encourages design and analysis transparency or says nothing. Journal articulates design transparency standards. Journal requires adherence to design transparency standards for review and publication. Journal requires and enforces adherence to design transparency standards for review and publication.
Preregistration: Study
Journal says nothing. Journal encourages preregistration of studies and provides link in article to preregistration if it exists. Journal encourages preregistration of studies and provides link in article and certification of meeting preregistration badge requirements. Journal requires preregistration of studies and provides link and badge in article to meeting requirements.
Preregistration: Analysis Plan
Journal says nothing. Journal encourages preanalysis plans and provides link in article to registered analysis plan if it exists. Journal encourages preanalysis plans and provides link in article and certification of meeting registered analysis plan badge requirements. Journal requires preregistration of studies with analysis plans and provides link and badge in article to meeting requirements.
Replication
Journal discourages submission of replication studies—or says nothing. Journal encourages submission of replication studies. Journal encourages submission of replication studies and conducts blind review of results. Journal uses Registered Reports as a submission option for replication studies with peer review before observing the study outcomes.

Besides adhering to the guidelines summarized in Table 7.1, authors should also:

While statistical significance is a key metric for empirical analysis, the AEA also places a strong emphasis on the economic relevance and implications of the findings. Authors should not only present statistically significant results but also explain their economic significance.

The AEA employs a rigorous double-blind peer review process that ensures that submissions are peer-reviewed without revealing the identity of the authors or reviewers. The editor of the respective journal is responsible to manage the process which involves identifying qualified reviewers and forwarding the paper to them for evaluation. This procedure serves to ensure the integrity and impartiality of the evaluation. Authors should be prepared to receive feedback and constructive criticism, which often requires a revision of the manuscript. A willingness to engage constructively with this feedback is crucial to refining the work and improving its chances of publication. Of course, this process has its disadvantages: It is slow and imposes a significant workload on both reviewers and editors. Considering the vast number of publications each year—a number that has significantly increased over the last few decades (see Paldam, 2021)—and the fact that the guidelines of many journals have become stricter during the same period, it raises a question. Have researchers become miraculously more productive, or are they simply working more?

Paldam, M. (2021). Methods used in economic research: An empirical study of trends and levels. Economics, 15(1), 28–42.