Exploring the unconventional: Nonstandard error patterns in finance

GSEM Professor Olivier Scaillet, former Ph.D. student Gaetan Ballaki, and more than 100 researcher teams co-authored an article published in the top-tier Journal of Finance. In a groundbreaking experience, the Finance Crowd Analysis Project gathered experts all around the globe to analyze a dataset containing more than 650 million trades in the Euro Stoxx 50 futures contract. Their goal is to study predefined hypotheses related to empirical finance. By examining analysis pluralism and knowledge formation, the project aims to provide insights into price discovery, bid-ask spreads, trade frequency, and trading revenues. The teams evaluate their findings through short papers, which undergo an expert review process. This initiative encourages global collaboration and offers opportunities for innovative research in financial metrics.

ABSTRACT

In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty—nonstandard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for more reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.

The study is available open access: Nonstandard Errors

> Click here to view the GSEM faculty’s publications in top-tier journals.