★ My

Encyclopedia › Trading terms ›

P-hacking and data snooping

In one line: Trying many tests until one looks good by luck, then only showing that one.

Test 100 random rules and a few will look great by pure chance. Show only those and you have p-hacking.

Head and colleagues (2015, PLOS Biology) describe how common p-hacking is in science. Harvey, Liu and Zhu look at finance. They argue there has been so much data mining that the usual bar for a "real" result (a t-ratio above 2.0) is too lowThe highest and lowest price of the past year. for new findings.

In trading the same thing is called data snooping (Sullivan, Timmermann and White 1999). Ask any strategy seller: how many versions did you try?

Related

Sources & studies

  1. Head et al. (2015), The Extent and Consequences of P-Hacking in Science, PLOS Biology
  2. Harvey, Liu & Zhu, …and the Cross-Section of Expected Returns, NBER w20592
  3. Sullivan, Timmermann & White (1999), Journal of Finance

Trading termsOverfittingSample size

Last checked: 2026-11-04 · Educational, not financial advice.

Education only, not financial advice. No signals. Backtests are hypothetical: real trading can be worse. Past results don't predict future results.