statcheck is a "spellchecker" for statistics. It checks whether your p-values match their accompanying test statistic and degrees of freedom.

statcheck searches for null-hypothesis significance test (NHST) in APA style (e.g., t(28) = 2.2, p < .05). It recalculates the p-value using the reported test statistic and degrees of freedom. If the reported and computed p-values don't match, statcheck will flag the result as an error.

If you click on a result that statcheck flagged as an error, you can see the recalculated p-value. Click on "Go to test" to jump to the location of the test in your document.

To fix any errors, go to your statistical software to check which of the three numbers (test statistic, degrees of freedom, and/or p-value) you need to correct.

statcheck recognizes correlations and t, F, χ², Z tests and Q tests, as long as they are reported completely (test statistic, degrees of freedom if applicable, and p-value) and in APA style.

statcheck takes into account that test statistics and p values may be exactly (=) or inexactly (< or >) reported. Different spacing has also been taken into account.

By default, statcheck treats all tests as two-tailed. If you want to take into account one-tailed tests, you can check the box "Correct for one-tailed tests".

When this box is ticked, statcheck will search the entire text for the keywords "one-tailed", "one-sided", and "directional" (taking spacing issues etc. into account). When statcheck finds at least one of those keywords AND an initially inconsistent result would be consistent if it was a one-tailed test, then statcheck treats this case as a one-tailed test and counts it as consistent.

Some common reasons why statcheck doesn't detect some results:

1. The result was not reported according to APA style. This includes minor deviations such as square brackets instead of parentheses, or a semi-colon instead of a comma.
2. The result was not reported completely. statcheck needs three ingredients to detect a result and recalculate the p-value: the reported test statistic, degrees of freedom, and p-value. If one or more of these are missing, statcheck will not pick it up.
3. The result is reported in a table.

statcheck flags result as an error when the reported p-value does not match the recalculated p-value. However, there may be cases in which you deliberately reported an inconsistent result. For example, when you conducted a one-tailed test, reported a Bonferroni corrected p-value, or corrected degrees of freedom.

Of course it is also possible that statcheck really made a mistake and erroneously flagged a result as inconsistent. If you think that statcheck wrongly flagged one of your results, please contact us (see below). For more information about statcheck's accuracy, see the next section.

In typical psychology journals, statcheck detects about 60% of the null hypothesis significance tests. In classifying extracted results as consistent or inconsistent, statcheck has an accuracy between 96.2% and 99.9%, depending on its settings. See Nuijten et al., 2017: for details.

• The manual: A detailed instruction manual with information on what statcheck can and cannot do, information on how to install and use the statcheck R package, and more.
• The web app: Upload a paper in one click and get a table of all detected statistics, classified as consistent, an inconsistency or a decision inconsistency.
• The R package: The R package has additional functionality which allows you to change more settings and to scan entire folders of papers.
• The paper: The seminal paper in which statcheck was introduced. We ran statcheck on over 30,000 psychology papers and report general inconsistency-prevalences over time and per journal.
• The validity study: We compared statcheck's performance with manual checks and assessed its accuracy in classifying results as consistent/inconsistent
• The GitHub page: Here you can find statcheck's latest developments.

You can contact us directly via Twitter at @MicheleNuijten and @WillemSleegers.

For other ways to contact us, see Michèle's website or Willem's website.