rowcheck: numeric claims checked against the paper's tables

contradicted

TÜLU 2 reaches an 85.1% win rate on AlpacaEval.

Table 4 (p. 8) reports 73.9, not 85.1%, for TÜLU 2 / AlpacaEval. 85.1 is TÜLU 2+DPO's AlpacaEval (the next row), Table 4 p. 8.

arXiv 2402.00838 · Table 4 · p. 8
Table 4, page 8, checked row marked
contradicted

Mistral 7B reaches 52.2% on MATH.

Table 2 (p. 4) reports 13.1%, not 52.2%, for Mistral 7B / MATH. The same row prints 52.2% under GSM8K.

arXiv 2310.06825 · Table 2 · p. 4
Table 2, page 4, checked row marked
contradicted

Gemma-2 9B scores 68.6 on GSM8K in a 0-shot setting.

Table 13 (p. 7) reports 68.6 for Gemma-2 9B / GSM8K, but not under 0-shot (the table says 5-shot).

arXiv 2408.00118 · Table 13 · p. 7
Table 13, page 7, checked row marked
contradicted

Stack-Edu has 21.8T Python tokens.

Table 7 (p. 20) reports 21.8 in B units for Stack-Edu Python / Stack-Edu tokens; the claim says T.

arXiv 2502.02737 · Table 7 · p. 20
Table 7, page 20, checked row marked
ambiguous

Mistral 7B scores 60.1% on MMLU in a 0-shot setting.

Table 2 (p. 4) reports 60.1% for Mistral 7B / MMLU; the table does not state 0-shot.

arXiv 2310.06825 · Table 2 · p. 4
Table 2, page 4, checked row marked
supported

Gemma-2 27B scores 75.2 on 5-shot MMLU.

Table 13 (p. 7) reports 75.2 for Gemma-2 27B / MMLU.

arXiv 2408.00118 · Table 13 · p. 7
Table 13, page 7, checked row marked