Adding Inkling-Small-2
Inkling 2 Small is a new model by thinking machines, let’s run it with different levels of reasoning effort. Background: endgame-120 is testing reasoning ability. Models need to solve a puzzle in a single turn, benchmark has 120 puzzles which are not published, while the harness for running is here.
It can solve puzzles at high reasoning effort, but uses a lot of tokens for that, even more than deepseek flash. Inkling supports arbitrary reasoning effort in [0; 1) range, with low/medium/high being defined as 0.2/0.7/0.9. Running simul setup will be hard - replicating Deepseek Flash study will likely exceed the context limit of 256K.
When running on Thinking Machines API endpoint each run was fairly slow - 50-70 tps, which translates to > 20 min for puzzle at high reasoning, so parallel runner is a must.