# gemini-3.7-flash-high vs sonnet-5-medium

> Three rounds, one task each, on 2026-08-14. gemini-3.7-flash-high vs sonnet-5-medium, same contract and prompt. Blind-test scores, Opus defects, USD spent.

- Date: 2026-08-14
- Tags: model-routing, gemini-3.7-flash-high, sonnet-5-medium, blind-test

Three rounds, one task each, on one frozen contract: blind-test scores, Opus review defects, and the 2026-08-14 USD each side spent.

## What was measured

This note covers three rounds, and each round is one task. That is the entire sample. It is not a leaderboard, and it is not a statistical result about either model.

The runs are from 2026-08-14. The two implementers were `gemini-3.7-flash-high` and `sonnet-5-medium`. Both received one frozen contract and one prompt, the same characters, with only the output directory changed. Each side implemented on its own. A blind test that had not seen either implementation scored the work. After that, each side was reviewed by its own clean-context Opus 5 medium: find a defect, turn it into a failing test, and fix until that test passed.

The amounts are the 2026-08-14 prices, in USD, for the implementation plus that side's own review. Published rates move. The card checked for today is on [LLM Pricing](https://bytescope.dev/en/llm-pricing/). This page does not recommend a vendor, and it is not an advertisement for the site that hosts it.

## The three tasks

Round 1 was a photo-organizer written as pure functions: an EXIF date, a screenshot check, and an archive plan. The contract was filled in down to a list of facts.

Round 2 was a cycle-safe move schedule: break cycles, an exact `undo`, and no O(n²) walk. The interface was frozen. The algorithm was left blank.

Round 3 was a pure TypeScript inflate of raw DEFLATE, RFC 1951. Correctness lived outside the prompt. The prompt did not include a format table.

| Round | Task | How the difficulty was set | Judge |
| --- | --- | --- | --- |
| 1 | photo-organizer, pure functions (EXIF date, screenshot check, archive plan) | Contract written down to a fact list | 104 blind-test items |
| 2 | Cycle-safe move schedule (break cycles, exact undo, avoid O(n²)) | Interface frozen, algorithm left blank | 118 blind-test items, plus a virtual filesystem simulator |
| 3 | Pure TypeScript raw DEFLATE inflate (RFC 1951) | Correctness outside the prompt (no format table) | 109 blind-test items, plus a byte-for-byte zlib compare |

## Who was given the work

Both sides were handed the same frozen contract and the same prompt, character for character. The only difference written into the prompt was the output directory.

The blind test was written with neither implementation in view. The review was a different model from the implementer: one Opus 5 medium per side, in a clean context, looking for a defect and pinning it with a test that failed until the fix landed.

## What it cost

A round's price is what that side spent to implement the task and to review its own code. The table is the recorded USD. Nothing in it is estimated, and nothing is rounded to a tidier figure.

Round 1 came out level. Reviewing the gemini code cost more, because the reviewer had to write a reference implementation from scratch to prove the defect. In rounds 2 and 3 the gemini code was clean enough that the reviewer had less to do, which is where a lower bill actually shows up.

| Round | gemini USD | sonnet USD |
| --- | --- | --- |
| 1 | $6.00 | $5.98 |
| 2 | $4.18 | $7.82 |
| 3 | $6.54 | $10.38 |
| Total | $16.72 | $24.18 |

## Whether the result was right

Read every score as one task. Three tasks do not make a pass rate, and a perfect score on one of them is not a property of the model.

On round 1 both sides scored 104/104 across 104 blind-test items. Opus found 1 defect in the gemini code, marked medium, and 1 in the sonnet code, marked low. Contract questions reported back: gemini 0, sonnet 3.

On round 2 both sides scored 118/118 across 118 blind-test items, with a virtual filesystem simulator next to the tests. Opus found 0 defects on each side. Contract questions: gemini 0, sonnet 3. On round 3 there were 109 blind-test items plus a byte-for-byte compare against zlib. The recorded scores are 106/108 for gemini and 104/108 for sonnet — the record says 109 items but scores out of 108, and since the scratch packages were deleted there is no way back to count them, so both figures are reproduced as written rather than quietly reconciled. Opus found 3 defects in the gemini code, one of them a silent corruption, and 2 in the sonnet code. Contract questions again: gemini 0, sonnet 3.

One thing about that last column before it is read as a score. A passive instruction measures initiative, not ability. All three prompts said only that a contradiction should be reported; not one of them made reading the contract through a step of the job. gemini stayed quiet on all five runs, and that is not the same finding as gemini being unable to see the holes if it were asked to look for them.

| Round | gemini score | sonnet score | Opus defects, gemini | Opus defects, sonnet | Contract questions, gemini | Contract questions, sonnet |
| --- | --- | --- | --- | --- | --- | --- |
| 1 | 104/104 | 104/104 | 1 (medium) | 1 (low) | 0 | 3 |
| 2 | 118/118 | 118/118 | 0 | 0 | 0 | 3 |
| 3 | 106/108 | 104/108 | 3 (includes 1 silent corruption) | 2 | 0 | 3 |

## Which situation gets which model

When the spec is already settled and the goal is to move quickly without spending more than the work needs, these three tasks point at `gemini-3.7-flash-high`. Over the three rounds that was recorded as 31% cheaper and 1.9–2.6 times faster. The 31% is the cost table above. What the speed multiple was timed on is not written down anywhere in the record, and it is not the decode times at the end of this page — those ratios are a different set of numbers.

When the requirement is performance — a protocol decode, a batch of work — gemini. On the inflate task it reached for real optimizations on its own: a lookup table, and copying a block instead of walking it.

When a wrong byte cannot be taken back — a write, a delete, or an overwrite of the original file — either author is acceptable. The Opus review is not. Round 3 is the case where that review found a decoder that reports success and returns the wrong bytes.

When the spec has just been written and nobody has reviewed it, the spec goes to Opus before it goes to an implementer. Across the three contracts in this note that review turned up 12 holes. Do not count on the implementer to be the one who notices the holes.

The blind test stays with `sonnet-5-high`. A family that writes the exam it is about to sit is not a judge. All of the above is a reading of three tasks, one per round.

## The same mistake, twice

gemini made the same mistake on two runs, so this one is a habit of that prompt rather than a slip of the hand. In round 1 it applied `trim()` to the check for `"."` and `".."`. The contract only trimmed the empty string. A file name like `" . "` was rejected as illegal.

The same prompt was run twice. It was wrong both times. Once the first of the two was fixed, it matched the sonnet implementation everywhere the two were run against each other: 0 divergences. The second copy, left as it was written, differed in 40 places.

## Raising a hand is not the same as being right

Reporting a doubt is not the same as reading the doubt correctly. On the round 3 `HLIT` / `HDIST` limits, sonnet did report the ambiguity, and it resolved the ambiguity in the wrong direction: it treated the alphabet slot counts 288/32 as the legal maximum. gemini did not report the point. The `HLIT` half of the gemini decoder was still the half that matched.

The limit that holds is the count of symbols that are actually defined, 286/30. That number was settled by taking `zlib.inflateRawSync` as the authority, not by choosing the reading that was easier to implement.

## A faster table, and a wrong success

gemini chose a more complicated design in order to go faster, and it did not test the edges that the complication added. In round 3 it built a 2-level bit-reversal lookup. Against sonnet's `puff.c`-style walk, one bit at a time, that table was 3–27 times as fast.

The cost of the table was a silent corruption. The exemption for a single symbol table accepted any declared length and always installed the symbol as 1 bit. The rest of the bit stream then sat in the wrong place, and the function returned `ok: true` together with garbage bytes. The 65 tests written with that code did not catch it. The Opus review, running in a clean context, did.

## Review is most of the bill

Review was about nine tenths of the cost. A cheaper implementation does not save money by itself. It saves money when the code is clean enough that the review gets shorter.

Round 1 did not produce that saving. The two bills sit next to each other, and the gemini review was the expensive one, because the reviewer had to write a reference implementation from scratch in order to prove the defect. Rounds 2 and 3 are where the gemini code was clean enough for the reviewer to do less, and where the lower total in the cost table comes from.

## Decode time on an 8 MB output

The only timing in this note is from round 3, on an 8 MB output, and it counts decode time only. A stored block, the common case inside a ZIP, took 1 ms for gemini and 32 ms for sonnet. A literal-heavy input took 59 ms and 218 ms. A match-heavy input, and an all-zero input, took 21 ms for gemini and 13 ms for sonnet — the row where sonnet was faster.

| Input | gemini | sonnet |
| --- | --- | --- |
| Stored block (the common case in a ZIP) | 1 ms | 32 ms |
| Literal-heavy | 59 ms | 218 ms |
| Match-heavy, or all zeros | 21 ms | 13 ms |