Compare AI fiction models on the same scene

A model can win a general benchmark and still write dialogue you do not want in your novel. A useful fiction comparison tests the work you actually need: drafting a scene, respecting established facts, and revising without breaking something else.

This page gives you a repeatable test sheet. It is not a new measured model leaderboard. The sample prose on our walkthrough was generated previously for style selection, and cost-calculator figures are estimates. Neither is evidence that one model is the best fiction writer.

Hold the scene and context fixed

Save a short premise, a fact sheet and one chapter instruction. Use exactly the same versions for every model. Here is a scene you can copy:

“The courier reaches a sealed city gate in heavy rain. A sealed letter must reach the council before dawn. The gatekeeper refuses entry. End with the courier outside, deciding to look for another route. Write about 800 words. Do not reveal the letter's contents.”

The fact sheet should state that the letter has never been opened, the gate remains closed until sunrise, and the courier does not know another route yet. These facts create observable failures: a model that reveals the letter, opens the gate without explanation, or moves the deadline to noon has not passed this scene.

Keep output-length targets and available generation settings comparable. Record any setting you cannot hold constant. Different providers and reasoning modes can change both time and cost, so record those rather than hiding them behind a model name.

Use a rubric before reading

Dimension	Question	Record
Continuity	Were the sealed letter, location and deadline respected?	Violations with quoted lines
Scene movement	Does each exchange change the situation?	Where the scene stalls
Voice	Does the prose fit the register you want?	Examples you would keep or edit
Revision	Can it fix a problem without introducing another?	New errors after the same request
Practical cost	What did the usable scene require?	Draft and revision costs, plus elapsed time

A fluent paragraph can still fail continuity. A cheap draft can become expensive if it needs four revisions. Keep those observations separate instead of collapsing everything into a mysterious quality score.

Give every model the same revision

Ask: “Make the gatekeeper's refusal more tense and less casual. Keep the gate closed, the letter sealed, and the deadline before dawn. Preserve the ending outside the gate.”

Reread the full result. If the revision fixes dialogue but opens the gate, record both outcomes. Count rejected attempts too: a model's cost per usable scene includes the drafts you could not use, not just the successful one.

For a comparison you intend to publish, repeat the exercise on more than one scene. Include dialogue, an action sequence and a quiet scene. Use multiple attempts, shuffle labels before judging if possible, and keep exact prompts and excerpts alongside your conclusions. A single favorite passage supports a preference, not a universal ranking.

Check cost at the length of your actual novel

A first scene is a poor forecast of chapter fifty. Earlier chapters increase input, compaction changes what remains in context, and caching depends on the provider and your writing sessions. Use the AI novel cost calculator to compare the same chapter count, length and revisions across models; then compare those estimates with your actual running spend.

The repository documents a repeated-request cache test on Claude Haiku 4.5: a cold request cost about $0.019 and an identical repeat about $0.0011. That is an internal cache measurement, not a current provider price quote or a prose-quality comparison. It illustrates why repeated context can matter to the bill.

Start with the sample scene, read choosing a model, and use the continuity checklist when you run your own test.