Why I generate math figures in code and SVG, not with an image model like gpt-image or nano banana?

I'm building a math item bank with an AI engine, and the figures have to match the question exactly: an angle labeled 60° has to measure 60°, a triangle with two given angles has to leave the third one positive. No image model gets there consistently, paid or free, no matter how detailed the prompt. It's not a prompt problem, it's a design problem. What I landed on is that the model never draws: it emits a typed spec, a deterministic engine renders it, and a layered verification checks it before the figure reaches a student.

Why can't an image model draw an exact math figure?

My first move was to assume the problem was mine: short prompt, not specific enough, no examples. I made it longer, added numeric constraints, asked for the exact angle with its label. Nothing changed. And this isn't just my anecdote, it's a published finding: "Text-to-Image Diffusion Models Cannot Count, and Prompt Refinement Cannot Help" (arXiv:2503.06884, 2025) evaluates every reference diffusion model, open and closed, and shows none of them count objects reliably, and that refining the prompt doesn't move the needle. It's an architectural limitation, not a prompting one.

The spatial side is even worse than the counting side. T2I-CompBench (arXiv:2307.06350, NeurIPS 2023) measures compositionality across 8 subcategories over thousands of prompts, and the spatial 2D category scores 0.08-0.13 on Stable Diffusion v1-4/v2, against 0.30-0.50 on color and texture. The paper says it plainly: spatial relationships are the hardest subcategory of all, 3 to 5 times worse than color or texture. It's not that the model draws the triangle's color wrong, it's that it doesn't know where to put a vertex.

And in math imagery specifically, MathGen (arXiv:2603.27959, 2026) is the newest benchmark and the closest to what I needed: 900 problems covering counting, angles, fractions, plane and solid geometry, functions and sets, with an executable verifier that checks every generated figure. The best model on the market, nano banana pro, scores 42% overall, and 20% on angles, that's 1 in 5. gpt-image-1.5 gets 35.7% overall and 17.1% on angles. Angles is, for both top models, the worst or second worst domain in the whole benchmark. It's literally the kind of figure I needed to generate.

None of this is a permanent verdict on image models. nano banana pro nearly triples its direct predecessor (14.9%) and Imagen 4 Ultra (14.2%) on the same benchmark. The frontier moves fast. But today, for a figure that has to be exact, that progress isn't enough yet, and a production pipeline can't bet on it being enough next week.

How does the industry actually solve this: code instead of pixels, verification on top?

The pattern that keeps showing up in papers and in products that already ship is the same one: don't ask the model to draw the figure, ask it for the figure's semantic description, and let a deterministic engine compute and draw it.

GGBench (arXiv:2511.11134, 2025) measures this directly on geometric construction: when the model generates and runs code to solve the problem, pass@1 reaches 79.02; when it generates the image directly, visual evaluation (VLM-I) drops to 57.08. The human baseline sits at 83.06. Running code gets close to human. Generating pixels falls behind. The paper's conclusion is direct: models that reason and execute code achieve much higher geometric correctness, interpretability and quality than direct image generation.

MagicGeo (arXiv:2502.13855, 2025) is the closest example to what I ended up building: an LLM autoformalizes the statement into coordinate constraints, a formal solver verifies those constraints, and only then does the final diagram get generated. The number that helped me most from this paper is the ablation: adding the solver verification loop (the model gets feedback when the solver can't find a solution, up to 5 iterations) raises autoformalization accuracy from 94.7% to 98.7%. It's not the model generating better on the first try, it's the full system verifying before accepting.

AlphaGeometry (Trinh, Wu, Le, He, Luong, Nature, 2024) is the classic reference for the same pattern: a neural model proposes auxiliary constructions, a symbolic deduction engine proves them, and that's how it solved 25 of 30 olympiad geometry problems against 10 of 30 for the prior system. The model supplies semantics. The deterministic engine executes and verifies.

And this is already product behavior, not just paper behavior. When you ask ChatGPT for a data chart, it doesn't generate an image with DALL-E: it runs Python with matplotlib in a sandbox and returns the render (OpenAI, "Data analysis with ChatGPT", Help Center). Claude's Artifacts support SVG and diagrams like Mermaid as first-class types, rendered deterministically, not as pixel images (Anthropic, "What are artifacts and how do I use them?", Help Center). Two big labs, same pattern, for the same reason: when data accuracy matters, code and renderer beat pixels.

How do you build a pipeline for exact figures, from spec to verified render?

With that decided, the design landed here: the model never draws. The model fills a typed spec, and everything else is deterministic or verified.

The first step decides whether the item needs a figure, and of what kind, before it drafts the question:

const ItemPlanSchema = z.object({
  claim: z.string(),
  evidence: z.string(),
  task: z.string(),
  needsFigure: z.boolean().default(false),
  figureKind: z.enum(FIGURE_KINDS).optional(),
});

FIGURE_KINDS is 7 values: bar_chart, partitioned_shape, triangle, angle, number_pyramid, balanza, data_table. If the plan says a figure is needed, the next step emits the question and, along with it, the matching FigureSpec. Not an SVG, not a free-text description: typed fields. For a triangle:

export const TriangleSpec = z.object({
  kind: z.literal('triangle'),
  a: z.number().gt(0).lt(180),
  b: z.number().gt(0).lt(180),
  unknown: z.enum(['A', 'B', 'C']).optional(),
  unknownLabel: z.string().optional(),
}).refine((s) => 180 - s.a - s.b > 0, {
  message: 'los dos ángulos dados deben dejar un tercero positivo',
});

That .refine is the first verification layer, and it's free: a geometrically impossible triangle never even reaches the renderer, it gets rejected at parse time. The renderer solves the vertices with real trigonometry, law of sines included, and never lets the model compute an angle on its own.

Once the spec is valid, comes the numeric cross-check: compare every "hard" value in the figure against the question text, with thousands separators and Chilean decimals handled.

export function crossCheckFigure(stem: string, spec: FigureSpec): { ok: boolean; issues: string[] } {
  const uniqueValues = [...new Set(collectHardNumbers(spec))];
  const issues = uniqueValues
    .filter((value) => !appearsInStem(stem, value))
    .map((value) => `el valor ${numberVariants(value)[0]} de la figura no aparece en el enunciado`);

  return { ok: issues.length === 0, issues };
}

If the question says "a triangle with angles of 40° and 65°" and the spec carries a third angle of 75° that the model mentioned somewhere in the text but not in the question the student actually reads, crossCheckFigure returns { ok: false, issues: ["el valor 75 de la figura no aparece en el enunciado"] }. It's purely numeric, it doesn't understand labels or relationships, that's what the next layer is for.

The figure gets rasterized to PNG and sent, along with the question and the correct answer, to a multimodal judge that answers two separate questions: does the figure match the question exactly, and can the stated answer be derived by reading only the figure.

const { object } = await generateObject({
  schema: FigureJudgeSchema,
  messages: [{
    role: 'user',
    content: [
      { type: 'text', text: judgePrompt(stem, correctAnswer) },
      { type: 'image', image: png },
    ],
  }],
});

If either layer finds a problem, the item doesn't get discarded, it gets repaired:

if (plan.needsFigure) {
  for (let attempt = 0; ; attempt++) {
    const gate = await evaluateFigureGate(item, common, deps.judge);
    figure = gate.figure;

    if (gate.issues.length === 0 || attempt >= FIGURE_MAX_REPAIRS) break;

    item = await call('item', {
      ...common,
      schema: GeneratedItemSchema,
      prompt: itemPrompt(ctx, plan, nAlts, gate.issues.join(' ')),
    });
  }
}

The model retries the item step, this time with the actual issue text appended to the prompt, not a generic "try again". FIGURE_MAX_REPAIRS defaults to 2: up to 2 automatic repairs before giving up. And when it gives up, the item still gets saved, marked pending for human review, with confidence capped. Nothing gets silently discarded, and one bad figure never blows up the whole generation run.

Why doesn't structured outputs guarantee the figure is correct?

This is where it's worth being precise, because it's easy to conflate two different things. OpenAI's Structured Outputs (official docs, August 2024) raises compliance with a complex JSON Schema from under 40% to 100%, via constrained decoding: the model literally cannot emit a token that breaks the schema. That's huge, and it's the reason I ask the model for a typed FigureSpec instead of free text.

But 100% schema compliance is 100% valid shape, not 100% correct meaning. JSONSchemaBench (arXiv:2501.10868, 2025) measures this with 10,000 real schemas: on the "GitHub Hard" subset, nested and complex schemas, coverage collapses, Guidance drops to 41%, Outlines to 3%. And the underlying limit is even more direct: a JSON object that's perfectly valid against the schema can still carry a 75° angle that appears nowhere in the question, or a triangle whose angles add up to 210°. OpenAI itself states it as a warning: structured outputs doesn't make the meaning of the output deterministic, it doesn't remove the need for validation in the application.

That's why the FigureSpec alone isn't enough, not even with .refine. The schema and the .refine guarantee the figure is geometrically possible. The cross-check guarantees its numbers are in the question. The multimodal judge guarantees the figure and the question tell the same story, and that the answer can actually be derived from what the student sees. Three layers, increasing cost, each one verifying something the last one can't.

Where do image models still work today, and what are this approach's limits?

None of this means the image model is useless. For illustrating something without an exact number to verify, a scene, a character, a decorative background, the image model is still the right tool, and the gap is closing fast: in GenExam (arXiv:2509.14232, 2025), on exam-style figures, gpt-image-1 scores 13.1% on strict score, and most of the 17 models evaluated land under 15%, but nano banana pro reaches 70.2% strict, a huge jump over its predecessor on the same benchmark. The frontier moves fast, and the day an image model verifies its own geometry consistently, this whole pipeline stops making sense. Today isn't that day.

And my own approach has limits I'd rather state than hide. It's 7 kind values, not 70: any figure that doesn't fit triangle, angle, balance, bar chart, table, number pyramid or partitioned shape simply can't be requested, there's no free-form SVG generation. That's a design decision, full control over the render in exchange for coverage, and JSONSchemaBench already showed why: a schema that grows in complexity collapses its own coverage, so the limit is intentional. An item with a figure can cost up to 6 model calls in the worst case: plan, item, up to 2 repairs of the item-judge loop, and the final critique. And the numeric cross-check doesn't understand text labels or qualitative relationships, "angle A is acute", that rests entirely on the multimodal judge, which is probabilistic and can produce false negatives. When the gate runs out without passing, the figure doesn't publish on its own: it stays pending, visible in the bank, for someone to review before it reaches a student.

So today the model never draws me a figure without something deterministic checking it first. For what has to be exact, that's the only way I trust the output.