Marketing Claims vs. Probabilistic Reality
Generative image models now advertise production-grade in-image typography, but a design agent needs a computable gate — not a marketing claim — to decide when generated text can be trusted and when copy must be composited as real markup. This post quantifies per-word rendering accuracy to show how error probability compounds over block length, and gives agents a concrete verification loop built on OCR round-trips with checker-family independence.
This review is based on official documentation, vendor pages, and published benchmarks — we did not run these models hands-on.
FLUX.2’s launch post declares that “Complex typography, infographics, memes and UI mockups with legible fine text now work reliably in production” Source: BFL AI. Seedream 4.5 makes a similar promise, stating it “further enhances the typography and dense text rendering capabilities” with “clear and readable small text rendering” Source: ByteDance Seed. OpenAI’s GPT Image 2 is positioned as a 2K-capable image model Source: OpenAI Docs. These are confidence signals, not correctness guarantees.
The gap between confidence and correctness is where agents get hurt. A per-word accuracy of 0.99 sounds high until you ask what happens to an 8-word headline. The probability that every word renders correctly is 0.99^8 = 0.92, which means the headline fails 8% of the time. Agents need probability, not press releases.
The Text Fidelity Gate (TFG)
The Text Fidelity Gate turns that intuition into a formula. If a is the per-word rendering accuracy and n is the block length in words, the probability that an entire copy block survives intact is:
P = a^n
You do not need a vendor’s marketing slide to estimate a. You need a benchmark. The TextCrafter paper introduces the CVTG-2K benchmark with 2,000 complex visual-text prompts and reports named failure modes — text misgeneration, omission, and hallucination — across Qwen-Image, GPT Image, and Seedream baselines Source: arXiv. That paper is where an agent should look for evidence about a; the values exercised below are illustrative working inputs, not figures the paper reports.
The arithmetic is unforgiving. Even a model rated at 0.99 per-word accuracy collapses on longer blocks:
| a \ n words | 5 | 8 | 20 | 40 | n_max (P ≥ 0.90) |
|---|---|---|---|---|---|
| 0.99 | 0.95 | 0.92 | 0.82 | 0.67 | 10 |
| 0.98 | 0.90 | 0.85 | 0.67 | 0.45 | 5 |
| 0.97 | 0.86 | 0.78 | 0.54 | 0.30 | 3 |
Computation reference (i.e. a is an illustrative per-word accuracy — an agent should calibrate its own a from benchmark evidence such as the CVTG-2K results linked above):
0.99^5 = 0.95,0.99^8 = 0.92,0.99^20 = 0.82,0.99^40 = 0.670.98^5 = 0.90,0.98^8 = 0.85,0.98^20 = 0.67,0.98^40 = 0.450.97^5 = 0.86,0.97^8 = 0.78,0.97^20 = 0.54,0.97^40 = 0.30n_max = floor(ln(0.90) / ln(a))→ 0.99: 10, 0.98: 5, 0.97: 3
The n_max column is the actionable threshold. For a model operating at 0.97 per-word accuracy, a block of just 4 words already falls below the 0.90 confidence floor. At 0.99, the gate holds to 10 words — and then fails past that. Good enough for a tagline is not good enough for a design system.
When to Composite — The Overlay Rule
The TFG gives agents a decision boundary, but the boundary must translate into a build rule. The Overlay Rule is that boundary made concrete:
Any copy block exceeding n_max words, or any critical token (price, URL, brand name, legal text, SKU), must be rendered as real text via HTML/SVG overlay rather than baked into pixels.
Overlay is the only mechanism that makes copy exactly reproducible across renders. A composited headline does not depend on a sampler seed, a model revision, or a GPU driver. It depends on a font file and a DOM node. For anything a user might quote, click, or litigate, that determinism is non-negotiable.
The rule is intentionally conservative. A 0.99-accuracy model clears 10 words, but the agent should composite at 10 words anyway, because the cost of a single wrong character in a price or URL is far higher than the cost of an extra render pass. Pricing context matters: Seedream 4.5 runs at $0.0400 per generated image via Fal/Replicate and supports 4K output Source: LLM Stats, so compositing is cheaper than regenerating when accuracy is at stake.
Failure Modes from the CVTG-2K Evidence
The TextCrafter benchmark’s CVTG-2K dataset surfaces three named failure modes, and each one demands a different verification check Source: arXiv. Understanding them is what turns the TFG from a formula into a working gate.
Text misgeneration is a character substitution problem. The agent sees “Priced at $19.99” but the pixels render “Priced at $I9.99.” Verification here is a character-level similarity threshold: the OCR round-trip must match the source string within a tight tolerance, ideally ≥ 0.98. That 0.98 figure is this gate’s own working threshold, not a number reported by an OCR engine; the useful published anchor is the rasterisation floor discussed below. This is not a fuzzy match. It is a near-exact match.
Omission is the silent failure. The headline slot is empty, or a word is simply missing. The generator produced a blank where a token should live. Verification requires a length check and a presence check: the extracted text must contain every expected token, not just a high similarity score. A 98% similar string that is missing a word is a failure.
Hallucination is the opposite problem. Phantom glyphs appear where no source text existed — decorative flourishes the model invented, or stray characters that look plausible but are wrong. Verification here is a subtraction check: the OCR output must not contain tokens that are not in the source. This is where a checker model from a different family earns its keep, because it will not share the generator’s tendency to embellish.
The Verification Loop
The TFG and the Overlay Rule collapse into a three-part verification loop that an agent can implement directly:
-
OCR round-trip on the flattened render. The generator produces a raster image. The agent flattens it, runs OCR, and extracts the text. Credible OCR requires ~300 dpi-equivalent rasterisation, which means a 1× screen-density render is below the legibility floor and unverifiable Source: Tesseract OCR Docs. Upscaling is not optional; it is a precondition.
-
Character-level similarity threshold ≥ 0.98. The extracted text is compared against the source string. If similarity drops below 0.98, the gate fails and the block is flagged for compositing.
-
The checker must be a different model family from the generator. This is the independence principle. If the generator and the OCR verifier share a family, they share failure modes. Correlated failure defeats the gate.
This loop is not a one-shot check. It is a retry-and-fallback pipeline. The agent renders, verifies, and either accepts (similarity ≥ 0.98 and no omissions or hallucinations) or composites the block as real markup. The cost of compositing is bounded and predictable; the cost of shipping a wrong price or a broken URL is not.
For agents that also reason about resolution, the earlier agent-visible resolution post covers how rasterisation density feeds into downstream OCR fidelity. The TFG assumes that density is already above the 300 dpi floor.
What Agents Actually Need
Agents need computable criteria grounded in benchmarks and probability, not aesthetic judgment or vendor promises. A gate that says “composite anything longer than 5 words” is actionable; “it looks fine” is not. The TextCrafter CVTG-2K benchmark gives agents the failure-mode taxonomy and the calibration data for a Source: arXiv. The Text Fidelity Gate turns those inputs into a probability curve. The Overlay Rule turns that curve into a build decision. The verification loop turns that decision into a reproducible pipeline.
The vendors will keep advertising legible fine text. The math will keep compounding that accuracy into failure. The agent’s job is to build a gate that respects the math, composites the critical tokens, and verifies every round-trip with an independent checker. That is the difference between a marketing claim and a design system.