The Punctuation Mark My Font Couldn't Draw

PuntuQuest is a punctuation riddle game I built. You see three marks in a row and pick the fourth. The first version had 26 riddles, all using marks you type every day: commas, semicolons, question marks. The system font on iOS has drawn those for decades.

Then I added 20 new riddles with marks I had never actually typed before. Card suits: ♠ ♥ ♦ ♣. Greek letters: α β γ δ. The traditional footnote order: *, then a dagger, then a double dagger, then a section sign. Mathematical symbols like plus-minus and its less famous sibling, minus-or-plus.

I ran the full test suite. 390 unit tests, all green. Six integration flows, all green. A rendering test that draws every riddle on the smallest iPhone screen, both themes, and checks that nothing overflows. Also green.

I opened the app on a simulator and played through to the Symbolic category.

Some of the marks were not there. Not wrong, not misaligned. Just gone. A blank space where a dagger should have been, a box where a Greek letter should have sat. Every test had passed, and the app was about to ship with invisible riddles.

The test that passed when it should have failed

The rendering test (riddle_rendering_test.dart) pumps every riddle into the real RiddleCard widget at 320 points wide, which is the iPhone SE. For each mark in the sequence, it calls find.text(mark) and expects to find it on screen. This catches clipping (a multi-character mark like et al. that overflows its pill) and it catches missing widgets (a mark the layout dropped entirely). Version 1.0.1 shipped because this test caught a clipping bug with multi-character tokens.

What it does not catch is a mark that the font cannot draw.

When a font lacks a glyph for a character, it does not throw an error. It draws a replacement: a blank rectangle, or sometimes nothing visible at all. The widget is still in the tree. find.text('†') still matches. The layout does not overflow. From the test framework's point of view, the dagger is there and everything is fine.

The test proved layout. It never claimed to prove typography.

What a widget test actually renders

Flutter's widget tests do not use the real system font. They use a test font called Ahem, which draws every character as a square block. This is on purpose: it makes layout assertions predictable. But it also means that in a widget test, every character looks the same. A dagger, a card suit, the letter A. All squares.

So even if I had written a test that tried to check pixel output, it would have passed. The test font draws every character, by design.

The integration test (content_shots_test.dart) runs on a real device, which uses the real font. But find.text still only checks the widget tree, not the pixels. The framework sees the Text widget, sees that it holds the string '†', and reports it as present. Whether the font drew that string as a dagger or as a blank box is not something the framework knows or cares about.

Making the invisible visible

The fix was not to make the test smarter. It was to accept that this particular check needs a pair of eyes.

I wrote an integration test that draws only the suspect marks, grouped by riddle. It runs on a real iOS device with flutter drive, takes a screenshot of each group, and saves them to build/screenshots/. The test file says it in the doc comment:

/// Run it for the pictures, not the pass.

The suspect list is short and specific:

const suspect = <String, List<String>>{
  'symbolic_1-footnotes': ['*', '†', '‡', '§', '¶', '¤'],
  'symbolic_3-legal': ['©', '®', '™', '℠'],
  'symbolic_5-primes': ['°', '′', '″', '‴'],
  'symbolic_7-greek': ['α', 'β', 'γ', 'δ', 'ε', 'Δ'],
  'symbolic_8-suits': ['♠', '♥', '♦', '♣', '★', '♡'],
  'symbolic_9-signs': ['+', '−', '±', '∓', '×', '÷', '≠'],
  'symbolic_4-fractions': ['¼', '½', '¾', '⅛', '⅔', '⅝'],
  'grammar_2-quotes': ['´', '`', '"', "'"],
  'grammar_3-dashes': ['-', '–', '—', '―'],
};

The test draws each group using the app's own RiddleCard and AnswerOptionsGrid widgets, at the same size a player would see. If anything shows a box instead of a mark, the screenshot makes it obvious. The testing checklist in the repo says it plainly: "Anything showing a box needs the riddle changed, since the font is Apple's and not ours to fix."

That is the constraint. The font belongs to Apple. I cannot add a glyph to San Francisco. If a mark does not draw, the riddle is the thing that has to change.

The checklist line that matters

The testing checklist for that sprint has a section called "On your device: only what no test can reach." It lists exactly three things, and the first one is the glyphs:

The new symbols actually draw, rather than showing a box: card suits, Greek letters, and the footnote marks all render on an iOS simulator. Worth one glance on a real phone, since the simulator and the device share a font but not a version.

This is the kind of check that feels like it should be automated. But the whole point is that it cannot be. The test framework does not see what the font draws. A pixel comparison test would break on every OS update when Apple tweaks its font rendering. The only reliable check is: open the app, look at the screen, and see whether the mark is a mark or a box.

The easy failures and the silent one

When I added the Sprint 8 content, two integration flows also broke. They had hardcoded the number of achievements and the number of stars on the Journey header. When the riddle count went from 26 to 46, those numbers changed, and the tests failed. I fixed both in a few minutes: make the test read the count from the code instead of asserting a number the content owns.

Those were the easy failures. They were loud, they pointed at the problem, they had obvious fixes.

The hard failure was the silent one. The mark that drew as nothing, passed every assertion, and would have shipped to players as a blank space if I had not opened the app and looked.

The content shots test does not assert anything. It takes pictures. It exists so that the next time I add a riddle with a character outside the font's range, there is a step in the checklist that says: run this, look at the screenshots, and decide whether the mark drew.

What I was working with

• Flutter 3.47.5 / Dart 3.13.4
• PuntuQuest 1.7.0 (the build that shipped the Sprint 8 content drop)
• iOS simulator and device, San Francisco system font
• flutter_test for widget tests, integration_test for device tests