AI Strikes Back
AI Strikes Back: How H.264 Motion Vectors Revealed Ghost Font
See how 159 frames of visual static hid HELLO HUMAN, why frame differencing failed, and how H.264 motion vectors revealed the Ghost Font.
Six seconds of static, one message in motion
Pause the Ghost Font video at almost any point and you get the same answer: black-and-white static. There is no stable outline to trace, no visible caption to enlarge, and no clean set of strokes for ordinary OCR to recognize. Yet the clip is not empty. Its temporal structure carries a two-line message:
HELLO
HUMAN
The useful question is not whether a still contains hidden letters, but which relationships survive between frames. The pixels refused to give up the answer. The H.264 encoder's prediction choices did not. This does not make Ghost Font unbreakable. It shows how a signal can disappear in the wrong video representation.
Evidence from the analyzed Ghost Font clip
The measurements below describe this specific source file, not every possible Ghost Font animation.
| Property | Observed value |
|---|---|
| Video codec | H.264/AVC |
| Resolution | 1280 x 720 |
| Container duration | About 6.44 seconds |
| Decoded timeline | About 6.21 seconds |
| Decoded frames | 159 |
| Effective frame rate | About 24.7 FPS |
| Recovered text | HELLO HUMAN |
All 159 frames decoded successfully. Regular and high-change samples remained unreadable. The evidence appeared only when the analysis moved from pixel intensity to block motion.
Three readings of the same video
The raw frame preserves appearance but discards history. The difference view adds change but reduces it to strength. Neither retains the direction and correspondence needed to group letter-shaped regions. The vector view instead records how the codec predicted blocks from reference frames. After common movement was reduced, those decisions formed recognizable typography.
Ghost Font is therefore better described as motion-defined text than as a downloadable typeface. Research on motion-defined letter recognition also separates noticing a shape from identifying its letter. Motion helps construct the perceived boundary rather than decorating a finished glyph.
The frames refused to confess
A contact sheet supplied the first reality check. Samples from the beginning, middle, end, and high-change moments all looked like static. Enlarging them did not expose faint strokes or a lucky timestamp where HELLO HUMAN appeared.
This rules out text that is merely faint, brief, or hidden in one selected frame. It also explains why a screenshot test is misleading: the system is denied the dimension carrying the signal.
Why frame differencing stalled
The next test measured absolute pixel change between consecutive grayscale frames. Full masks and short-window motion-energy images remained dominated by the carrier because nearly the whole screen changes. Averaging softened random variation but still discarded origin, destination, and direction. Differencing confirmed widespread activity; Ghost Font required correspondence, not just change.
The codec kept a second description
H.264 compresses many frames by predicting blocks from decoded reference pictures. The encoder records a displacement to a useful matching region, then encodes the remaining error. That displacement is a motion vector.
FFmpeg can expose this decoder side data for supported codecs through its
documented export_mvs flag. In this
clip, the first frame was intra-coded and had no inter-frame vectors.
Motion-vector side data was available on 157 predicted frames. One early
example contained about 7,650 vectors across the textured screen.
These vectors are neither a secret text layer nor perfect physical motion. They are compression choices. Yet when letter-shaped regions behave differently from the background, the prediction field can retain their geometry while displayed frames look random.
How HELLO HUMAN surfaced
The extraction pipeline turned thousands of block decisions into one readable field:
- Export the horizontal and vertical vector components for each predicted frame.
- Paint each vector across its destination block and average overlapping contributions.
- Subtract the median horizontal and vertical motion so movement shared by most of the frame becomes less prominent.
- Map residual direction to hue and residual magnitude to brightness.
- Normalize brightness against the 98th percentile, preventing a few extreme vectors from dimming the rest of the field.
The result repeatedly formed the same two-line phrase.
A single suggestive block pattern could be a compression accident. This result is stronger because all ten letters have coherent geometry, the two-line layout recurs across time, and the same transformation was applied without tracing individual characters. Repetition across separate predicted frames is the persuasive evidence.
Motion vectors are not optical flow
Codec vectors and optical flow both expose temporal structure, but are not interchangeable. The OpenCV optical-flow guide describes optical flow as an estimate of apparent image motion between frames. H.264 vectors instead reflect rate-distortion choices, reference pictures, block partitions, search strategy, and encoder settings.
Re-encoding can therefore change the vectors even when playback looks similar. A different encoder or bitrate may preserve, weaken, or reorganize the pattern. This visualization reveals this bitstream, not every Ghost Font video.
AI strikes back by keeping time
Ghost Font can defeat screenshot OCR without defeating machine vision. A system can compare frames, estimate flow, inspect codec data, aggregate evidence, and send reconstructed shapes to OCR. It starts by preserving what a static pipeline discards.
Video models also vary in how much temporal evidence they actually receive. Google's Gemini video documentation states that visual input is sampled at one frame per second by default, warns that fast action may lose detail, and permits a custom rate. This is one product's behavior, not a verdict on all AI, but it shows why sampling matters.
Research on OCR abilities in video language models separates recognition, motion detection, and temporal localization. A clear title card and motion-defined letters pose different tasks. Ghost Font exposes that gap; it does not prove machines can never read the message.
A clever CAPTCHA signal, not a security boundary
Motion-defined text could frustrate bots that rely on screenshots, making it an experimental challenge signal rather than a complete security decision.
The OWASP bot-management guidance recommends layered defenses and treats visible CAPTCHAs as a step-up measure. Validation, expiration, replay resistance, rate limits, and monitoring still matter. Attackers can build a temporal solver once the trick is known.
Accessibility sets another firm limit. The W3C CAPTCHA accessibility guidance documents how challenges can exclude real users. A motion test needs an equivalent alternative independent of vision and motion perception.
What the Ghost Font case really proves
Video may carry structure in direction, frequency, phase, prediction modes, or residuals that vanishes when frames are isolated. Here, static inspection failed, differencing measured too little, and the H.264 motion field supplied correspondence. The pixels said static. The codec spelled HELLO HUMAN.
Ghost Font is a perception experiment and an AI benchmark, not encryption. Its power lies in showing how confidently any observer can look in the wrong place.
FAQ
What is Ghost Font?
In this experiment, Ghost Font is typography defined by coordinated motion inside a noisy field. The letter shapes exist in temporal relationships rather than as stable outlines in an individual frame.
Can AI read Ghost Font?
Potentially, yes. Results depend on sampling rate, preprocessing, compression, tools, and the model's ability to reason across time. A static OCR system may fail while a purpose-built temporal pipeline succeeds.
Are H.264 motion vectors the same as optical flow?
No. Codec motion vectors are encoder prediction decisions. Optical flow is an image-analysis estimate of apparent motion. Both can reveal structure, but their meanings and failure modes differ.
Would the hidden message survive re-encoding?
The visible motion effect may survive, but the exact codec vectors can change. Recoverability depends on the new encoder, bitrate, reference choices, and how strongly the temporal pattern remains in the pixels.
Should Ghost Font be used as a CAPTCHA?
Only as an experimental signal within layered protection, never as the sole security control. Any real challenge also needs an accessible alternative.