How a Korean AI startup's disciplined optimization strategy won where raw speed fell short
TL;DR: VIDRAFT, a Korean Pre-AGI AI startup, achieved a verified top score of 510.58 tokens per second (TPS) in The First Gemma Challenge, an inference speed competition using a single fixed GPU and a fixed model. A competing entry recorded a higher raw speed of 535.91 TPS but was disqualified for exceeding the contest's perplexity quality threshold. VIDRAFT's result passed independent re-verification on a hidden prompt set, earning it the recognized state-of-the-art position on the leaderboard.
VIDRAFT, the Korean Pre-AGI AI startup, claimed the verified top ranking in The First Gemma Challenge in August 2026, demonstrating that disciplined software optimization — not hardware brute force — can push inference performance to its practical ceiling. The result, analyzed in depth by Japanese tech writer Satoru Kono on note.com, offers a rare look at the engineering decisions behind a competition win that hinged as much on quality control as on raw throughput.
The First Gemma Challenge required all participants to work within identical hardware and model constraints: a single NVIDIA A10G GPU and Google's google/gemma-4-E4B-it model. No additional GPUs could be provisioned, and no lighter model substitutions were permitted. The only variable was software optimization.
The competition metric was TPS — tokens generated per second — but speed alone was not enough to win. Entries were also evaluated on perplexity (PPL), a quality measure where lower scores indicate more accurate predictions. Any result with a PPL exceeding approximately 2.42 was ruled invalid. Critically, organizers re-evaluated all submissions against a hidden set of prompts that participants had never seen, closing off any strategy that amounted to overfitting to self-prepared test data.
VIDRAFT's final submission logged 510.58 TPS with a PPL of 2.3930, under the configuration name vidraft-fw188-ctk49-n64-patchbridge-v1. That result passed the hidden-prompt re-verification. A separate entry had recorded 535.91 TPS — numerically faster — but its PPL reached approximately 2.44, breaching the quality threshold. That entry was therefore disqualified, leaving VIDRAFT's submission as the verified benchmark leader.
The gap between 535 TPS and 510 TPS illustrates a tension that practitioners encounter regularly: an optimization that looks dominant on internal benchmarks can fail in production-realistic conditions. The contest's hidden-prompt re-verification was specifically designed to surface this failure mode. As the note.com analysis observes, optimizing only against self-selected test data risks a kind of "teaching to the test" outcome — configurations that are locally fast but not robustly fast.
VIDRAFT's approach addressed this by deliberately disabling a self-benchmarking flag (PRECACHE_BENCH=0) that would have inflated internally measured TPS without reflecting real-world throughput. The team aligned their own measurement methodology to what the hidden evaluation would see, rather than maximizing the number on their own dashboard.
Three technical levers stand out in the published configuration. First, a sliding window setting of 188 tokens was applied to limit the KV cache memory bandwidth requirement during generation; the non-round number suggests it was arrived at empirically rather than set by default. Second, a centroid top-k value of 49 — found by stepping through nearby values such as 44, 48, and 49 — was selected as the point that maximized throughput while staying within the PPL budget. Third, a warmup sequence of 64 single-token prompts was run before the timed measurement window, moving CUDA graph compilation and JIT initialization costs outside the scored interval. The analysis notes that skipping this warmup would have cost approximately 15 TPS. Speculative decoding was also enabled, using 7 speculative tokens per step with the MTP method, allowing the system to generate more tokens per inference pass.
The broader lesson the note.com analysis draws is about the ordering of constraints: quality gates should be applied before sorting by speed, not after. A team that ranks candidates by throughput first and then checks quality tends to rationalize borderline results; applying the quality ceiling first enforces discipline from the start.
Q: What is The First Gemma Challenge and what did VIDRAFT achieve in it?
A: The First Gemma Challenge is an inference speed competition where all participants use the same single GPU and the same Gemma model. VIDRAFT achieved the top verified score of 510.58 TPS while keeping perplexity below the required quality threshold.
Q: Why was the 535 TPS entry ranked below VIDRAFT's 510 TPS result?
A: The 535.91 TPS entry's perplexity score reached approximately 2.44, exceeding the contest's quality limit of roughly 2.42. Because quality compliance was mandatory, that submission was disqualified despite its higher raw throughput.
Q: What software techniques drove VIDRAFT's top inference speed?
A: Key techniques included a narrowed sliding window to reduce KV cache memory pressure, an empirically tuned centroid top-k parameter, a 64-step single-token warmup phase to move initialization costs outside the measurement window, and speculative decoding with 7 tokens per step.
Source: note (일본) (2026-08-15) — original article