A simple data-split change reveals how easily drug-discovery benchmarks can flatter the models they're meant to evaluate.
TL;DR: VIDRAFT, a Korean AI startup, has publicly released LEADBOARD — a 21-board drug-property prediction leaderboard built on FINAL-Bench. Testing the same baseline model on the same hERG dataset, FINAL-Bench recorded a 0.211 AUROC difference depending solely on whether the data was divided by time or by random assignment. The leaderboard spans seven disciplines and withholds labels for 18,382 held-out compounds to prevent score inflation.
VIDRAFT, a Korean AI startup led by CEO Min-sik Kim, launched LEADBOARD on August 22, 2026 — a structured public leaderboard designed to make drug-property prediction benchmarks harder to game and easier to trust. The announcement was published as a community article through Hugging Face, which also hosts the leaderboard itself.
FINAL-Bench is VIDRAFT's benchmarking project for drug-property prediction AI. Its new LEADBOARD consists of 21 boards spanning seven scientific disciplines, built on a training set of 212,670 compounds. Crucially, the labels for the 18,382 held-out test compounds are withheld from participants — a design choice intended to prevent models from being quietly tuned to match known answers.
The headline finding that motivated much of FINAL-Bench's design came from an experiment on hERG, a potassium ion channel routinely screened early in drug development because blocking it can trigger life-threatening cardiac arrhythmias. FINAL-Bench pulled 12,021 hERG records with document-year metadata from ChEMBL 37, retained 11,972 after unit-checking and parsing, and then reduced those to 9,788 unique compounds.
With that dataset in hand, the team trained a standard LightGBM model using Morgan fingerprints. In the time-split condition — where the model trained on compounds reported before 2022 and was tested on those reported afterward — it scored 0.606 AUROC. In the random-split condition, dividing the identical dataset without regard to when compounds were reported, the same model averaged 0.818 AUROC across five random seeds, with individual runs ranging from 0.803 to 0.830. The model architecture, fingerprints, and hyperparameters were unchanged throughout. Only the split logic differed.
That single methodological variable produced a gap of 0.211 AUROC — a difference large enough to shift a model from appearing mediocre to appearing strong, without the model itself improving at all.
FINAL-Bench's design also incorporates assay noise floors as an explicit consideration. When measurement noise in a laboratory assay is larger than the performance gap being reported, a benchmark win may be indistinguishable from random variation. Making noise floors visible is part of LEADBOARD's stated mission to standardize how the field compares drug-prediction tools.
Beyond the technical work, Kim has been transparent about his broader ambitions. At 28, he paused a computer-engineering degree at Korea Cyber University to found VIDRAFT as a scientific research company. According to VIDRAFT's website, he is a named inventor on 12 patent applications and a co-author of four papers. His long-term goal, as stated on the company's site, is to use commercial products and intellectual property to fund a Korean scientific institution modeled on the Max Planck Society.
Drug-discovery AI has produced a steady stream of reported benchmark improvements over the past several years. The problem is that those improvements are often difficult to compare: different teams use different datasets, different ways of dividing those datasets, and different laboratory measurements that carry their own inherent noise. A model that looks like a breakthrough under one set of conditions may look ordinary — or worse — under another.
VIDRAFT's FINAL-Bench is an attempt to standardize the measuring stick before declaring winners. The hERG split experiment makes the core problem concrete: a 0.211 AUROC swing from a bookkeeping decision, not a scientific one, is not an edge case. Medicinal chemists routinely synthesize large families of structurally related compounds by modifying a shared molecular scaffold. When those chemical cousins end up on both sides of a random train-test boundary, a model can look like it generalizes when it is largely interpolating within a chemical neighborhood it has already seen.
Hidden labels, time-aware splits, and explicit noise floors are the tools FINAL-Bench is deploying to close that gap between reported performance and real-world utility. For an industry where a confident-but-wrong prediction can waste years of laboratory work, that kind of benchmark discipline has practical consequences well beyond the leaderboard itself.
Q: What is VIDRAFT's FINAL-Bench LEADBOARD?
A: LEADBOARD is a public drug-property prediction leaderboard developed by VIDRAFT and hosted on Hugging Face. It comprises 21 boards across seven scientific disciplines and uses hidden test labels and time-aware data splits to produce more rigorous model evaluations.
Q: Why does the data-split method matter so much in drug-prediction benchmarks?
A: Because medicinal chemistry generates structurally similar compound families, a random split can place chemical relatives on both sides of a train-test boundary, allowing a model to appear to generalize when it is effectively interpolating. A time-based split better simulates how a model would perform on genuinely new chemistry.
Q: Who leads VIDRAFT and what are the company's stated goals?
A: VIDRAFT is led by CEO Min-sik Kim, who founded the company after pausing his studies at Korea Cyber University. According to VIDRAFT's website, his long-term goal is to use the company's commercial and IP revenues to establish a Korean scientific research institution modeled on Germany's Max Planck Society.
Source: RuntimeWire (2026-08-22) — original article