Leaklens

See the sentence that gave the fix away.

Search any SWE-bench Verified task by instance ID, repo or pull-request number. Every flag quotes the text that backs it and shows the fix commit’s date, so you can check it in under a minute.

Try

Flagged tasks to start with

Click a task to read the quoted evidence.

Browse all 500 Verified tasks Download the table (CSV)

Quoted, never paraphrased

Every highlighted passage is a word-for-word substring of the public issue or discussion. If a quote cannot be found in the text, it is dropped.

Checked twice

Each task is judged by two independently worded checks. Where they differ, the task is marked and kept out of the headline counts.

Dated against git history

The upstream fix commit and its date sit beside each verdict, so exposure is a date you can compare with any model’s cutoff.