SWE-Bench tainted by answer leakage; real pass rates significantly lower (arxiv.org)
127 points by joshwa 2 hours ago | 26 comments
1127 points by joshwa 2 hours ago | 26 comments
1559 points by helsinkiandrew 5 hours ago | 481 comments
280 points by philip1209 2 hours ago | 35 comments
3199 points by riverdroid 3 hours ago | 108 comments
453 points by therealoliver 3 hours ago | 3 comments
5184 points by lis 3 days ago | 82 comments
66 points by rglover 38 minutes ago | 1 comment
721 points by bookofjoe an hour ago | 9 comments
855 points by jxmorris12 an hour ago | 32 comments
999 points by NoRagrets a day ago | 75 comments
103 hours ago
1131 points by doppp 2 hours ago | 7 comments
1227 points by coryodaniel 3 hours ago | 30 comments
13244 points by KoftaBob 10 hours ago | 200 comments
1427 points by rdabra 3 hours ago | 4 comments
15339 points by 9woc 11 hours ago | 53 comments
16275 points by todsacerdoti 12 hours ago | 315 comments
179 points by meetpateltech 2 hours ago | 0 comments
18183 points by debone 5 hours ago | 132 comments
1936 points by T-A an hour ago | 8 comments
2035 points by consumer451 an hour ago | 4 comments
2158 points by tommoor 2 hours ago | 13 comments
22757 points by akanet a day ago | 179 comments
2319 points by lermontov 4 days ago | 4 comments
244 points by weesperdev 3 days ago | 1 comment
2518 points by xnx an hour ago | 3 comments
26535 points by isaacfrond 10 hours ago | 356 comments
27143 points by gnabgib 4 days ago | 68 comments
28101 points by edu4rdshl 11 hours ago | 63 comments
29416 points by sshh12 21 hours ago | 85 comments
30