SWE-MERA: A Dynamic Benchmark for Agenticly Evaluating Large Language Models on Software Engineering Tasks

The authors introduce SWE-MERA, a dynamic benchmark designed to mitigate data contamination and improve the evaluation of LLMs on software engineering tasks.

Firefly links to the original publisher. The summary above is AI-generated for orientation and may differ from the source. The “current rank score” decays over time so newer significant stories surface first.