Also, it's insane to me that this whole article has been written without specifying anywhere the models being used.

Like, of course you're not gonna be productive if all you have is Sonnet 4.6???

4/2024: Baseline metrics

8/2024: Participants given Github Copilot licenses

11/2024: Conducted surveys, 261 people invited and 97 responded in survey

9/2026: Study published

> To establish a before-Copilot baseline, we used data from April, May, and June 2024. After-Copilot data was represented by the period of September, October, and November 2024

> GitHub Copilot usage varied significantly among engineers, the tool demonstrably fostered positive changes in perceived engineer value, reduced time spent on various engineering activities, and boosted motivation and perceived skills

> subsequent monitoring of PRs and LOC for participants from December 2024 to May 2025 showed no statistical improvements

> the implications of more advanced capabilities, such as retrieval-augmented generation (RAG) over enterprise codebases or deeper engineering workflow integrations, warrant separate investigation

I think it is just out of date, habits have changed as well. I have not seen much gain personally at that period except in the last 12 months. Also, models not named, token counts not shown. Not to mention it was the older autocomplete + chat that were in use, these days it is much more advanced with RAG, cli use, MCPs, etc.