GitHub and Microsoft have released the results of a joint practical experiment named Codename MDASH, aimed at addressing the crisis of security vulnerability reports generated by artificial intelligence tools that overwhelm open-source software developers and maintainers. Instead of flooding developers with raw scanning results that often contain duplicates or inaccurate alerts, the experiment focused on delivering pre-verified, ready-to-use code fixes.
The experiment relied on a synergistic scanning system powered by Microsoft AI Security models (MAI Cyber), alongside filtering and verification mechanisms backed by GitHub Security Labs and Microsoft's FORGE Lab. The initiative covered six widely used open-source projects, including Caddy, AutoGPT, ImageMagick, Netty, Express, and FastMCP, where thousands of issues were filtered both manually and programmatically to reach developers as 99 private reports accompanied by a proposed fix.
How the Codename MDASH System Worked
- Scanning: Scanning projects using the AI security model and proposing initial fixes for discovered vulnerabilities.
- Filtering: Using an intelligent scanning agent built on Copilot skills to remove duplicates and focus on high-impact vulnerabilities.
- Verification: Rigorous manual review by security researchers to ensure result accuracy before sending them to developers.
- Reporting and Fixing: Privately sending reports via GitHub's vulnerability reporting feature while providing the proposed fix code for maintainers to review and merge.
What Changes Practically for Developers?
Results through mid-November showed that developers of the six projects managed to fix more than half of the submitted reports (55 fixes), with 14 of them merged into actual software releases within a few days. Maintainers affirmed that receiving a report backed by a proposed and pre-reviewed fix saves long hours of effort that would otherwise be wasted on reading and dissecting unfiltered raw scan results.
Next Steps and Scaling Up
Microsoft and GitHub plan to build upon these lessons by integrating similar review techniques into private vulnerability reports and security advisories, alongside announcing the second version of the MAI Cyber model. Plans also include expanding support through the GitHub Secure Open Source Fund to provide eligible projects with AI-powered security scans and prior human review.