Microsoft announced code-testing-generator, an open-source, multilingual agent for generating unit tests. The agent does not merely create tests that appear correct; it first learns from the project repository, then identifies the code that needs testing, the testing framework in use, where to place the files, and the build and run commands, before verifying that the tests are discovered within the repository’s usual test path.
The agent is available through the dotnet-test extension in the dotnet/skills repository. It isolates the code under test while creating mocks for external services and other dependencies. Its current scope does not include integration tests, end-to-end tests, browser tests, or performance tests.
It starts by understanding the repository before writing tests
Rather than immediately beginning to generate code, the agent searches for the part that needs testing, identifies the language and testing framework, and reviews existing tests to determine where new tests should be placed and how they should be written. It also identifies the commands used to build and run the tests.
This approach helps avoid a situation in which a new test project builds and runs successfully on its own but is not included in the solution or the test command used by the continuous integration environment. Therefore, the agent examines how the repository discovers tests and ensures that the new tests appear within that process.
The agent selects its level of operation according to the size of the request:
- Direct: Read the relevant code, write the tests, and then verify the result.
- Single pass: Search and plan once, then execute the plan.
- Iterative: Repeat the search, planning, and execution cycle to cover a large request or reach a specific coverage target.
It verifies that the tests are useful, not merely successful
For larger tasks, the agent lists the code that needs tests and starts with the simpler parts before moving on to code with more dependencies. It also maps each behavior to a test file, follows the project’s local conventions, and runs the tests while working.
If the resulting code does not build, the agent fixes it, and if an assertion is incorrect, it returns to the source and corrects the test. It does not change production code while generating tests, and it avoids unit tests that call external URLs, open ports, or depend on precise timing.
Before finishing, the agent performs a set of checks, including small changes that are expected to make the tests fail, as a simplified form of mutation testing; searches for weak or missing assertions; and verifies that there is a test for every required scenario. It also builds the entire workspace and runs the complete test suite, verifying that the repository’s test command can find the new tests.
Internal test results
In the latest internal benchmark, the agent completed 140 of 152 tasks, a rate of 92.1%, compared with 120 tasks and a rate of 78.9% for standard Copilot, using the same model. Microsoft said that the number of failures fell by 63%.
The largest differences appeared in ambiguous requests that left most decisions to the agent; the specialized agent succeeded in 79 of 89 tasks, compared with 59 tasks for standard Copilot, while failures fell from 30 to 10. For detailed requests, the results were similar, with the agent succeeding in 61 of 63 tasks.
In 15 tasks that requested tests for a specific code change, the agent succeeded in every task, while standard Copilot did not succeed in any of them. Across the combined results, both systems succeeded in 119 tasks, while the specialized agent alone succeeded in 21 tasks, standard Copilot alone succeeded in one task, and both systems failed in 11 tasks.
The agent generated 6,963 tests, compared with 7,129 tests for standard Copilot, with average final line coverage of 72.4% versus 72.2%, and average branch coverage of 49.8% versus 49.1%. The average task time was also 359 seconds for the specialized agent and 380 seconds for standard Copilot. Microsoft says the gain came from reliability, not from producing a larger number of tests.
Multilingual support and extension availability
The accompanying instructions include support for .NET, Python, TypeScript, JavaScript, Java, Go, Ruby, Rust, Swift, Kotlin, PowerShell, and C++. The tests showed that the agent learns the conventions of each repository instead of applying C# patterns to all projects.
In another test involving 44 tasks from the SWE Atlas benchmark, the agent completed 16 tasks, a rate of 36.4%, compared with 12 tasks and a rate of 27.3% for standard Copilot. The generated tests also passed 550 cases and detected 360 injected bugs, compared with 493 and 316, respectively, for the standard system. Microsoft acknowledges that this benchmark is more difficult and that its completion rates are much lower.
The extension can be used in GitHub Copilot CLI and is also available in Visual Studio Code and VS Code Insiders through extension support, which is still in preview. Microsoft is also working on Visual Studio support. To install the extension in GitHub Copilot CLI, the user adds the extension marketplace with the command /plugin marketplace add dotnet/skills, then installs it using /plugin install dotnet-test@dotnet-agent-skills, restarts the CLI, and selects the code-testing-generator agent.