I was reading #950 and went one level down into CountFilteredTests, since it is the second of the two fixes suggested there. It has a separate problem.
CountFilteredTests (Tests.Run.cs:317-377) sets matches = true if any one of assembly / namespace / class / method matches — an OR across the four parameters. CreateTestFilter maps those four onto three Unity fields (:213, :231, :234): assemblyNames, groupNames (namespace and class), testNames (method). RuntimeTestRunnerFilter.BuildNUnitFilter adds one filter per populated field and ANDs them, ORing values within a field.
The regex patterns themselves do mirror Unity's groupNames semantics, as Unity-MCP-Plugin/.claude/skills/tests-run/SKILL.md describes. It is the combining step that differs.
Measured on a copy of your Unity-Tests/6000.3.1f1 (as Unity-Tests/repro), editor 6000.5.6f1, EditMode — CountFilteredTests against the tests Unity actually executed, same project, same Filter object, one editor invocation per row:
| filter |
fields |
plugin_predicted |
actually ran |
| assembly / namespace / class / method, each alone |
1 |
6 / 1127 / 2 / 1 |
same |
| namespace + class |
1 (both groupNames) |
1127 |
1127 |
| assembly + namespace |
2 |
1127 |
6 |
| assembly + class |
2 |
6 |
2 |
| assembly + method |
2 |
6 |
1 |
| namespace + method |
2 |
1127 |
1 |
| class + method |
2 |
2 |
1 |
Touching two fields is necessary but not sufficient — where the two selections happen to pick the same tests, say a class holding exactly the one method named, both sides still agree.
Where they differ, the plugin's number is always the larger one, and the only consumer is if (testCount == 0) return Error.NoTestsFound(...) in ValidateTestFilters — so that guard cannot fire when it should. testAssembly: "EditorTests" with a class that exists nowhere counts 6 and runs 0, and the caller gets a summary of a run that executed nothing.
That was measured on 1.7.0 rather than the versions your CI resolves, but the AND is in every version I looked at (1.1.33 through 1.7.0), and the OR side is your own code, so I do not think the version matters here.
Happy to attach the harness — two small files, no state carried between invocations.
I was reading #950 and went one level down into
CountFilteredTests, since it is the second of the two fixes suggested there. It has a separate problem.CountFilteredTests(Tests.Run.cs:317-377) setsmatches = trueif any one of assembly / namespace / class / method matches — an OR across the four parameters.CreateTestFiltermaps those four onto three Unity fields (:213,:231,:234):assemblyNames,groupNames(namespace and class),testNames(method).RuntimeTestRunnerFilter.BuildNUnitFilteradds one filter per populated field and ANDs them, ORing values within a field.The regex patterns themselves do mirror Unity's
groupNamessemantics, asUnity-MCP-Plugin/.claude/skills/tests-run/SKILL.mddescribes. It is the combining step that differs.Measured on a copy of your
Unity-Tests/6000.3.1f1(asUnity-Tests/repro), editor 6000.5.6f1, EditMode —CountFilteredTestsagainst the tests Unity actually executed, same project, sameFilterobject, one editor invocation per row:groupNames)Touching two fields is necessary but not sufficient — where the two selections happen to pick the same tests, say a class holding exactly the one method named, both sides still agree.
Where they differ, the plugin's number is always the larger one, and the only consumer is
if (testCount == 0) return Error.NoTestsFound(...)inValidateTestFilters— so that guard cannot fire when it should.testAssembly: "EditorTests"with a class that exists nowhere counts 6 and runs 0, and the caller gets a summary of a run that executed nothing.That was measured on 1.7.0 rather than the versions your CI resolves, but the AND is in every version I looked at (1.1.33 through 1.7.0), and the OR side is your own code, so I do not think the version matters here.
Happy to attach the harness — two small files, no state carried between invocations.