Correct the claim that all setup helpers route through setup()

This statement is inaccurate for at least setupForStationLogin, which performs its own context/page initialization and does not call setup() in playwright/test-manager.ts. Contributors following this guidance may assume SetupConfig options and shared setup behavior apply uniformly to every helper, leading to incorrectly structured tests or missing setup/cleanup steps when using station-login flows.

Useful? React with 👍 / 👎.

Point conference sets to existing suite files

SET_7/SET_8/SET_9 are mapped to conference-tests1.spec.ts, conference-tests2.spec.ts, and conference-tests3.spec.ts, but there are no matching files under playwright/suites (repo-wide filename search returns none), so these projects will run with zero matched tests and silently skip the intended conference coverage while still incurring OAuth/setup work for those sets.

Useful? React with 👍 / 👎.

M1: TestManager Class Documentation Missing

TestManager is 627 lines and is the single class every test file depends on. It gets one bullet here and one line in the data flow section. This is insufficient — an AI agent cannot write or modify tests without this.

Needs a dedicated ## TestManager section documenting:

1. Constructor:
new TestManager(projectName: string, maxRetries?: number) — projectName is the SET key (e.g., 'SET_1'), used to resolve set-scoped env vars like ${projectName}_AGENT1_ACCESS_TOKEN.

2. SetupConfig interface — all 10 options and defaults:

interface SetupConfig {
  needsAgent1?: boolean;        // default: true
  needsAgent2?: boolean;        // default: false
  needsCaller?: boolean;        // default: false
  needsExtension?: boolean;     // default: false
  needsChat?: boolean;          // default: false
  needsMultiSession?: boolean;  // default: false
  agent1LoginMode?: LoginMode;  // default: 'Desktop'
  enableConsoleLogging?: boolean;  // default: true
  enableAdvancedLogging?: boolean; // default: false
  needDialNumberLogin?: boolean;   // default: false
}

3. Seven page properties available after setup:
agent1Page, agent2Page, callerPage, agent1ExtensionPage, chatPage, multiSessionAgent1Page, dialNumberPage

4. Convenience methods (method → config preset):
basicSetup(), setupForAdvancedTaskControls(), setupForAdvancedCombinations(), setupForDialNumber(), setupForStationLogin(), setupForIncomingTaskDesktop(), setupForIncomingTaskExtension(), setupForIncomingTaskMultiSession()

5. Cleanup:

• softCleanup() — handles stray tasks only (between test files via afterAll)
• cleanup() — full logout + close all pages/contexts (end of entire suite)

6. 3-phase parallel setup:
Phase 1: Create all browser contexts in parallel → Phase 2: Login + widget init in parallel for independent pages → Phase 3: Console logging setup

M2: Utils Layer Documentation Missing

8 utility files with 50+ exported functions — none documented. This line says only playwright/Utils/*.ts.

Needs a ## Utils Reference section with at minimum:

Without this, an AI agent will either duplicate existing logic or fabricate non-existent functions.

M3: Constants Documentation Missing

constants.ts defines every enum, type, and timeout used across tests — currently undocumented.

Key enums:

Timeout hierarchy (critical for choosing correct timeout per operation):

Without this, an agent will use arbitrary timeout values or wrong enum names.

M4: Console Log Verification Pattern Undocumented

The framework's primary assertion mechanism for verifying SDK behavior is console log monitoring — used across nearly every test file. Not documented anywhere.

How it works:

1. Setup: setupConsoleLogging(page) or setupAdvancedConsoleLogging(page) registers a page.on('console', ...) handler that filters for specific SDK event strings.

2. Captured events (basic controls):

   • WXCC_SDK_TASK_HOLD_SUCCESS / WXCC_SDK_TASK_RESUME_SUCCESS
   • WXCC_SDK_TASK_PAUSE_RECORDING_SUCCESS / WXCC_SDK_TASK_RESUME_RECORDING_SUCCESS
   • onHoldResume invoked / onRecordingToggle invoked / onEnd invoked

3. Captured events (advanced controls):

   • WXCC_SDK_TASK_TRANSFER_SUCCESS / WXCC_SDK_TASK_CONSULT_START_SUCCESS / WXCC_SDK_TASK_CONSULT_END_SUCCESS
   • AgentConsultTransferred / onTransfer invoked / onConsult invoked

4. Captured events (state changes):

   • WXCC_SDK_AGENT_STATE_CHANGE_SUCCESS
   • onStateChange invoked with state name: <stateName> (matched via CONSOLE_PATTERNS.ON_STATE_CHANGE_REGEX)

5. Verification functions: verifyHoldLogs(), verifyRecordingLogs(), verifyEndLogs(), verifyTransferSuccessLogs(), verifyConsultStartSuccessLogs(), validateConsoleStateChange(), checkCallbackSequence()

6. Sequence validation: checkCallbackSequence() verifies that SDK API success event appears in logs before the onStateChange callback.

This section should be added under a ## Console Log Verification Pattern heading. Without it, an AI agent writing new tests would not know how to verify SDK behavior and would write incomplete assertions.

M5: Set → Suite → Test File Mapping Missing

The file topology lists suites and tests separately but doesn't show which test factories each suite imports. This is the critical wiring an agent needs to know where to add a new test.

Add a ## Set → Suite → Test Mapping section:

Without this, an agent asked to 'add a new incoming task test' won't know it should go in SET_1's suite.

M6: Sample App (SUT) Context Missing

The tests are testing CC widgets running inside samples-cc-react-app at localhost:3000. playwright.config.ts starts it via:

webServer: {
  command: 'yarn workspace samples-cc-react-app serve',
  url: 'http://localhost:3000',
}

This context is completely absent. An agent wouldn't know what the system under test actually is. Also missing: the Chrome launch flags critical for WebRTC testing:

• --use-fake-ui-for-media-stream / --use-fake-device-for-media-stream — allows media access without real devices
• --use-file-for-fake-audio-capture=dummyAudio.wav — provides fake audio input
• --remote-debugging-port=${9221 + index} — unique debug port per set

Add a ## System Under Test paragraph in the Dynamic Project Generation section explaining what is actually being tested and how the browser is configured.

M7: Factual Error — testInfo.project.name

This line says TestManager "reads set-scoped env values using testInfo.project.name".

Actually, TestManager constructor takes projectName: string directly:

constructor(projectName: string, maxRetries: number = DEFAULT_MAX_RETRIES)

Tests instantiate it as: new TestManager(test.info().project.name). The distinction matters — TestManager doesn't access testInfo at all; it receives the project name as a constructor argument.

Fix to: "TestManager receives the project name (SET key) via its constructor, which tests pass from test.info().project.name."

M8: Governance Gap — Doc Updates Not Enforced

This Done Criteria is missing mandatory doc update steps. Currently has 3 items, none mention AGENTS.md or ARCHITECTURE.md.

Should include:

• - [ ] playwright/ai-docs/AGENTS.md baseline updated (if new suite/set added)
• - [ ] playwright/ai-docs/ARCHITECTURE.md file topology and set mapping updated (if new files/sets added)

Also:

• 03-framework-and-doc-updates.md says "update only if required" — should be mandatory in the 00-master workflow when adding new tests/suites/sets
• 04-validation.md should explicitly check: - [ ] AGENTS.md baseline matches actual suites and - [ ] ARCHITECTURE.md file topology matches actual files
• The PR description claims playwright/ai-docs/specs/multiparty-conference.spec.md was added — but this file is not in the diff. Either implement the specs/ concept or remove the claim.
• When fixing/stabilizing tests, there's no guidance to update docs to reflect what changed — add a fix → doc-update loop.

Map SET_1 to actually wired test factories

This mapping lists dial-number-task-control-test.spec.ts under SET_1, but playwright/suites/digital-incoming-task-tests.spec.ts only registers createDigitalIncomingTaskAndTaskControlsTests via test.describe(...) and never wires the dial-number factory, so contributors using this table can add scenarios in the wrong file and silently miss coverage in SET_1. Please document the factories that are actually executed (or explicitly call out imported-but-unwired files).

Useful? React with 👍 / 👎.