Skip to main content

Verifying Web UI changes (Playwright + rich HTML report)

When you modify the Web UI (any Vue file under frontend/src/), verify it end-to-end with a Playwright sweep that captures screenshots and packages them into a self-contained HTML report. This is the same workflow used to verify Spec 046 v2 — see specs/046-local-first-onboarding/verification/ for a worked example.

The standing sweep (one command)​

The core-screen sweep is committed and scripted — run it before you hand-roll anything:

./scripts/run-web-smoke.sh --show-report   # boots a throwaway instance, runs e2e/web-ui-sweep

The launcher builds ./mcpproxy if needed, serves a throwaway instance on 127.0.0.1:18080 under a freshly generated throwaway API key (your own MCPPROXY_API_KEY is deliberately ignored — the key ends up in report URLs and traces), installs Chromium from the committed e2e/web-ui-sweep/package-lock.json via npm ci, and runs e2e/web-ui-sweep/web-ui-sweep.spec.ts — servers list, server detail (+ security tab), tools page and search, activity log, settings — failing on uncaught page exceptions. Pass MCPPROXY_FIXTURE_PATH=$(go build -o /tmp/mcpfixture ./cmd/mcpfixture && echo /tmp/mcpfixture) to register a live stdio upstream so the server- and tool-dependent checks run instead of skipping. The HTML report lands in tmp/web-smoke-artifacts/playwright-report/.

Alongside it, e2e/web-ui-sweep/visual-a11y-sweep.spec.ts guards the appearance contract that the 2026-08 UX audit found broken: it walks every visible text node on the main screens in both shipped themes, resolves the effective foreground/background through a canvas (theme tokens are oklch(), so a naive rgb() parse silently measures nothing) and fails anything below WCAG AA; it also checks the 390/820/1440px layouts, accessible names on every form control, the activity log's aria-live region and caption, the system-theme resolution, and that the header search button is not disabled at rest. Same launcher, same environment variables.

The release QA gate runs this exact script on every tag as its advisory web-ui-sweep job — see Release Gate. Extend the committed sweep when you add a screen worth guarding on releases; use the ad-hoc pattern below for the deeper, spec-specific verification that ships beside a spec.

Ad-hoc, spec-specific verification​

The pattern, in order:

  1. Stand up a fresh mcpproxy. Use a throwaway data-dir so persisted state doesn't bleed between runs: 
    pkill -f 'mcpproxy serve.*<port>' 2>/dev/null; sleep 1
    rm -rf /tmp/mcpproxy-uitest/{config.db,index.bleve,logs} 2>/dev/null
    cat > /tmp/mcpproxy-uitest/mcp_config.json <<'EOF'
    { "listen": "127.0.0.1:18081", "data_dir": "/tmp/mcpproxy-uitest", "api_key": "uitest", "enable_web_ui": true, "enable_socket": false, "telemetry": {"enabled": false}, "mcpServers": [] }
    EOF
    ./mcpproxy serve --config=/tmp/mcpproxy-uitest/mcp_config.json --listen=127.0.0.1:18081 --log-level=info > /tmp/mcpproxy-uitest/server.log 2>&1 &
    until curl -sf -H "X-API-Key: uitest" http://127.0.0.1:18081/api/v1/status >/dev/null; do sleep 1; done
  2. Reuse the existing Playwright install. e2e/playwright/node_modules already has Playwright + Chromium 1217. Symlink it into your scratch dir: 
    mkdir -p /tmp/uitest && cd /tmp/uitest
    ln -sfn /Users/user/repos/mcpproxy-go/e2e/playwright/node_modules ./node_modules
  3. Pin the Chromium binary in playwright.config.ts so Playwright doesn't try to download a different version: 
    import { defineConfig } from '@playwright/test';
    export default defineConfig({
      testDir: '.', timeout: 30000, fullyParallel: false, workers: 1, retries: 0,
      use: {
        headless: true,
        viewport: { width: 1440, height: 900 },
        launchOptions: {
          executablePath: '/Users/user/Library/Caches/ms-playwright/chromium-1217/chrome-mac-arm64/Google Chrome for Testing.app/Contents/MacOS/Google Chrome for Testing',
        },
      },
    });
  4. Write the spec. Use data-test attributes already on the components (the project convention). For new components, add them. Drive scenarios with page.locator('[data-test="..."]'). Always use page.waitForLoadState('domcontentloaded') — networkidle hangs because of the SSE channel. Snapshot each state with page.screenshot({ path: ... }). Number screenshots in execution order so the report renders left-to-right.
  5. Run. ./node_modules/.bin/playwright test --reporter=list. Iterate until green.
  6. Build the rich HTML report. A short Python script that base64-embeds each PNG and wraps it in a styled <details> per scenario produces a single self-contained HTML file the user can open offline. Pattern: top summary card with pass/fail counts, then one collapsible per scenario with Expected / Observed / inline screenshot. The reference implementation is /tmp/wizard-v2-verify/build-report.py from the v2 work — clone it and update the SCENARIOS list. Output goes to specs/<feature>/verification/report.html.
  7. Drop screenshots + report alongside the spec. Always commit them with the spec changes — they're part of the trace.
  8. Surface the report. End your reply with open <path-to-report.html> so the user can review without re-running the suite.

Key gotchas:

  • The wizard's <dialog> element renders as [open] only when the Vue store sets it. To assert open/closed state robustly, query the dialog property in page.evaluate(), not aria-hidden or styling.
  • The default config from a stub file does NOT trigger applyFirstRunDockerIsolation — that only runs when the config file is absent at boot. To test the "Docker auto-enabled" path, either let mcpproxy create the config or pre-set docker_isolation.enabled: true in your stub.
  • For browser-driven verification of subtle states (badge counts, empty/loaded transitions), prefer the Playwright spec over ad-hoc screenshots from the chrome-in-chrome MCP — the spec is reproducible and a CI agent can re-run it.