Testing Tool Suite — Design Document

Status: Accepted ADR: ADR 0042 — Manual Testing Tool SuiteADO Epic: AB#4415

---

Purpose

This document describes how each tool in the testing suite is set up, used, and interpreted. It is the hands-on companion to ADR 0042, which records the decision. Each section covers one layer of the stack.

---

1. Infrastructure — PSRule for Azure

What it tests: Bicep templates in infrastructure/ against Azure Well-Architected Framework rules and Microsoft naming/security best practices.

Why: Catches misconfigurations (missing TLS enforcement, public blob access, insecure defaults) before a deployment touches Azure. Runs entirely offline — no subscription required.

Setup

Install-Module PSRule -Scope CurrentUser -Force
Install-Module PSRule.Rules.Azure -Scope CurrentUser -Force

Run

Invoke-PSRule -InputPath infrastructure/ -Module 'PSRule.Rules.Azure' -Format File

Interpreting results

• Pass — template meets the rule.
• Fail — rule violation with the resource name, rule ID, and a link to the relevant Azure docs.
• None — no resources matched the rule (not a failure).

To suppress a rule that intentionally does not apply (e.g. a cost-related rule not relevant to this scale), add a .ps-rule/suppress.yaml:

apiVersion: github.com/microsoft/PSRule/v1
kind: SuppressionGroup
metadata:
  name: intentional-suppressions
spec:
  rule:
    - 'Azure.Storage.MinTLS'   # example — replace with actual rule ID
  if:
    name: '.'

Document every suppression with a comment explaining why.

---

2. Infrastructure — ARM Template Toolkit (arm-ttk)

What it tests: The ARM JSON that Bicep compiles to — structural correctness, parameter hygiene, naming conventions.

Why: Catches errors in compiled output that PSRule does not cover, such as incorrect dependsOn chains or missing required parameters.

Setup

Install-Module Az.Resources -Scope CurrentUser -Force
# arm-ttk ships with Az.Resources, or clone directly:
# git clone https://github.com/Azure/arm-ttk.git D:\tools\arm-ttk
Import-Module D:\tools\arm-ttk\arm-ttk\arm-ttk.psd1

Run

First compile Bicep to ARM JSON, then test:

az bicep build --file infrastructure/modules/main.bicep --outdir infrastructure/compiled/
Test-AzTemplate -TemplatePath infrastructure/compiled/

Interpreting results

Each test produces Pass / Fail / Warning. Failures block deployment; warnings are advisory. Common failures and fixes are documented at the arm-ttk wiki.

---

3. API — Unit and Integration Tests (Vitest)

What it tests: Every service class in apps/api/src/features/ in isolation (mocked Prisma), plus auth guards, error envelopes, and route registration via Fastify app.inject().

Why: Verifies that service logic, validation rules, RBAC checks, and error handling all behave as written — without a database or live Clerk connection.

How the mock works

apps/api/src/test/setup.ts provides a shared Prisma mock for every model. The mock is injected at test startup via vi.mock('../adapters/db/index', ...). Tests override individual mock return values with vi.mocked(...).mockResolvedValueOnce(...) per test case.

When the Prisma schema changes, setup.ts must be updated to add any new models or methods — otherwise tests silently use undefined and produce misleading results.

Run (from bld-01 or WSL)

pnpm --filter @hch/api test

To run a single file:

pnpm --filter @hch/api exec vitest run src/features/members/members.service.test.ts

To run with coverage:

pnpm --filter @hch/api exec vitest run --coverage

Interpreting results

• Green — all assertions passed.
• Red — assertion failure with the expected vs. received values and the stack trace pointing to the exact line.
• If a test fails due to Cannot read properties of undefined on a mock — the model or method is missing from setup.ts.

---

4. API — Live Endpoint Testing (Bruno)

What it tests: The deployed API at https://api.heritageva.app — real HTTP requests, real responses, real auth tokens.

Why: Vitest mocks the database. Bruno hits the live system and catches issues that only surface with real data, real Clerk tokens, and real Azure infrastructure.

Setup

Download Bruno from usebruno.com — no account required, no cloud sync.

Request collections live in apps/api/requests/. Open Bruno and point it at that folder.

Environment variables

Bruno uses environment files (not committed) to store the Clerk session token:

# apps/api/requests/environments/production.bru
vars {
  base_url: https://api.heritageva.app
  auth_token: <paste Clerk JWT here>
}

Get a Clerk JWT by signing in at https://heritageva.app, opening browser DevTools → Application → Cookies, and copying the __session value.

Run

Open the collection in Bruno and run individual requests or the full collection. Each request shows the status code, response body, and response time.

VS Code alternative

.http files in apps/api/requests/ work with the VS Code REST Client extension as an alternative to Bruno. Same collections, editor-native.

---

5. Web — Component Tests (Vitest + React Testing Library)

What it tests: Key pages and components in apps/web/src/ — rendering, conditional visibility, user interactions, role-based UI gating.

Why: Verifies that UI components behave correctly without opening a browser — catches broken conditionals, missing elements, and interaction bugs early.

Run (from bld-01 or WSL)

pnpm --filter @hch/web test

What is covered

• SignInPage — logo present, Clerk widget rendered, child sign-in link visible
• AboutPage — section structure, feedback form open/close, conditional fields by type
• MembersPage — heading, search, empty/loading/error states, grouped list, alpha sort

Interpreting results

Same as API Vitest — green/red with assertion details. If a test fails because a component changed its structure, update the test to match the new structure (the test is wrong, not the component, unless the change was unintentional).

---

6. Web — Browser E2E (Playwright)

What it tests: Critical user flows in a real Chromium browser against the live deployed environment — sign-in redirect, authenticated page access, API health.

Why: The only layer that exercises the full stack end-to-end: CDN → SWA → Clerk → API → Postgres. Catches integration failures invisible to unit tests.

Setup

pnpm add -D @playwright/test   # at repo root if not already present
npx playwright install chromium

Run

E2E_BASE_URL=https://heritageva.app npx playwright test

What is covered (smoke suite in e2e/)

• Sign-in page loads with logo
• Child sign-in link is visible
• Unauthenticated access to /app/about redirects to /sign-in
• Unauthenticated access to /app/members redirects to /sign-in
• GET /health returns HTTP 200

Interpreting results

Playwright produces an HTML report (playwright-report/). Open it with npx playwright show-report. Each failed test includes a screenshot, trace, and the exact assertion that failed.

---

7. Shared Packages (Vitest)

What it tests: @hch/shared-config (featureRegistry filtering, role logic) and @hch/shared-types (ROLE_LEVEL shape, FEATURE_ROLES set membership).

Why: These packages are consumed by both the API and the web app. A bug here breaks everything.

Run (from bld-01 or WSL)

pnpm --filter @hch/shared-config test
pnpm --filter @hch/shared-types test

---

8. PowerShell Scripts (Pester 5)

What it tests: Any PowerShell automation scripts in this repo — infrastructure helpers, seed scripts, tooling.

Why: Pester is the PowerShell standard. Scripts that are run in production (seeding, migrations, operational tasks) should have tests confirming they behave correctly before being run against live infrastructure.

Setup

Install-Module Pester -MinimumVersion 5.0 -Scope CurrentUser -Force

Run

Invoke-Pester -Path ./scripts/ -Output Detailed

Test files follow the naming convention <ScriptName>.Tests.ps1 and live alongside the script they test.

---

9. Dependency Scanning (Snyk)

What it tests: All npm packages in the monorepo for known CVEs.

Why: Third-party packages introduce vulnerabilities that are invisible in code review. Snyk checks against the NVD and Snyk vulnerability database.

Setup

npm install -g snyk
snyk auth   # one-time browser login

Run

npx snyk test --severity-threshold=high

Interpreting results

• Lists vulnerable packages with CVE ID, severity, affected version, and the fixed version.
• high threshold means critical and high severity vulnerabilities fail the check; medium and low are reported but do not fail.
• Fix by upgrading the package: pnpm update <package-name>.

---

10. Secret Scanning (Gitleaks)

What it tests: The full git history for accidentally committed secrets, tokens, connection strings, or private keys.

Why: Secrets committed to git persist in history even after deletion. Gitleaks scans every commit.

Setup

winget install gitleaks   # or download from github.com/gitleaks/gitleaks/releases

Run

gitleaks detect --source . --verbose --redact

--redact replaces matched secret values in output with REDACTED so the scan result itself does not leak secrets.

Interpreting results

• No findings — clean.
• Finding — shows the file, commit SHA, line number, and rule that matched. If it is a false positive, add a .gitleaks.toml allowlist entry with a comment.

---

11. Static Analysis (ESLint + TypeScript)

What it tests: Type correctness, unsafe patterns, and style violations across all TypeScript packages.

Why: The TypeScript compiler catches type errors that runtime tests miss. ESLint enforces patterns that prevent bugs (no implicit any, exhaustive switch, etc.).

Run

pnpm turbo lint

This runs tsc --noEmit and eslint across all packages via Turborepo. Fix all errors before deploying — warnings are advisory.

---

Pre-deployment checklist

Run the tools relevant to what changed before pushing to production:

Changed area	Tools to run
infrastructure/	PSRule, arm-ttk
apps/api/src/	Vitest (API), Bruno (live endpoints)
apps/web/src/	Vitest (web), Playwright (smoke)
packages/	Vitest (shared packages)
scripts/ (PowerShell)	Pester
Any package.json change	Snyk
Any file (before release)	Gitleaks, ESLint + tsc

---

References

• ADR 0042 — Manual Testing Tool Suite
• AB#4415 — Build comprehensive automated test suite
• AB#4416 — API service unit tests (220+ tests, complete)
• AB#4417 — API controller/route tests
• AB#4418 — API integration tests
• AB#4419 — Web component tests
• AB#4420 — Shared package tests
• AB#4421 — E2E smoke tests (Playwright)