An observable QA pipeline: .NET, Playwright, GitHub Actions, Datadog CI Visibility

The Anvil

The client had already built some pipelines and had a rough idea of what they wanted to automate for E2E testing via CI/CD. What they were missing: a clear, understandable approach to observability — and a clean pattern for reusable actions and workflows in GitHub Actions.

The Strike

I led design and build of the QA automation pipeline end-to-end. The pieces:

• Test layer — .NET test runner driving Playwright for browser / E2E coverage. Page-object architecture, a parallel sharding scheme, and out-of-the-box Playwright setup/teardown wired through Datadog for observability.
• CI orchestration — GitHub Actions workflows tuned for PR feedback under 5 minutes, even with dozens of tests across multiple repos. Reusable workflows, artifact handling, retry policy, secrets management, and observability baked in by default.
• Observability — Datadog CI Visibility wired in, so every run reports test durations, flakiness rates, and failure patterns to the same dashboards the rest of the platform uses. Flaky-test triage moved from anecdote to data.

The Edge

PR feedback time dropped from 8 minutes to 4. The flaky-test list went from “vibes” to a ranked dashboard, alongside rich Datadog CI and RUM dashboards. The team now owns the pipeline confidently, and the tools see daily use with consistent praise from stakeholders.