Columbia Games · CCM v2 / August 2026 27.
Live QA has to test the SHA that is actually running
Post-deploy QA was tied to the exact production SHA, with a local Lighthouse fallback for PageSpeed API quota failures. A third-party service limit could no longer make a release effectively blind.
Situation
With asynchronous QA workflows, the branch can move to another commit before the checks even start. That creates the dangerous possibility of a green QA result for a revision other than the one that was actually deployed. At the same time, PageSpeed API quota exhaustion or a transient API failure must not mean that performance verification disappears entirely from the release process.
Approach
The production workflow began dispatching live smoke and extended QA with the explicit deployed SHA, and the report verified that identity. For PageSpeed, I added quota-aware handling: when the external API was unavailable, a locked Lighthouse runtime ran as the fallback, while the report still made the degraded path visible. I also preserved the post-deploy QA exit code through workflow orchestration so a real failure could not be accidentally swallowed.
Outcome
QA evidence became traceable to a specific release, and the performance signal no longer depended on a single external quota-controlled service. The release pipeline could distinguish a genuine test failure from a degraded measurement path without silently skipping the check.