630 critical and high CVEs to zero, with no downtime
Three HIPAA production services carried 191, 244 and 195 critical and high findings on end-of-life base images, and their infrastructure code had drifted so far that the next apply would have downgraded the production database. I took all three to zero and made the infrastructure code trustworthy again.
Context
An audit of three HIPAA production services handling patient data at Tembo Health showed 191, 244 and 195 critical and high container findings on end-of-life Node 20 and Alpine base images. At the same time, the infrastructure code for all three had drifted from reality. The Aurora clusters had been rebuilt outside it, the templates still described the old database version, and a Terraform pipeline had been failing silently for nine months. On its next successful apply it would have downgraded the production database.
The constraints: zero findings on all three services, infrastructure code that matches what is running, no downtime, and no writes to the production database.
What I did
I wrote a plan per service with explicit decision points before touching anything. Instead of patching symptoms in each service, I fixed the root causes in the shared libraries: 18 releases across 12 libraries, each verified against the consuming service before publishing.
For the infrastructure, I previewed every change read-only first, repaired the Terraform state, and re-imported the production cluster into CloudFormation with a retain-and-import sequence rather than accept the replacement the tooling proposed. Eight Lambdas moved off Node 18 in the same pass.
Result
All three services scan clean at every severity on ECR Inspector and Trivy. Images shrank about 70%. The Terraform pipeline applies cleanly again, with deletion protection on the production database. Zero downtime, zero database writes, about three weeks of work.
Stack
ECR Inspector · Trivy · Docker · Terraform · CloudFormation · Aurora · Lambda · Node.js