A HIPAA data pipeline for a 100K+ patient partner
An AWS Transfer Family SFTP connector and a nightly idempotent sync that ingest thousands of patient records per file from a population-health partner, operated by non-technical staff from a folder, driving net-new patient enrollment.
Context
Tembo Health partnered with one of the larger population-health management companies in the U.S., serving a patient population over 100,000. The partner sends files containing thousands of patient records on a regular cadence, some of them multi-gigabyte, and Tembo sends reports and invoices back. All of it had to be automated inside HIPAA constraints and operable by people who are not engineers.
Tembo’s two earlier SFTP integrations had partners pushing files to Tembo’s server. This partner needed Tembo to connect to theirs. I owned the design and delivery end to end.
What I did
I chose a pull architecture instead of forcing the existing push pattern: an AWS Transfer Family SFTP connector with S3 as the state store and a TypeScript Lambda job that syncs nightly, dedupes by size against what already landed, batches transfers, and recovers on its own from interrupted runs. Access is scoped with prefix-level least-privilege IAM, credentials live in Secrets Manager, and every failure raises a CloudWatch alarm in Slack.
On the operations side, two stateless n8n workflows sync Google Drive folders to S3, so the only manual step for the operations team is dropping a file in the right folder.
Result
Files with thousands of patient records each ingest cleanly without intervention. The integration directly drives net-new patient enrollment at the partner’s scale, and the pull-connector pattern became the template for the next partner integration.
Stack
AWS Transfer Family · S3 · Lambda · TypeScript · IAM · Secrets Manager · CloudWatch · n8n