Skip to content

Selected work · DevOps · Infrastructure · 1 min read

Running it in production: workers, monitoring and pipelines

Media workers scaled from 1 to 8 behind NGINX, with monitoring that finds new hosts by itself and pipelines that refuse to do damage.

TL;DR

  • Scaled voice WebSocket handling from 1 to 8 workers behind NGINX least_conn, with Prometheus/Grafana and 5 Ansible roles.
  • Hardened the LLM path as DRI for 150K+ calls/month: 7 s timeout with fallback, tool-call guards, a leak guard tested on 269 replies.
  • Built pipelines that refuse to do damage: integrity checks block destructive steps; dry-run first.
Role
Intern and FDE at Exotel; side projects
Timeline
2025 – 2026
Links
Internal work; no public links

How it fits together

Voice media path

  1. Calls
  2. NGINX least_conn
  3. 8 Pipecat WebSocket workers
  4. Prometheus exporters
  5. Grafana dashboards

Provisioning

  1. 5 Ansible roles
  2. EC2 discovery
  3. New hosts show up in Grafana
Simplified view of the voice media path and how hosts are provisioned.

Voice media pipeline

Scaled Pipecat WebSocket handling from 1 to 8 workers behind NGINX least_conn. Added Prometheus exporters and Grafana dashboards with EC2 service discovery, 5 Ansible roles for automated provisioning, and logs for debugging. Eight workers is a worker count, not a capacity multiple.

LLM path hardening (DRI for 150K+ calls/month)

A 7-second wall-clock timeout with a fallback, parallel-tool-call guards, a TTS markdown filter, and a reasoning-leak guard tested on 269 labelled production replies. Reliability work, not model work.

Campaign automation

A twice-daily Jenkins pipeline (Python, AWS S3/SES, headless Chromium) with integrity checks that block destructive steps, a dry-run mode, UAT mocks and separate failure notices. Batching cut ~10K-row loads from ~25 to ~2 minutes.

Side projects

  • Cognitive-OS ships as Docker images to Azure Container Apps through GitHub Actions.
  • The Wedding Home runs typecheck, 11 Vitest suites and a Playwright journey in CI on every push, with a daily Vercel cron.
  • AURA runs on Azure App Service.

Build log: Scaling the media workers

Hover a step to see where it happened in the diagram above.

  1. step· 8 Pipecat WebSocket workers

    1 worker

  2. fail· 8 Pipecat WebSocket workers

    Bottleneck under concurrent calls

  3. fix· NGINX least_conn

    Add 7 workers behind NGINX least_conn

  4. pass· Prometheus exporters

    Add exporters + Grafana panels

  5. pass· 5 Ansible roles

    Provision with Ansible roles

  6. pass· EC2 discovery

    EC2 discovery so new hosts appear automatically

Results

  • 1 → 8

    WebSocket workers

    Worker count behind NGINX least_conn, not a capacity multiple

  • 5

    Ansible roles

    Automated provisioning

  • 269

    labelled production replies

    Test set for the reasoning-leak guard

  • 7/7

    successful production builds

    Campaign pipeline, first observation window

What I'd do differently

  • Load-test with a recorded call mix before and after, so the scaling change has a measured number.
  • Codify the dashboards as code.

Stack

  • NGINX
  • Pipecat
  • Prometheus
  • Grafana
  • Ansible
  • AWS (EC2, S3, SES)
  • Jenkins
  • Docker
  • GitHub Actions
  • Azure Container Apps / App Service
  • Vercel