ai_ml_operations/server-infra
2
0
Fork 0
Code Issues 3 Pull Requests Actions Packages Projects Releases Wiki Activity

Auto-start monica_site dj-queue worker on adama during deploy #17

New Issue
Closed
opened 2026-08-08 11:55:56 -07:00 by westfarn · 1 comment
westfarn commented 2026-08-08 11:55:56 -07:00
Owner
Copy Link
Copy Source

Problem

Campaign test email sends synchronously (Django mail) and works.
Real campaign send enqueues Django Tasks via dj-queue. Beta/prod use DjQueueBackend; the web container only enqueues.

The compose worker service (profile: worker) was documented as a manual ops step on adama. Catalog deploy only runs docker compose up -d for web, so the worker never starts. Campaign messages stay Queued forever — no SMTP send.

Also: plain up -d --remove-orphans without the worker profile can drop an already-running worker container on later deploys.

Goal

On deploy, start the dj-queue worker automatically on adama only (singleton) for both monica_site beta and prod. Do not start it on roslin / ai-server-4080.

Proposed change

  1. Optional compose_profiles on host_apps entries (e.g. [worker] on adama for monica_site beta+prod).
  2. roles/app-deploy/tasks/django.yml: pass profiles into docker compose up (via COMPOSE_PROFILES or --profile) so adama brings up web + worker together and keeps worker across redeploys.
  3. Update app_catalog comments + IMPLEMENTATION.md (remove “manual worker” wording).

Verify

# after deploy on adama
docker compose -f docker-compose.prod.yml --env-file .env ps
# expect web + worker for monica_site_beta / monica_site_prod

Portal: send campaign → recipient status leaves Queued → Sent.

Related

  • Closed: #14 (monica_site catalog; worker called out as manual)
  • App: monica_site messaging enqueue_campaign_send / send_campaign_message
## Problem Campaign **test email** sends synchronously (Django mail) and works. Real campaign send enqueues Django Tasks via **dj-queue**. Beta/prod use `DjQueueBackend`; the web container only enqueues. The compose `worker` service (`profile: worker`) was documented as a **manual** ops step on adama. Catalog deploy only runs `docker compose up -d` for `web`, so the worker never starts. Campaign messages stay **Queued** forever — no SMTP send. Also: plain `up -d --remove-orphans` without the worker profile can drop an already-running worker container on later deploys. ## Goal On deploy, start the dj-queue worker **automatically on adama only** (singleton) for both `monica_site` beta and prod. Do **not** start it on roslin / ai-server-4080. ## Proposed change 1. Optional `compose_profiles` on `host_apps` entries (e.g. `[worker]` on adama for monica_site beta+prod). 2. `roles/app-deploy/tasks/django.yml`: pass profiles into `docker compose up` (via `COMPOSE_PROFILES` or `--profile`) so adama brings up `web` + `worker` together and keeps worker across redeploys. 3. Update `app_catalog` comments + `IMPLEMENTATION.md` (remove “manual worker” wording). ## Verify ```bash # after deploy on adama docker compose -f docker-compose.prod.yml --env-file .env ps # expect web + worker for monica_site_beta / monica_site_prod ``` Portal: send campaign → recipient status leaves **Queued** → **Sent**. ## Related - Closed: #14 (monica_site catalog; worker called out as manual) - App: `monica_site` messaging `enqueue_campaign_send` / `send_campaign_message`
westfarn commented 2026-08-08 11:58:42 -07:00
Author
Owner
Copy Link
Copy Source

Implementation (local branch)

Branch: issue-17-monica-dj-queue-worker (from master).

Changes

  • roles/app-deploy/tasks/django.yml — pass COMPOSE_PROFILES from optional host_apps.compose_profiles into docker compose up
  • inventory/host_vars/adama.yml — monica_site beta+prod set compose_profiles: [worker]
  • roslin / ai-server-4080 — web only (no profiles)
  • Docs: app_catalog comment + IMPLEMENTATION.md

Not committed/pushed yet — say if you want a PR + deploy smoke.

## Implementation (local branch) Branch: `issue-17-monica-dj-queue-worker` (from `master`). ### Changes - `roles/app-deploy/tasks/django.yml` — pass `COMPOSE_PROFILES` from optional `host_apps.compose_profiles` into `docker compose up` - `inventory/host_vars/adama.yml` — `monica_site` beta+prod set `compose_profiles: [worker]` - `roslin` / `ai-server-4080` — web only (no profiles) - Docs: `app_catalog` comment + `IMPLEMENTATION.md` Not committed/pushed yet — say if you want a PR + deploy smoke.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
twestfall westfarn (Ryan Westfall)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: ai_ml_operations/server-infra#17