ai_ml_operations/scha
2
0
Fork 0
Code Issues 16 Pull Requests Actions Packages Projects Releases Wiki Activity

Dockerize scha + Gitea CI/CD auto-deploy via server-infra #19

New Issue
Closed
opened 2026-07-11 05:51:50 -07:00 by westfarn · 0 comments
westfarn commented 2026-07-11 05:51:50 -07:00
Owner
Copy Link
Copy Source

Summary

Dockerize the scha (SCHA Wheaton) Django site and wire it into the same automated deploy pipeline used by company_site and dta_service. On merge/push to master, Gitea Actions runs tests, builds the image, then calls server-infra/scripts/deploy.sh to deploy active/active to all webservers (adama + roslin) behind NPM, against the shared external Postgres.

This mirrors the pattern documented in server-infra/IMPLEMENTATION.md (line item #10: "Dockerize company_site — Future (separate ticket)") and the reference implementation in company_site.

Current state of scha

  • Django 5.2, single app schasite, project scha.
  • Dependencies via requirements.txt (no uv, no gunicorn, no Postgres driver).
  • scha/settings.py is dev-only and blocks production:
    • DEBUG = True, ALLOWED_HOSTS = []
    • Hardcoded SECRET_KEY
    • SQLite (db.sqlite3) — must move to shared Postgres
    • Hardcoded secrets committed to git: Stripe keys, reCAPTCHA keys (STRIPE_*, RECAPTCHA_*)
  • No Dockerfile, no .gitea/workflows, no entrypoint/deploy scripts.

Target architecture (matches company_site / dta_service)

  • Django app = one docker compose project per env: project name scha_<env>, host port from host_apps.
  • Runs active/active on adama + roslin; NPM load-balances adama:PORT + roslin:PORT.
  • Shared external Postgres at 10.0.0.230:5432 (each app+env gets its own DB; beta and prod must NOT share a DB).
  • Secrets live in a per-app .env on the control node (~/Documents/secrets/scha/scha_<env>.env), pushed to hosts at deploy time to /opt/apps/env/scha_<env>.env. Never committed to git.
  • CI: push/PR to master runs unit tests; on green push, deploy job on the self-hosted runner (ai-server-4080) calls server-infra/scripts/deploy.sh --app scha --env prod --ref <sha>.

Proposed ports (no conflicts with existing catalog)

App prod beta
company_site 8000 8010
dta_service 8001 8011
dta_webapp (nginx) 8080 8081
scha (new) 8002 8012 (optional)

Prod is required. Beta optional — enable only if we want a staging replica.

Part A — Changes in THIS repo (scha)

A1. Migrate packaging to uv

  • Add pyproject.toml + uv.lock (drop/replace requirements.txt).
  • Dependencies: existing (django, django-phonenumber-field, django-recaptcha, phonenumbers, stripe, requests) plus gunicorn, psycopg2-binary, dj-database-url (or manual DATABASE_URL parsing), whitenoise (static serving), and a dev group with pre-commit.
  • Rationale: app_catalog.migrate_cmd is uv run python manage.py migrate --noinput and the Docker build uses uv sync --frozen --no-dev.

A2. Refactor scha/settings.py to be environment-driven

Model it on company_site. Read from env vars with safe dev defaults:

  • DJANGO_ENV (dev | beta | prod) → controls DEBUG default + logging level
  • DJANGO_SECRET_KEY (no committed default in prod)
  • DJANGO_DEBUG, DJANGO_ALLOWED_HOSTS, DJANGO_CSRF_TRUSTED_ORIGINS (derive https origins from allowed hosts when unset)
  • DATABASE_URL → Postgres (SQLite fallback for pure-local dev only)
  • Move all currently-hardcoded secrets to env: STRIPE_PUBLISHABLE_KEY, STRIPE_SECRET_KEY, STRIPE_ENDPOINT_SECRET, RECAPTCHA_PUBLIC_KEY, RECAPTCHA_PRIVATE_KEY
  • Whitenoise middleware + STATIC_ROOT for collectstatic
  • Preserve existing CSRF trusted origins: schawheaton.aimloperations.com, www.schawheaton.aimloperations.com, schawheaton.com, www.schawheaton.com

⚠️ Security: the committed Stripe/reCAPTCHA keys must be treated as compromised and rotated once moved to env-only. (Note: these appear to be Stripe test keys, but rotate to be safe and to set real prod keys.)

A3. Docker assets (copy/adapt from company_site)

  • Dockerfile — python:3.12-slim + uv, uv sync --frozen --no-dev, copy app, entrypoint, EXPOSE 8000.
  • .dockerignore — .git, .venv, __pycache__, db.sqlite3, .env, staticfiles/, etc.
  • scripts/docker-entrypoint.sh — wait for DB → migrate --noinput → collectstatic --noinput → exec gunicorn scha.wsgi:application.
  • scripts/validate-env.sh — assert required prod env vars present.
  • docker-compose.yml (dev, bundled Postgres for local).
  • docker-compose.prod.yml — web service reads DATABASE_URL/DB_* pointing at the shared external Postgres; host port via ${WEB_PORT}. Per IMPLEMENTATION.md, prod must NOT bundle its own DB.
  • .env.example (dev) and .env.prod.example (server template).

A4. Gitea workflows (.gitea/workflows/)

  • unittests.yml — on push + PR to master: checkout, install uv, uv sync --frozen, uv run python manage.py test.
  • deploy.yml — workflow_run after Unit Tests completes on master, push events only:
    1. docker job: build image + run containerized tests.
    2. deploy job (needs docker): run "$SERVER_INFRA_ROOT/scripts/deploy.sh" --app scha --env prod --ref "${{ gitea.event.workflow_run.head_sha }}" on self-hosted.
  • Deploy never runs on PRs (separate workflow files, matching company_site).
  • (Optional) ci.yml for PR-only checks.

A5. Docs

  • Update README.md with local dev (uv + docker), env matrix, and CI/deploy flow.

Part B — Changes in server-infra (register scha)

B1. Add scha to app_catalog (inventory/group_vars/all.yml)

scha:
  type: django
  repo: "{{ git_base_url }}/ai_ml_operations/scha.git"
  default_branch: master
  compose_file: docker-compose.prod.yml
  web_service: web
  migrate_cmd: "uv run python manage.py migrate --noinput"

B2. Add scha to host_apps on all app hosts

Add to inventory/host_vars/adama.yml and inventory/host_vars/roslin.yml (ports MUST match across hosts for NPM balancing):

- { name: scha, env: prod, port: 8002 }
# optional: - { name: scha, env: beta, port: 8012 }

(ai-server-4080 stays workload-free.)

B3. Shared Postgres (10.0.0.230)

  • Create DB scha (+ scha_beta if beta) and grant westfarn.

B4. Control-node secret

  • Create ~/Documents/secrets/scha/scha_prod.env with DATABASE_URL=postgres://westfarn:<pw>@10.0.0.230:5432/scha, DJANGO_ENV=prod, DJANGO_SECRET_KEY, DJANGO_ALLOWED_HOSTS, WEB_PORT=8002, and the real Stripe/reCAPTCHA prod values. Mode 600, never committed.

B5. NPM reverse proxy / load balancing

  • Point schawheaton.aimloperations.com, schawheaton.com (+ www) at backends adama:8002 + roslin:8002 (Advanced upstream {} block for active/active).

Acceptance criteria

  • scha builds and runs via docker compose locally (dev) and docker-compose.prod.yml (prod against external Postgres).
  • No secrets in git; all secrets sourced from env; old committed keys rotated.
  • Push/PR to master runs unit tests in Gitea Actions.
  • Push to master (post-tests) auto-deploys scha prod to adama + roslin via server-infra/scripts/deploy.sh.
  • scha registered in server-infra app_catalog + host_apps on all webservers.
  • Shared Postgres DB + control-node secret provisioned.
  • Site reachable via NPM (active/active) on the schawheaton domains.

References

  • company_site (reference impl): Dockerfile, docker-compose*.yml, scripts/, .gitea/workflows/{unittests,ci,deploy}.yml
  • server-infra/IMPLEMENTATION.md — architecture, ports, shared DB, "Required changes IN each app repo", item #10
  • server-infra/scripts/deploy.sh, playbooks/deploy-apps.yml, roles/app-deploy/tasks/{main,django}.yml
## Summary Dockerize the `scha` (SCHA Wheaton) Django site and wire it into the same automated deploy pipeline used by `company_site` and `dta_service`. On merge/push to `master`, Gitea Actions runs tests, builds the image, then calls `server-infra/scripts/deploy.sh` to deploy active/active to all webservers (adama + roslin) behind NPM, against the shared external Postgres. This mirrors the pattern documented in `server-infra/IMPLEMENTATION.md` (line item #10: "Dockerize company_site — Future (separate ticket)") and the reference implementation in `company_site`. --- ## Current state of `scha` - Django 5.2, single app `schasite`, project `scha`. - Dependencies via `requirements.txt` (no `uv`, no `gunicorn`, no Postgres driver). - `scha/settings.py` is dev-only and blocks production: - `DEBUG = True`, `ALLOWED_HOSTS = []` - Hardcoded `SECRET_KEY` - **SQLite** (`db.sqlite3`) — must move to shared Postgres - Hardcoded secrets committed to git: Stripe keys, reCAPTCHA keys (`STRIPE_*`, `RECAPTCHA_*`) - No `Dockerfile`, no `.gitea/workflows`, no entrypoint/deploy scripts. --- ## Target architecture (matches company_site / dta_service) - Django app = one docker compose project per env: project name `scha_<env>`, host port from `host_apps`. - Runs active/active on **adama** + **roslin**; NPM load-balances `adama:PORT` + `roslin:PORT`. - Shared **external Postgres** at `10.0.0.230:5432` (each app+env gets its own DB; beta and prod must NOT share a DB). - Secrets live in a per-app `.env` on the control node (`~/Documents/secrets/scha/scha_<env>.env`), pushed to hosts at deploy time to `/opt/apps/env/scha_<env>.env`. Never committed to git. - CI: push/PR to `master` runs unit tests; on green push, deploy job on the self-hosted runner (ai-server-4080) calls `server-infra/scripts/deploy.sh --app scha --env prod --ref <sha>`. ### Proposed ports (no conflicts with existing catalog) | App | prod | beta | |-----|------|------| | company_site | 8000 | 8010 | | dta_service | 8001 | 8011 | | dta_webapp (nginx) | 8080 | 8081 | | **scha (new)** | **8002** | **8012** (optional) | Prod is required. Beta optional — enable only if we want a staging replica. --- ## Part A — Changes in THIS repo (`scha`) ### A1. Migrate packaging to `uv` - Add `pyproject.toml` + `uv.lock` (drop/replace `requirements.txt`). - Dependencies: existing (`django`, `django-phonenumber-field`, `django-recaptcha`, `phonenumbers`, `stripe`, `requests`) **plus** `gunicorn`, `psycopg2-binary`, `dj-database-url` (or manual `DATABASE_URL` parsing), `whitenoise` (static serving), and a `dev` group with `pre-commit`. - Rationale: `app_catalog.migrate_cmd` is `uv run python manage.py migrate --noinput` and the Docker build uses `uv sync --frozen --no-dev`. ### A2. Refactor `scha/settings.py` to be environment-driven Model it on `company_site`. Read from env vars with safe dev defaults: - `DJANGO_ENV` (`dev` | `beta` | `prod`) → controls DEBUG default + logging level - `DJANGO_SECRET_KEY` (no committed default in prod) - `DJANGO_DEBUG`, `DJANGO_ALLOWED_HOSTS`, `DJANGO_CSRF_TRUSTED_ORIGINS` (derive https origins from allowed hosts when unset) - `DATABASE_URL` → Postgres (SQLite fallback for pure-local dev only) - Move **all** currently-hardcoded secrets to env: `STRIPE_PUBLISHABLE_KEY`, `STRIPE_SECRET_KEY`, `STRIPE_ENDPOINT_SECRET`, `RECAPTCHA_PUBLIC_KEY`, `RECAPTCHA_PRIVATE_KEY` - Whitenoise middleware + `STATIC_ROOT` for `collectstatic` - Preserve existing CSRF trusted origins: `schawheaton.aimloperations.com`, `www.schawheaton.aimloperations.com`, `schawheaton.com`, `www.schawheaton.com` > ⚠️ Security: the committed Stripe/reCAPTCHA keys must be treated as compromised and **rotated** once moved to env-only. (Note: these appear to be Stripe *test* keys, but rotate to be safe and to set real prod keys.) ### A3. Docker assets (copy/adapt from company_site) - `Dockerfile` — `python:3.12-slim` + `uv`, `uv sync --frozen --no-dev`, copy app, entrypoint, `EXPOSE 8000`. - `.dockerignore` — `.git`, `.venv`, `__pycache__`, `db.sqlite3`, `.env`, `staticfiles/`, etc. - `scripts/docker-entrypoint.sh` — wait for DB → `migrate --noinput` → `collectstatic --noinput` → `exec gunicorn scha.wsgi:application`. - `scripts/validate-env.sh` — assert required prod env vars present. - `docker-compose.yml` (dev, bundled Postgres for local). - `docker-compose.prod.yml` — `web` service reads `DATABASE_URL`/`DB_*` pointing at the **shared external Postgres**; host port via `${WEB_PORT}`. Per `IMPLEMENTATION.md`, prod must NOT bundle its own DB. - `.env.example` (dev) and `.env.prod.example` (server template). ### A4. Gitea workflows (`.gitea/workflows/`) - `unittests.yml` — on push + PR to `master`: checkout, install uv, `uv sync --frozen`, `uv run python manage.py test`. - `deploy.yml` — `workflow_run` after **Unit Tests** completes on `master`, push events only: 1. `docker` job: build image + run containerized tests. 2. `deploy` job (needs docker): run `"$SERVER_INFRA_ROOT/scripts/deploy.sh" --app scha --env prod --ref "${{ gitea.event.workflow_run.head_sha }}"` on `self-hosted`. - Deploy never runs on PRs (separate workflow files, matching company_site). - (Optional) `ci.yml` for PR-only checks. ### A5. Docs - Update `README.md` with local dev (uv + docker), env matrix, and CI/deploy flow. --- ## Part B — Changes in `server-infra` (register scha) ### B1. Add `scha` to `app_catalog` (`inventory/group_vars/all.yml`) ```yaml scha: type: django repo: "{{ git_base_url }}/ai_ml_operations/scha.git" default_branch: master compose_file: docker-compose.prod.yml web_service: web migrate_cmd: "uv run python manage.py migrate --noinput" ``` ### B2. Add `scha` to `host_apps` on all app hosts Add to `inventory/host_vars/adama.yml` and `inventory/host_vars/roslin.yml` (ports MUST match across hosts for NPM balancing): ```yaml - { name: scha, env: prod, port: 8002 } # optional: - { name: scha, env: beta, port: 8012 } ``` (ai-server-4080 stays workload-free.) ### B3. Shared Postgres (10.0.0.230) - Create DB `scha` (+ `scha_beta` if beta) and grant `westfarn`. ### B4. Control-node secret - Create `~/Documents/secrets/scha/scha_prod.env` with `DATABASE_URL=postgres://westfarn:<pw>@10.0.0.230:5432/scha`, `DJANGO_ENV=prod`, `DJANGO_SECRET_KEY`, `DJANGO_ALLOWED_HOSTS`, `WEB_PORT=8002`, and the real Stripe/reCAPTCHA prod values. Mode 600, never committed. ### B5. NPM reverse proxy / load balancing - Point `schawheaton.aimloperations.com`, `schawheaton.com` (+ www) at backends `adama:8002` + `roslin:8002` (Advanced `upstream {}` block for active/active). --- ## Acceptance criteria - [ ] `scha` builds and runs via `docker compose` locally (dev) and `docker-compose.prod.yml` (prod against external Postgres). - [ ] No secrets in git; all secrets sourced from env; old committed keys rotated. - [ ] Push/PR to `master` runs unit tests in Gitea Actions. - [ ] Push to `master` (post-tests) auto-deploys scha prod to adama + roslin via `server-infra/scripts/deploy.sh`. - [ ] `scha` registered in `server-infra` `app_catalog` + `host_apps` on all webservers. - [ ] Shared Postgres DB + control-node secret provisioned. - [ ] Site reachable via NPM (active/active) on the schawheaton domains. ## References - `company_site` (reference impl): Dockerfile, docker-compose*.yml, scripts/, .gitea/workflows/{unittests,ci,deploy}.yml - `server-infra/IMPLEMENTATION.md` — architecture, ports, shared DB, "Required changes IN each app repo", item #10 - `server-infra/scripts/deploy.sh`, `playbooks/deploy-apps.yml`, `roles/app-deploy/tasks/{main,django}.yml`
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
twestfall westfarn (Ryan Westfall)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: ai_ml_operations/scha#19