Dockerize chat_backend + Gitea CI/CD auto-deploy via server-infra (multi-host + Ollama LAN) #6

Closed
opened 2026-07-14 03:11:33 -07:00 by westfarn · 1 comment
Owner

Summary

Dockerize chat_backend and wire it into the same automated deploy pipeline used by dta_service / company_site. On merge/push to the default branch, Gitea Actions runs tests, builds the image, then calls server-infra/scripts/deploy.sh to deploy active/active to adama + roslin + ai-server-4080 behind NPM, against shared external Postgres.

Companion frontend work: track in chat_web_app (mirror dta_webappnode-static, not Docker).

This mirrors server-infra/IMPLEMENTATION.md and the reference impl in dta_service / company_site.


Current state

  • Django app under llm_be/ (project layout similar to other Django services).
  • Dependencies via requirements.txt / requirements.dev (no uv, no prod compose, no Gitea workflows).
  • No Dockerfile, no docker-compose.prod.yml, no entrypoint/validate-env scripts.
  • Ollama hardcoded to loopback in llm_be/chat_backend/client.py:
    • ollama.Client(host="http://127.0.0.1:11434")
  • Langchain OllamaLLM(...) call sites do not set base_url → also default to localhost. Broken once the app runs in Docker / on another host.

Target architecture (match dta_service)

  • Django app = one docker compose project per env: project name chat_backend_<env>, host port from host_apps.
  • Runs active/active on adama + roslin + ai-server-4080; NPM balances all three upstreams.
  • Shared external Postgres at 10.0.0.230:5432 (each app+env its own DB; beta and prod MUST NOT share a DB).
  • Secrets in control-node env files: ~/Documents/secrets/chat_backend/chat_backend_<env>.env → pushed to /opt/apps/env/chat_backend_<env>.env.
  • CI: tests on push/PR; on green push, deploy via server-infra/scripts/deploy.sh --app chat_backend --env prod --ref <sha> (and beta if enabled).

Proposed ports (no conflict with current catalog)

App prod beta
company_site 8000 8010 (reserved)
dta_service 8001 8011
scha 8002 8012 (reserved)
chat_backend (new) 8003 8013
dta_webapp (nginx) 8080 8081
chat_web_app (nginx) 8082 8083

Part A — Changes in THIS repo (chat_backend)

A1. Packaging → uv (match dta_service / company_site)

  • Add pyproject.toml + uv.lock (replace/retire plain requirements.txt for deploy).
  • Prod deps must include: gunicorn, psycopg2-binary, whitenoise (or static strategy already used), plus existing Django/LangChain/ollama stack.
  • Rationale: app_catalog.migrate_cmd is uv run python manage.py migrate --noinput; Docker build uses uv sync --frozen --no-dev.

A2. Environment-driven settings

  • DJANGO_ENV / DJANGO_SECRET_KEY / DJANGO_DEBUG / DJANGO_ALLOWED_HOSTS / DJANGO_CSRF_TRUSTED_ORIGINS
  • DATABASE_URL → shared Postgres (SQLite only for pure local if needed)
  • Move any hardcoded secrets to env
  • Preserve production domains for chat backend / CSRF as currently used (e.g. chatbackend.aimloperations.com)

A3. Ollama: stop using localhost

Ollama lives on the GPU host (ai-server-4080 = 10.0.0.128). All app hosts (including containers on other VMs) must reach it over the LAN.

  • Introduce env var, e.g. OLLAMA_BASE_URL / OLLAMA_HOST (recommended default for prod/beta: http://10.0.0.128:11434).
  • Update all Ollama clients / OllamaLLM / embeddings call sites to use that URL (not only client.py — also services that construct OllamaLLM(...) without base_url).
  • Dev may keep http://127.0.0.1:11434 when Ollama is local.
  • Document that firewall / Ollama listen address must allow 10.0.0.0/2411434 on ai-server-4080.

A4. Docker assets (copy/adapt from dta_service / company_site)

  • Dockerfile, .dockerignore
  • scripts/docker-entrypoint.sh (wait DB → migrate → collectstatic → gunicorn)
  • scripts/validate-env.sh
  • docker-compose.yml (dev)
  • docker-compose.prod.ymlno bundled DB; host port ${WEB_PORT}
  • .env.example / .env.prod.example including OLLAMA_* + WEB_PORT=8003

A5. Gitea workflows (.gitea/workflows/)

Mirror dta_service / company_site:

  • Unit tests on push + PR
  • Deploy after green tests on default branch (push only) → server-infra/scripts/deploy.sh --app chat_backend --env prod --ref <sha>
  • Optional separate beta deploy workflow if beta is enabled

Part B — Changes in server-infra

B1. app_catalog (inventory/group_vars/all.yml)

chat_backend:
  type: django
  repo: "{{ git_base_url }}/ai_ml_operations/chat_backend.git"
  default_branch: master   # confirm actual default branch
  compose_file: docker-compose.prod.yml
  web_service: web
  migrate_cmd: "uv run python manage.py migrate --noinput"

B2. host_apps on all deploy hosts (ports must match)

Add to adama.yml, roslin.yml, and ai-server-4080.yml:

- { name: chat_backend, env: prod, port: 8003 }
# optional: - { name: chat_backend, env: beta, port: 8013 }

B3. Shared Postgres

  • Create DB chat_backend (+ chat_backend_beta if beta) and grant westfarn.

B4. Control-node secret

  • ~/Documents/secrets/chat_backend/chat_backend_prod.env with DATABASE_URL, Django vars, WEB_PORT=8003, OLLAMA_BASE_URL=http://10.0.0.128:11434, mode 600.

B5. NPM

  • Point chatbackend.aimloperations.com (and any aliases) at adama:8003 + roslin:8003 + ai-server-4080:8003.

B6. Docs

  • Update IMPLEMENTATION.md apps/ports/Postgres/NPM tables for chat_backend (+ companion chat_web_app when that ticket lands).

Acceptance criteria

  • chat_backend builds/runs via docker compose locally and docker-compose.prod.yml against external Postgres
  • All Ollama traffic uses OLLAMA_BASE_URL (prod/beta → http://10.0.0.128:11434), never hardcoded localhost in deployed code
  • Gitea unit tests + auto-deploy via server-infra/scripts/deploy.sh
  • Registered in app_catalog + host_apps on adama, roslin, and ai-server-4080
  • Shared Postgres DB + control-node secret + NPM active/active

References

  • dta_service — Django docker + Gitea deploy workflows (pattern to copy)
  • company_site — settings/env/Dockerfile reference
  • server-infraIMPLEMENTATION.md, scripts/deploy.sh, roles/app-deploy
  • Companion: chat_web_app ticket (node-static like dta_webapp)
## Summary Dockerize `chat_backend` and wire it into the same automated deploy pipeline used by `dta_service` / `company_site`. On merge/push to the default branch, Gitea Actions runs tests, builds the image, then calls `server-infra/scripts/deploy.sh` to deploy **active/active** to **adama + roslin + ai-server-4080** behind NPM, against shared external Postgres. Companion frontend work: track in `chat_web_app` (mirror `dta_webapp` — **node-static**, not Docker). This mirrors `server-infra/IMPLEMENTATION.md` and the reference impl in `dta_service` / `company_site`. --- ## Current state * Django app under `llm_be/` (project layout similar to other Django services). * Dependencies via `requirements.txt` / `requirements.dev` (no `uv`, no prod compose, no Gitea workflows). * No `Dockerfile`, no `docker-compose.prod.yml`, no entrypoint/validate-env scripts. * Ollama hardcoded to loopback in `llm_be/chat_backend/client.py`: * `ollama.Client(host="http://127.0.0.1:11434")` * Langchain `OllamaLLM(...)` call sites do not set `base_url` → also default to localhost. **Broken once the app runs in Docker / on another host.** --- ## Target architecture (match `dta_service`) * Django app = one docker compose project per env: project name `chat_backend_<env>`, host port from `host_apps`. * Runs active/active on **adama + roslin + ai-server-4080**; NPM balances all three upstreams. * Shared **external Postgres** at `10.0.0.230:5432` (each app+env its own DB; beta and prod MUST NOT share a DB). * Secrets in control-node env files: `~/Documents/secrets/chat_backend/chat_backend_<env>.env` → pushed to `/opt/apps/env/chat_backend_<env>.env`. * CI: tests on push/PR; on green push, deploy via `server-infra/scripts/deploy.sh --app chat_backend --env prod --ref <sha>` (and beta if enabled). ### Proposed ports (no conflict with current catalog) | App | prod | beta | |-----|------|------| | company_site | 8000 | 8010 (reserved) | | dta_service | 8001 | 8011 | | scha | 8002 | 8012 (reserved) | | **chat_backend (new)** | **8003** | **8013** | | dta_webapp (nginx) | 8080 | 8081 | | chat_web_app (nginx) | 8082 | 8083 | --- ## Part A — Changes in THIS repo (`chat_backend`) ### A1. Packaging → `uv` (match dta_service / company_site) * Add `pyproject.toml` + `uv.lock` (replace/retire plain `requirements.txt` for deploy). * Prod deps must include: gunicorn, psycopg2-binary, whitenoise (or static strategy already used), plus existing Django/LangChain/ollama stack. * Rationale: `app_catalog.migrate_cmd` is `uv run python manage.py migrate --noinput`; Docker build uses `uv sync --frozen --no-dev`. ### A2. Environment-driven settings * `DJANGO_ENV` / `DJANGO_SECRET_KEY` / `DJANGO_DEBUG` / `DJANGO_ALLOWED_HOSTS` / `DJANGO_CSRF_TRUSTED_ORIGINS` * `DATABASE_URL` → shared Postgres (SQLite only for pure local if needed) * Move any hardcoded secrets to env * Preserve production domains for chat backend / CSRF as currently used (e.g. `chatbackend.aimloperations.com`) ### A3. Ollama: stop using localhost Ollama lives on the GPU host (**ai-server-4080 = `10.0.0.128`**). All app hosts (including containers on other VMs) must reach it over the LAN. * Introduce env var, e.g. `OLLAMA_BASE_URL` / `OLLAMA_HOST` (recommended default for prod/beta: `http://10.0.0.128:11434`). * Update **all** Ollama clients / `OllamaLLM` / embeddings call sites to use that URL (not only `client.py` — also services that construct `OllamaLLM(...)` without `base_url`). * Dev may keep `http://127.0.0.1:11434` when Ollama is local. * Document that firewall / Ollama listen address must allow `10.0.0.0/24` → `11434` on ai-server-4080. ### A4. Docker assets (copy/adapt from `dta_service` / `company_site`) * `Dockerfile`, `.dockerignore` * `scripts/docker-entrypoint.sh` (wait DB → migrate → collectstatic → gunicorn) * `scripts/validate-env.sh` * `docker-compose.yml` (dev) * `docker-compose.prod.yml` — **no bundled DB**; host port `${WEB_PORT}` * `.env.example` / `.env.prod.example` including `OLLAMA_*` + `WEB_PORT=8003` ### A5. Gitea workflows (`.gitea/workflows/`) Mirror `dta_service` / `company_site`: * Unit tests on push + PR * Deploy after green tests on default branch (push only) → `server-infra/scripts/deploy.sh --app chat_backend --env prod --ref <sha>` * Optional separate beta deploy workflow if beta is enabled --- ## Part B — Changes in `server-infra` ### B1. `app_catalog` (`inventory/group_vars/all.yml`) ```yaml chat_backend: type: django repo: "{{ git_base_url }}/ai_ml_operations/chat_backend.git" default_branch: master # confirm actual default branch compose_file: docker-compose.prod.yml web_service: web migrate_cmd: "uv run python manage.py migrate --noinput" ``` ### B2. `host_apps` on **all** deploy hosts (ports must match) Add to `adama.yml`, `roslin.yml`, **and** `ai-server-4080.yml`: ```yaml - { name: chat_backend, env: prod, port: 8003 } # optional: - { name: chat_backend, env: beta, port: 8013 } ``` ### B3. Shared Postgres * Create DB `chat_backend` (+ `chat_backend_beta` if beta) and grant `westfarn`. ### B4. Control-node secret * `~/Documents/secrets/chat_backend/chat_backend_prod.env` with `DATABASE_URL`, Django vars, `WEB_PORT=8003`, **`OLLAMA_BASE_URL=http://10.0.0.128:11434`**, mode 600. ### B5. NPM * Point `chatbackend.aimloperations.com` (and any aliases) at `adama:8003` + `roslin:8003` + `ai-server-4080:8003`. ### B6. Docs * Update `IMPLEMENTATION.md` apps/ports/Postgres/NPM tables for `chat_backend` (+ companion `chat_web_app` when that ticket lands). --- ## Acceptance criteria - [ ] `chat_backend` builds/runs via docker compose locally and `docker-compose.prod.yml` against external Postgres - [ ] All Ollama traffic uses `OLLAMA_BASE_URL` (prod/beta → `http://10.0.0.128:11434`), never hardcoded localhost in deployed code - [ ] Gitea unit tests + auto-deploy via `server-infra/scripts/deploy.sh` - [ ] Registered in `app_catalog` + `host_apps` on **adama, roslin, and ai-server-4080** - [ ] Shared Postgres DB + control-node secret + NPM active/active ## References * `dta_service` — Django docker + Gitea deploy workflows (pattern to copy) * `company_site` — settings/env/Dockerfile reference * `server-infra` — `IMPLEMENTATION.md`, `scripts/deploy.sh`, `roles/app-deploy` * Companion: `chat_web_app` ticket (node-static like `dta_webapp`)
westfarn self-assigned this 2026-07-14 03:11:33 -07:00
Author
Owner

Companion frontend ticket: ai_ml_operations/chat_web_app#13

(chat_web_app = node-static like dta_webapp, not Docker.)

Companion frontend ticket: https://git.aimloperations.com/ai_ml_operations/chat_web_app/issues/13 (`chat_web_app` = node-static like `dta_webapp`, not Docker.)
Sign in to join this conversation.
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: ai_ml_operations/chat_backend#6