ai_ml_operations/server-infra
2
0
Fork 0
Code Issues 3 Pull Requests Actions Packages Projects Releases Wiki Activity

Register url_shortening_service for django deploy (closes #22) #23

Merged
westfarn merged 1 commits from issue-22-register-url-shortening-service into master 2026-08-30 04:51:31 -07:00
Conversation 0 Commits 1 Files Changed 8
+35 -3
8 changed files with 35 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
IMPLEMENTATION.md
+15 -1
View File
@@ -215,6 +215,7 @@ After Docker install, re-SSH so the `docker` group membership takes effect.
| `chat_web_app` | node-static (CRA) | all webservers | beta + prod | active/active; built to `/var/www/<env>.chat.aimloperations/html`, served by web-static nginx | | `chat_web_app` | node-static (CRA) | all webservers | beta + prod | active/active; built to `/var/www/<env>.chat.aimloperations/html`, served by web-static nginx |
| `chat_backend` | django (docker) | all webservers | beta + prod | active/active behind NPM; Ollama `http://10.0.0.128:11434`; SearxNG `http://10.0.0.128:8088` (`SEARXNG_BASE_URL`) | | `chat_backend` | django (docker) | all webservers | beta + prod | active/active behind NPM; Ollama `http://10.0.0.128:11434`; SearxNG `http://10.0.0.128:8088` (`SEARXNG_BASE_URL`) |
| `monica_site` | django (docker) | all webservers | beta + prod | active/active behind NPM; no bundled Postgres (like `scha`); dj-queue **worker singleton on adama** only (`compose --profile worker`); Ollama social drafting via `10.0.0.128:11434` | | `monica_site` | django (docker) | all webservers | beta + prod | active/active behind NPM; no bundled Postgres (like `scha`); dj-queue **worker singleton on adama** only (`compose --profile worker`); Ollama social drafting via `10.0.0.128:11434` |
| `url_shortening_service` | django (docker) | all webservers | beta + prod | active/active behind NPM; no bundled Postgres; **no worker**. Two public hosts, same container: short domain (`GET /`, `GET /<code>` 302) and API host (`/api/links/`, Bearer required). |
Django apps use a **shared external Postgres** (via `DATABASE_URL` in each host's Django apps use a **shared external Postgres** (via `DATABASE_URL` in each host's
env file) so active/active replicas share one database. Beta and prod never share env file) so active/active replicas share one database. Beta and prod never share
@@ -242,6 +243,7 @@ future beta replica.
| scha | 8012 (*not deployed*) | 8002 | all webservers | | scha | 8012 (*not deployed*) | 8002 | all webservers |
| chat_backend | 8013 | 8003 | all webservers | | chat_backend | 8013 | 8003 | all webservers |
| monica_site | 8014 | 8004 | all webservers | | monica_site | 8014 | 8004 | all webservers |
| url_shortening_service | 8015 | 8005 | all webservers |
| dta_webapp (nginx) | 8081 | 8080 | all webservers | | dta_webapp (nginx) | 8081 | 8080 | all webservers |
| chat_web_app (nginx) | 8083 | 8082 | all webservers | | chat_web_app (nginx) | 8083 | 8082 | all webservers |
| SearxNG (LAN only) | — | **8088** | ai-server-4080 only (`searxng_stack`); not an NPM upstream | | SearxNG (LAN only) | — | **8088** | ai-server-4080 only (`searxng_stack`); not an NPM upstream |
@@ -305,6 +307,8 @@ point each domain at the backend(s):
| chat_web_app | `chat.aimloperations.com` (+ www); `beta.chat.aimloperations.com` | `adama:8082` / `8083` + same on roslin / starbuck / apollo / ai-server-4080 | | chat_web_app | `chat.aimloperations.com` (+ www); `beta.chat.aimloperations.com` | `adama:8082` / `8083` + same on roslin / starbuck / apollo / ai-server-4080 |
| chat_backend | `chatbackend.aimloperations.com`; `beta.chatbackend.aimloperations.com` | `adama:8003` / `8013` + same on roslin / starbuck / apollo / ai-server-4080 | | chat_backend | `chatbackend.aimloperations.com`; `beta.chatbackend.aimloperations.com` | `adama:8003` / `8013` + same on roslin / starbuck / apollo / ai-server-4080 |
| monica_site | `mkdrealtor.com` (+ www); `monica-preview.aimloperations.com` (beta) | `adama:8004` / `8014` + same on roslin / starbuck / apollo / ai-server-4080 | | monica_site | `mkdrealtor.com` (+ www); `monica-preview.aimloperations.com` (beta) | `adama:8004` / `8014` + same on roslin / starbuck / apollo / ai-server-4080 |
| url_shortening_service (short) | `aiml.pw` and/or `cidinn.li` (pick when DNS is ready); `short-beta.aimloperations.com` (beta). Proxy `/` + `/[a-z0-9]{4,8}` only — 404 `/api/`, `/admin/`, `/debug/` | `adama:8005` / `8015` + same on roslin / starbuck / apollo / ai-server-4080 |
| url_shortening_service (API) | `shortener.aimloperations.com`; `shortener-beta.aimloperations.com` (beta). Proxy `/api/` only — 404 `/admin/` | same ports as short host (one container) |
### Required changes IN each app repo (owned separately) ### Required changes IN each app repo (owned separately)
@@ -339,11 +343,20 @@ do not).
| chat_backend | beta | `chat_backend_beta` | `postgres://westfarn:<pw>@10.0.0.230:5432/chat_backend_beta` | | chat_backend | beta | `chat_backend_beta` | `postgres://westfarn:<pw>@10.0.0.230:5432/chat_backend_beta` |
| monica_site | prod | `monica_site` | `postgres://westfarn:<pw>@10.0.0.230:5432/monica_site` | | monica_site | prod | `monica_site` | `postgres://westfarn:<pw>@10.0.0.230:5432/monica_site` |
| monica_site | beta | `monica_site_beta` | `postgres://westfarn:<pw>@10.0.0.230:5432/monica_site_beta` | | monica_site | beta | `monica_site_beta` | `postgres://westfarn:<pw>@10.0.0.230:5432/monica_site_beta` |
| url_shortening_service | prod | `url_shortener` | `postgres://westfarn:<pw>@10.0.0.230:5432/url_shortener` |
| url_shortening_service | beta | `url_shortener_beta` | `postgres://westfarn:<pw>@10.0.0.230:5432/url_shortener_beta` |
Server prereqs on 10.0.0.230: create each DB + grant `westfarn`; Server prereqs on 10.0.0.230: create each DB + grant `westfarn`;
`listen_addresses` covers LAN; `pg_hba.conf` allows `10.0.0.0/24`; firewall opens `listen_addresses` covers LAN; `pg_hba.conf` allows `10.0.0.0/24`; firewall opens
5432 to `10.0.0.0/24` only. 5432 to `10.0.0.0/24` only.
`url_shortening_service` extra env (control-node secrets, not in git):
`SHORT_PUBLIC_HOSTS` / `SHORT_API_HOSTS` / `SHORT_ADMIN_HOSTS` (admin = `localhost,127.0.0.1` only),
`SHORTENER_API_TOKENS` (`monica:<token>`), `SHORT_ALLOWED_HOSTS`, `CLICK_IP_PEPPER`
(distinct from `DJANGO_SECRET_KEY`). `DJANGO_DEBUG=false` for prod and beta.
Caller `monica_site` uses `SHORTENER_BASE_URL=https://<api-host>` and Bearer mint;
do not mint via the short hostname.
### One-time host bootstrap (per target) ### One-time host bootstrap (per target)
- [x] Gitea SSH key: the `gitea-key` role (in `site.yml`) generates a key per - [x] Gitea SSH key: the `gitea-key` role (in `site.yml`) generates a key per
@@ -418,7 +431,8 @@ Store vault password for CI in a file readable only by the Act runner (e.g. `~/.
| 10c | Register + deploy `chat_backend` (django, ports 8003/8013) | Done (prod); beta ([#7](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/7), [chat_backend#26](https://git.aimloperations.com/ai_ml_operations/chat_backend/issues/26)) | | 10c | Register + deploy `chat_backend` (django, ports 8003/8013) | Done (prod); beta ([#7](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/7), [chat_backend#26](https://git.aimloperations.com/ai_ml_operations/chat_backend/issues/26)) |
| 10d | Register + deploy `monica_site` (django, ports 8004/8014) | Done ([#14](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/14)) | | 10d | Register + deploy `monica_site` (django, ports 8004/8014) | Done ([#14](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/14)) |
| 10e | Auto-start `monica_site` dj-queue worker on adama (`compose_profiles`) | Done ([#17](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/17)) | | 10e | Auto-start `monica_site` dj-queue worker on adama (`compose_profiles`) | Done ([#17](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/17)) |
| 10f | Add starbuck + apollo as app hosts (same workloads as roslin) | This PR ([#20](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/20)) | | 10f | Add starbuck + apollo as app hosts (same workloads as roslin) | Done ([#20](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/20)) |
| 10g | Register + deploy `url_shortening_service` (django, ports 8005/8015) | This PR ([#22](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/22)) |
| 11 | Gitea container registry (optional) | Future | | 11 | Gitea container registry (optional) | Future |
## Open Decisions ## Open Decisions
inventory/group_vars/all.yml
+8
View File
@@ -112,6 +112,14 @@ app_catalog:
migrate_cmd: "uv run python manage.py migrate --noinput" migrate_cmd: "uv run python manage.py migrate --noinput"
# dj-queue worker = compose profile `worker` (singleton). Deploy starts it # dj-queue worker = compose profile `worker` (singleton). Deploy starts it
# via host_apps.compose_profiles on adama only (see host_vars/adama.yml). # via host_apps.compose_profiles on adama only (see host_vars/adama.yml).
url_shortening_service:
type: django
repo: "{{ git_base_url }}/ai_ml_operations/url_shortening_service.git"
default_branch: master
compose_file: docker-compose.prod.yml
web_service: web
migrate_cmd: "uv run python manage.py migrate --noinput"
# No compose worker profile. Two public NPM hosts share one container/port.
# Deploy filter vars. CI passes these; manual runs may leave them undefined # Deploy filter vars. CI passes these; manual runs may leave them undefined
# to (re)deploy every app listed in the host's host_apps. # to (re)deploy every app listed in the host's host_apps.
inventory/host_vars/adama.yml
+2
View File
@@ -20,3 +20,5 @@ host_apps:
# compose_profiles: worker → dj-queue singleton (not on other app hosts). # compose_profiles: worker → dj-queue singleton (not on other app hosts).
- { name: monica_site, env: prod, port: 8004, compose_profiles: [worker] } - { name: monica_site, env: prod, port: 8004, compose_profiles: [worker] }
- { name: monica_site, env: beta, port: 8014, compose_profiles: [worker] } - { name: monica_site, env: beta, port: 8014, compose_profiles: [worker] }
- { name: url_shortening_service, env: prod, port: 8005 }
- { name: url_shortening_service, env: beta, port: 8015 }
inventory/host_vars/ai-server-4080.yml
+3 -1
View File
@@ -36,4 +36,6 @@ host_apps:
- { name: chat_backend, env: beta, port: 8013 } - { name: chat_backend, env: beta, port: 8013 }
# monica_site worker NOT here — adama host_apps sets compose_profiles: [worker]. # monica_site worker NOT here — adama host_apps sets compose_profiles: [worker].
- { name: monica_site, env: prod, port: 8004 } - { name: monica_site, env: prod, port: 8004 }
- { name: monica_site, env: beta, port: 8014 } - { name: monica_site, env: beta, port: 8014 }
- { name: url_shortening_service, env: prod, port: 8005 }
- { name: url_shortening_service, env: beta, port: 8015 }
inventory/host_vars/apollo.yml
+2
View File
@@ -16,3 +16,5 @@ host_apps:
# monica_site worker NOT here — adama host_apps sets compose_profiles: [worker]. # monica_site worker NOT here — adama host_apps sets compose_profiles: [worker].
- { name: monica_site, env: prod, port: 8004 } - { name: monica_site, env: prod, port: 8004 }
- { name: monica_site, env: beta, port: 8014 } - { name: monica_site, env: beta, port: 8014 }
- { name: url_shortening_service, env: prod, port: 8005 }
- { name: url_shortening_service, env: beta, port: 8015 }
inventory/host_vars/roslin.yml
+2
View File
@@ -16,3 +16,5 @@ host_apps:
# monica_site worker NOT here — adama host_apps sets compose_profiles: [worker]. # monica_site worker NOT here — adama host_apps sets compose_profiles: [worker].
- { name: monica_site, env: prod, port: 8004 } - { name: monica_site, env: prod, port: 8004 }
- { name: monica_site, env: beta, port: 8014 } - { name: monica_site, env: beta, port: 8014 }
- { name: url_shortening_service, env: prod, port: 8005 }
- { name: url_shortening_service, env: beta, port: 8015 }
inventory/host_vars/starbuck.yml
+2
View File
@@ -16,3 +16,5 @@ host_apps:
# monica_site worker NOT here — adama host_apps sets compose_profiles: [worker]. # monica_site worker NOT here — adama host_apps sets compose_profiles: [worker].
- { name: monica_site, env: prod, port: 8004 } - { name: monica_site, env: prod, port: 8004 }
- { name: monica_site, env: beta, port: 8014 } - { name: monica_site, env: beta, port: 8014 }
- { name: url_shortening_service, env: prod, port: 8005 }
- { name: url_shortening_service, env: beta, port: 8015 }
scripts/deploy.sh
+1 -1
View File
@@ -17,7 +17,7 @@ Deploy applications with deploy-apps.yml.
HOST Optional. Limit to one host: adama, roslin, starbuck, apollo, or ai-server-4080. HOST Optional. Limit to one host: adama, roslin, starbuck, apollo, or ai-server-4080.
Options: Options:
--app NAME App to deploy (company_site, dta_service, dta_webapp, scha, chat_web_app, chat_backend, monica_site) --app NAME App to deploy (company_site, dta_service, dta_webapp, scha, chat_web_app, chat_backend, monica_site, url_shortening_service)
--env ENV Environment: beta or prod --env ENV Environment: beta or prod
--ref REF Git ref/sha to deploy (default: master) --ref REF Git ref/sha to deploy (default: master)
--check Dry run --check Dry run