Compare commits
2
Commits
master
..
853eff020d
| Author | SHA1 | Date | |
|---|---|---|---|
 westfarn
|
853eff020d | ||
|
westfarn
|
9a0cbf80d9 |
No files matched your search
+45
-278
@@ -10,8 +10,6 @@ flowchart TB
|
||||
Control1["ai-server-4080\n(control node)"]
|
||||
Control1 -->|ansible-playbook site.yml| Adama
|
||||
Control1 -->|ansible-playbook site.yml| Roslin
|
||||
Control1 -->|ansible-playbook site.yml| Starbuck
|
||||
Control1 -->|ansible-playbook site.yml| Apollo
|
||||
end
|
||||
|
||||
subgraph cicd ["CI/CD (every merge to master)"]
|
||||
@@ -21,8 +19,6 @@ flowchart TB
|
||||
Deploy --> AnsibleDeploy["ansible-playbook deploy-apps.yml"]
|
||||
AnsibleDeploy --> Adama2["adama"]
|
||||
AnsibleDeploy --> Roslin2["roslin"]
|
||||
AnsibleDeploy --> Starbuck2["starbuck"]
|
||||
AnsibleDeploy --> Apollo2["apollo"]
|
||||
end
|
||||
```
|
||||
|
||||
@@ -30,7 +26,6 @@ flowchart TB
|
||||
|----------|------|----------|---------------|
|
||||
| **Provision** | New VM, OS change, firewall, Docker install | `site.yml` | ai-server-4080 — run manually |
|
||||
| **Deploy** | Green unit tests on `master` | `deploy-apps.yml` | Gitea Act runner on ai-server-4080 |
|
||||
| **Stop** | Take a compose app down | `stop-apps.yml` (`scripts/stop.sh`) | ai-server-4080 — run manually |
|
||||
|
||||
Both pipelines share the same inventory (`inventory/hosts.yml`).
|
||||
|
||||
@@ -40,8 +35,6 @@ Both pipelines share the same inventory (`inventory/hosts.yml`).
|
||||
|------|-----|------|
|
||||
| adama | 10.0.0.77 | Ubuntu Server VM (Proxmox) — app host |
|
||||
| roslin | 10.0.0.176 | Ubuntu Server VM (Proxmox) — app host |
|
||||
| starbuck | 10.0.0.44 | Ubuntu Server VM (Proxmox) — app host |
|
||||
| apollo | 10.0.0.7 | Ubuntu Server VM (Proxmox) — app host |
|
||||
| ai-server-4080 | 10.0.0.128 | Control node + Gitea act runner + Ollama + SearxNG + observability; also runs app replicas |
|
||||
|
||||
Hostname on this machine: `ryan-development-1`
|
||||
@@ -58,17 +51,13 @@ server-infra/
|
||||
│ ├── hosts.yml
|
||||
│ ├── group_vars/
|
||||
│ │ └── all.yml # vars + app_catalog
|
||||
│ ├── host_vars/
|
||||
│ │ ├── adama.yml # host_apps (django + static; monica worker)
|
||||
│ │ ├── roslin.yml # host_apps (mirrors adama, no worker)
|
||||
│ │ ├── starbuck.yml # host_apps (mirrors roslin)
|
||||
│ │ ├── apollo.yml # host_apps (mirrors roslin)
|
||||
│ │ └── ai-server-4080.yml # control node / act runner / SearxNG / observability
|
||||
│ └── wordpress_sites.yml # WordPress NFS mounts + generated compose inputs
|
||||
│ └── host_vars/
|
||||
│ ├── adama.yml # host_apps (django + dta_webapp)
|
||||
│ ├── roslin.yml # host_apps (mirrors adama)
|
||||
│ └── ai-server-4080.yml # control node / act runner / SearxNG / observability
|
||||
├── playbooks/
|
||||
│ ├── site.yml # Phase 1: provision
|
||||
│ ├── deploy-apps.yml # Phase 2: CI deploy
|
||||
│ └── stop-apps.yml # compose down one app+env
|
||||
│ └── deploy-apps.yml # Phase 2: CI deploy
|
||||
├── roles/
|
||||
│ ├── common/ # Base packages
|
||||
│ ├── ufw/ # Firewall
|
||||
@@ -80,12 +69,10 @@ server-infra/
|
||||
│ ├── searxng/ # SearxNG JSON API for chat_backend (#10)
|
||||
│ ├── alloy/ # log/metrics shipper
|
||||
│ ├── app-deploy/ # django (docker) + node-static deploy
|
||||
│ ├── web-static/ # nginx container serving /var/www builds
|
||||
│ └── wordpress/ # NFS wp-content + compose rendered per site
|
||||
│ └── web-static/ # nginx container serving /var/www builds
|
||||
└── scripts/
|
||||
├── provision.sh # Wrapper with --limit support
|
||||
├── deploy.sh # Wrapper for deploy playbook
|
||||
└── stop.sh # Wrapper for stop playbook (compose down)
|
||||
└── deploy.sh # Wrapper for deploy playbook
|
||||
```
|
||||
|
||||
## Prerequisites (One-Time Bootstrap)
|
||||
@@ -97,15 +84,11 @@ Ansible needs SSH + sudo on each target before playbooks work.
|
||||
```bash
|
||||
ssh-copy-id westfarn@10.0.0.77
|
||||
ssh-copy-id westfarn@10.0.0.176
|
||||
ssh-copy-id westfarn@10.0.0.44
|
||||
ssh-copy-id westfarn@10.0.0.7
|
||||
```
|
||||
3. Confirm passwordless SSH:
|
||||
```bash
|
||||
ssh westfarn@10.0.0.77
|
||||
ssh westfarn@10.0.0.176
|
||||
ssh westfarn@10.0.0.44
|
||||
ssh westfarn@10.0.0.7
|
||||
```
|
||||
4. **First-time only** — grant passwordless sudo on each new host before the first
|
||||
`provision.sh` run. Ubuntu 26.04 ships `sudo-rs` by default; Ansible's
|
||||
@@ -159,8 +142,6 @@ New hosts need the one-time passwordless sudo bootstrap in
|
||||
|
||||
# Same for other hosts
|
||||
./scripts/provision.sh roslin
|
||||
./scripts/provision.sh starbuck
|
||||
./scripts/provision.sh apollo
|
||||
./scripts/provision.sh ai-server-4080
|
||||
```
|
||||
|
||||
@@ -175,22 +156,8 @@ New hosts need the one-time passwordless sudo bootstrap in
|
||||
```bash
|
||||
./scripts/deploy.sh adama
|
||||
./scripts/deploy.sh --check roslin
|
||||
./scripts/deploy.sh starbuck
|
||||
./scripts/deploy.sh apollo
|
||||
```
|
||||
|
||||
### Stop a compose app
|
||||
|
||||
```bash
|
||||
./scripts/stop.sh --app print_forge --env prod
|
||||
./scripts/stop.sh --app print_forge --env beta
|
||||
./scripts/stop.sh adama --app chat_backend --env beta --check
|
||||
```
|
||||
|
||||
`stop.sh` is `docker compose down` for one `--app` + `--env` (no `-v`). Checkouts and
|
||||
secrets stay. Next `deploy.sh` of that app+env starts it again unless `host_apps`
|
||||
sets `enabled: false`. Node-static apps have no compose project; stop.sh skips them.
|
||||
|
||||
Under the hood, scripts pass `--limit <hostname>` to `ansible-playbook`.
|
||||
|
||||
## Phase 1: Provision (`site.yml`)
|
||||
@@ -199,7 +166,7 @@ Applies roles in order to the `webservers` group:
|
||||
|
||||
| Role | Purpose |
|
||||
|------|---------|
|
||||
| `common` | apt update, git, python3, pip, curl, ca-certificates, nfs-common |
|
||||
| `common` | apt update, git, python3, pip, curl, ca-certificates |
|
||||
| `ufw` | Firewall: SSH from LAN only, HTTP/HTTPS public |
|
||||
| `docker` | Docker CE, compose plugin, add `westfarn` to `docker` group |
|
||||
| `nodejs` | Node.js + npm + npx (NodeSource) for `dta_webapp` builds |
|
||||
@@ -225,23 +192,13 @@ After Docker install, re-SSH so the `docker` group membership takes effect.
|
||||
|
||||
| App | Type | Hosts | Envs | Notes |
|
||||
|-----|------|-------|------|-------|
|
||||
| `company_site` | django (docker) | all webservers | prod | active/active behind NPM; beta port reserved |
|
||||
| `dta_service` | django (docker) | all webservers | beta + prod | active/active behind NPM |
|
||||
| `dta_webapp` | node/vite static | all webservers | beta + prod | active/active; built to `/var/www/<env>.realpath.app/html`, served by web-static nginx |
|
||||
| `dta_blog` | node-static (Python SSG) | all webservers | beta + prod | active/active; built to `/var/www/<env>.blog.realpath.app/html`; **own hosts** (`blog.realpath.app` / `beta.blog.realpath.app`), not `realpath.app/blog`; Tianji ids baked at build |
|
||||
| `scha` | django (docker) | all webservers | prod | active/active behind NPM; beta port reserved |
|
||||
| `chat_web_app` | node-static (CRA) | all webservers | beta + prod | active/active; built to `/var/www/<env>.chat.aimloperations/html`, served by web-static nginx |
|
||||
| `chat_backend` | django (docker) | all webservers | beta + prod | active/active behind NPM; Ollama `http://10.0.0.128:11434`; SearxNG `http://10.0.0.128:8088` (`SEARXNG_BASE_URL`) |
|
||||
| `monica_site` | django (docker) | all webservers | beta + prod | active/active behind NPM; no bundled Postgres (like `scha`); dj-queue **worker singleton on adama** only (`compose --profile worker`); Ollama social drafting via `10.0.0.128:11434` |
|
||||
| `url_shortening_service` | django (docker) | all webservers | beta + prod | active/active behind NPM; no bundled Postgres; **no worker**. Two public hosts, same container: short domain (`GET /`, `GET /<code>` 302) and API host (`/api/links/`, Bearer required). |
|
||||
| `college_craft` | django (docker) | all webservers | beta + prod | active/active behind NPM; no bundled Postgres (like `scha` / `monica_site`); dj-queue **worker singleton on adama** only (`compose --profile worker`); Ollama social drafting via `10.0.0.128:11434`; Nominatim `http://10.0.0.128:8089`; prod `SITE_UNDER_CONSTRUCTION=true` until launch |
|
||||
| `print_forge` | django (docker) | all webservers | beta + prod | **stopped** (`enabled: false`, [#33](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/33)). Ports reserved. Re-enable + `deploy.sh` to bring back. Worker was adama-only. |
|
||||
| `abc_be` | django (docker) | all webservers | **beta only** | AI Benefits Coach API; no bundled Postgres; Ollama `http://10.0.0.128:11434`; shares DB `abc_be_beta` with `abc_worker` |
|
||||
| `abc_worker` | django-type compose (FastAPI) | all webservers | **beta only** | same Postgres as `abc_be`; `migrate_cmd: true`; WS on host port 8018 |
|
||||
| `abc_voice` | django-type compose (FastAPI) | all webservers | **beta only** | Same repo as `abc_worker`, compose `docker-compose.voice.yml`, join API on host port 8020. One uvicorn worker. |
|
||||
| `abc_fe` | node-static (Vite) | all webservers | **beta only** | built to `/var/www/<env>.abc.aimloperations/html`; no prod `host_apps` row |
|
||||
| `livekit` | django-type compose (LiveKit SFU) | all webservers | **beta only** | Docker like `abc_worker`. Host network `:7880`. **Do not NPM-balance** until Redis — pick one upstream (adama). Router DNAT UDP 3478 + 50000–60000 and TCP 7881 to that host. Secrets `~/Documents/secrets/livekit/livekit_beta.env` |
|
||||
| `disney_hotel_service` | django (docker) | all webservers | beta + prod | active/active behind NPM; no bundled Postgres. Alert worker (`db_worker`) **singleton on adama** only (`compose --profile worker`). Repo `DisneyHotel/disney_hotel_service` (local folder `disney_reservation_finder`). Reads `POSTGRES_*`, not `DATABASE_URL`. |
|
||||
| `company_site` | django (docker) | adama + roslin (+ ai-server-4080) | prod | active/active behind NPM; beta port reserved |
|
||||
| `dta_service` | django (docker) | adama + roslin + ai-server-4080 | beta + prod | active/active behind NPM |
|
||||
| `dta_webapp` | node/vite static | adama + roslin (+ ai-server-4080) | beta + prod | active/active; built to `/var/www/<env>.realpath.app/html`, served by web-static nginx |
|
||||
| `scha` | django (docker) | adama + roslin + ai-server-4080 | prod | active/active behind NPM; beta port reserved |
|
||||
| `chat_web_app` | node-static (CRA) | adama + roslin + ai-server-4080 | beta + prod | active/active; built to `/var/www/<env>.chat.aimloperations/html`, served by web-static nginx |
|
||||
| `chat_backend` | django (docker) | adama + roslin + ai-server-4080 | beta + prod | active/active behind NPM; Ollama `http://10.0.0.128:11434`; SearxNG `http://10.0.0.128:8088` (`SEARXNG_BASE_URL`) |
|
||||
| `monica_site` | django (docker) | adama + roslin + ai-server-4080 | beta + prod | active/active behind NPM; no bundled Postgres (like `scha`); dj-queue **worker singleton on adama** only (`compose --profile worker`); Ollama social drafting via `10.0.0.128:11434` |
|
||||
|
||||
Django apps use a **shared external Postgres** (via `DATABASE_URL` in each host's
|
||||
env file) so active/active replicas share one database. Beta and prod never share
|
||||
@@ -252,13 +209,9 @@ a DB.
|
||||
- `app_catalog` (`group_vars/all.yml`) — how each app is built (repo, type, compose file, migrate cmd).
|
||||
- `host_apps` (`host_vars/<host>.yml`) — which app+env+port runs on that host.
|
||||
Optional `compose_profiles: [worker]` activates docker compose profiles on that
|
||||
host only (used for `monica_site` / `college_craft` / `print_forge` dj-queue
|
||||
and `disney_hotel_service` alert-worker singleton on adama).
|
||||
Optional `enabled: false` skips deploy (CI included) but keeps the row for ports
|
||||
and `stop.sh`. Omit the key (or `true`) to deploy as before. Do not use Jinja
|
||||
`rejectattr('enabled')` — missing key is an error, not "enabled".
|
||||
host only (used for `monica_site` dj-queue singleton on adama).
|
||||
- Django app = one compose project per env: project name `<app>_<env>`, host port from `host_apps`.
|
||||
Ports match across app hosts so NPM can balance `adama:PORT` + `roslin:PORT` + `starbuck:PORT` + `apollo:PORT`.
|
||||
Ports match across adama/roslin so NPM can balance `adama:PORT` + `roslin:PORT`.
|
||||
|
||||
### Ports
|
||||
|
||||
@@ -268,31 +221,20 @@ future beta replica.
|
||||
|
||||
| App | beta | prod | Deployed on |
|
||||
|-----|------|------|-------------|
|
||||
| company_site | 8010 (*not deployed*) | 8000 | all webservers |
|
||||
| dta_service | 8011 | 8001 | all webservers |
|
||||
| scha | 8012 (*not deployed*) | 8002 | all webservers |
|
||||
| chat_backend | 8013 | 8003 | all webservers |
|
||||
| monica_site | 8014 | 8004 | all webservers |
|
||||
| url_shortening_service | 8015 | 8005 | all webservers |
|
||||
| college_craft | 8016 | 8006 | all webservers |
|
||||
| print_forge | **8019** (*stopped*) | **8007** (*stopped*) | all webservers (`enabled: false`) |
|
||||
| abc_be | **8017** | 8009 (*not deployed*) | all webservers |
|
||||
| abc_worker | **8018** | 8008 (*not deployed*) | all webservers |
|
||||
| abc_voice | **8020** | — | all webservers |
|
||||
| dta_webapp (nginx) | 8081 | 8080 | all webservers |
|
||||
| chat_web_app (nginx) | 8083 | 8082 | all webservers |
|
||||
| abc_fe (nginx) | **8085** | 8084 (*not deployed*) | all webservers |
|
||||
| dta_blog (nginx) | **8087** | **8086** | all webservers |
|
||||
| livekit | **7880** | — | all webservers (signaling). UDP 3478 + 50000–60000 + TCP 7881 via router DNAT to the NPM upstream host |
|
||||
| disney_hotel_service | **8024** | **8023** | all webservers. **8021/8022** are WordPress `relawfirm` — do not reuse |
|
||||
| company_site | 8010 (*not deployed*) | 8000 | adama, roslin, ai-server-4080 |
|
||||
| dta_service | 8011 | 8001 | adama, roslin, ai-server-4080 |
|
||||
| scha | 8012 (*not deployed*) | 8002 | adama, roslin, ai-server-4080 |
|
||||
| chat_backend | 8013 | 8003 | adama, roslin, ai-server-4080 |
|
||||
| monica_site | 8014 | 8004 | adama, roslin, ai-server-4080 |
|
||||
| dta_webapp (nginx) | 8081 | 8080 | adama, roslin, ai-server-4080 |
|
||||
| chat_web_app (nginx) | 8083 | 8082 | adama, roslin, ai-server-4080 |
|
||||
| SearxNG (LAN only) | — | **8088** | ai-server-4080 only (`searxng_stack`); not an NPM upstream |
|
||||
|
||||
Host-local services on ai-server-4080 (not balanced by NPM):
|
||||
|
||||
| Service | Port | Notes |
|
||||
|---------|------|-------|
|
||||
| Ollama | 11434 | Not Ansible-managed today; GPU host (`monica_site` / `college_craft` social drafting) |
|
||||
| Nominatim | 8089 | Not Ansible-managed today; LAN address autocomplete for `college_craft` / `print_forge` |
|
||||
| Ollama | 11434 | Not Ansible-managed today; GPU host |
|
||||
| SearxNG | 8088 | `roles/searxng` (#10); JSON API for chat_backend grounded search |
|
||||
| Loki | 3100 | `roles/observability` |
|
||||
| Prometheus | 9090 | `roles/observability` |
|
||||
@@ -310,16 +252,7 @@ chat_backend secrets must use `SEARXNG_BASE_URL=http://10.0.0.128:8088`.
|
||||
--app company_site --env prod --ref "${{ gitea.sha }}"
|
||||
```
|
||||
3. `deploy-apps.yml` runs against `webservers`; each host deploys only the
|
||||
matching **enabled** app+env from its `host_apps` (`enabled: false` is skipped).
|
||||
|
||||
To stop a compose app without deleting inventory:
|
||||
|
||||
```bash
|
||||
./scripts/stop.sh --app print_forge --env prod
|
||||
```
|
||||
|
||||
Set `enabled: false` on that `host_apps` row so the next CI deploy cannot start it
|
||||
again. `stop.sh` still matches disabled rows.
|
||||
matching app+env from its `host_apps`.
|
||||
|
||||
### `app-deploy` role behavior
|
||||
|
||||
@@ -327,14 +260,11 @@ again. `stop.sh` still matches disabled rows.
|
||||
to host `{{ apps_env_dir }}` → git checkout at ref → copy `.env` into checkout →
|
||||
`docker compose build` → `up -d` (with `COMPOSE_PROFILES` from optional
|
||||
`host_apps.compose_profiles`) → migrate (run once, shared DB).
|
||||
**`monica_site` / `college_craft` / `print_forge` / `disney_hotel_service` worker:**
|
||||
adama `host_apps` sets `compose_profiles: [worker]` so deploy starts the worker
|
||||
with web. Other hosts omit profiles (web only).
|
||||
**`monica_site` worker:** adama `host_apps` sets `compose_profiles: [worker]` so
|
||||
deploy starts dj-queue with web. Other hosts omit profiles (web only).
|
||||
- **node-static**: git checkout at ref → `npm ci` → `npm run build:<env>`
|
||||
(writes to the app's `webroot_pattern`, e.g. `/var/www/{env}.realpath.app/html`,
|
||||
`/var/www/{env}.blog.realpath.app/html`, or `/var/www/{env}.chat.aimloperations/html`).
|
||||
Optional catalog `error_page_404` (used by `dta_blog`) serves that file instead
|
||||
of the SPA `/index.html` fallback.
|
||||
(writes to the app's `webroot_pattern`, e.g. `/var/www/{env}.realpath.app/html`
|
||||
or `/var/www/{env}.chat.aimloperations/html`).
|
||||
- **web-static** role: one nginx container per app host serving the static roots
|
||||
on their ports (from `host_apps`); NPM balances across hosts. Before `compose up`,
|
||||
removes any container currently publishing those host ports (`docker ps --filter
|
||||
@@ -347,29 +277,18 @@ point each domain at the backend(s):
|
||||
|
||||
- Single host: standard Proxy Host → `adama:PORT`.
|
||||
- Active/active: jc21 NPM's UI Proxy Host is single-target. To balance
|
||||
app hosts you need the **Advanced** tab with a custom `upstream {}` block
|
||||
adama+roslin you need the **Advanced** tab with a custom `upstream {}` block
|
||||
(or a real LB). Confirm this before relying on active/active.
|
||||
|
||||
| App | Domains | Backends |
|
||||
|-----|---------|----------|
|
||||
| company_site | aimloperations.com (+ www) | `adama:8000` + `roslin:8000` + `starbuck:8000` + `apollo:8000` |
|
||||
| dta_service | (see DTA NPM hosts) | `adama:8001` / `8011` + same on roslin / starbuck / apollo / ai-server-4080 |
|
||||
| dta_webapp | (see DTA NPM hosts) | `adama:8080` / `8081` + same on roslin / starbuck / apollo |
|
||||
| dta_blog | `blog.realpath.app` (prod); `beta.blog.realpath.app` (beta). **Do not** reverse-proxy onto `realpath.app/blog` | `adama:8086` / `8087` + same on roslin / starbuck / apollo / ai-server-4080 |
|
||||
| scha | `schawheaton.aimloperations.com`, `schawheaton.com` (+ www) | `adama:8002` + `roslin:8002` + `starbuck:8002` + `apollo:8002` (+ `ai-server-4080:8002`) |
|
||||
| chat_web_app | `chat.aimloperations.com` (+ www); `beta.chat.aimloperations.com` | `adama:8082` / `8083` + same on roslin / starbuck / apollo / ai-server-4080 |
|
||||
| chat_backend | `chatbackend.aimloperations.com`; `beta.chatbackend.aimloperations.com` | `adama:8003` / `8013` + same on roslin / starbuck / apollo / ai-server-4080 |
|
||||
| monica_site | `mkdrealtor.com` (+ www); `monica-preview.aimloperations.com` (beta) | `adama:8004` / `8014` + same on roslin / starbuck / apollo / ai-server-4080 |
|
||||
| url_shortening_service (short) | `aiml.pw` and/or `cidinn.li` (pick when DNS is ready); `short-beta.aimloperations.com` (beta). Proxy `/` + `/[a-z0-9]{4,8}` only — 404 `/api/`, `/admin/`, `/debug/` | `adama:8005` / `8015` + same on roslin / starbuck / apollo / ai-server-4080 |
|
||||
| url_shortening_service (API) | `shortener.aimloperations.com`; `shortener-beta.aimloperations.com` (beta). Proxy `/api/` only — 404 `/admin/` | same ports as short host (one container) |
|
||||
| college_craft | `collegecraft.com` (+ www); `college-craft-preview.aimloperations.com` (beta) | `adama:8006` / `8016` + same on roslin / starbuck / apollo / ai-server-4080 |
|
||||
| print_forge | `printforgeprints.com` (+ www); `print-forge-preview.aimloperations.com` (beta) — **app stopped** (`enabled: false`). Disable NPM hosts or expect 502. | `:8007` / `:8019` reserved |
|
||||
| abc_fe | `beta.abc.aimloperations.com` (beta only) | `adama:8085` + same on roslin / starbuck / apollo / ai-server-4080 |
|
||||
| abc_be | `beta.abc.be.aimloperations.com` (beta only) | `adama:8017` + same on roslin / starbuck / apollo / ai-server-4080 |
|
||||
| abc_worker | `beta.abc.worker.aimloperations.com` (beta only; HTTP + WebSocket upgrade) | `adama:8018` + same on roslin / starbuck / apollo / ai-server-4080 |
|
||||
| abc_voice | `beta.abc.voice.aimloperations.com` (beta only). Balance this host. Do not balance LiveKit. | `adama:8020` + same on roslin / starbuck / apollo / ai-server-4080 |
|
||||
| livekit | `LIVEKIT_DOMAIN` (e.g. `livekit.aimloperations.com`) — NPM Proxy Host, WebSocket, **single upstream** | `adama:7880` (do not balance until Redis) |
|
||||
| disney_hotel_service | `disneyhotel.app` (+ www); `beta.disneyhotel.app` (beta) | `adama:8023` / `8024` + same on roslin / starbuck / apollo / ai-server-4080 |
|
||||
| company_site | aimloperations.com (+ www) | `adama:8000` + `roslin:8000` |
|
||||
| dta_service | (see DTA NPM hosts) | `adama:8001` / `8011` + same on roslin / ai-server-4080 |
|
||||
| dta_webapp | (see DTA NPM hosts) | `adama:8080` / `8081` + same on roslin |
|
||||
| scha | `schawheaton.aimloperations.com`, `schawheaton.com` (+ www) | `adama:8002` + `roslin:8002` (+ `ai-server-4080:8002`) |
|
||||
| chat_web_app | `chat.aimloperations.com` (+ www); `beta.chat.aimloperations.com` | `adama:8082` / `8083` + same on roslin / ai-server-4080 |
|
||||
| chat_backend | `chatbackend.aimloperations.com`; `beta.chatbackend.aimloperations.com` | `adama:8003` / `8013` + same on roslin / ai-server-4080 |
|
||||
| monica_site | `mkdrealtor.com` (+ www); `monica-preview.aimloperations.com` (beta) | `adama:8004` / `8014` + same on roslin / ai-server-4080 |
|
||||
|
||||
### Required changes IN each app repo (owned separately)
|
||||
|
||||
@@ -381,9 +300,6 @@ point each domain at the backend(s):
|
||||
(keep the test/docker jobs).
|
||||
- [ ] `dta_webapp`: `npm run build:beta` / `build:prod` output to
|
||||
`/var/www/beta.realpath.app/html` / `/var/www/prod.realpath.app/html`.
|
||||
- [ ] `dta_blog`: `npm run build:beta` / `build:prod` output to
|
||||
`/var/www/beta.blog.realpath.app/html` / `/var/www/prod.blog.realpath.app/html`
|
||||
(`python3 build.py --env <env>` copies `dist/`). Default branch is `main`.
|
||||
- [ ] `chat_web_app`: `npm run build:beta` / `build:prod` output to
|
||||
`/var/www/beta.chat.aimloperations/html` / `/var/www/prod.chat.aimloperations/html`.
|
||||
|
||||
@@ -407,152 +323,11 @@ do not).
|
||||
| chat_backend | beta | `chat_backend_beta` | `postgres://westfarn:<pw>@10.0.0.230:5432/chat_backend_beta` |
|
||||
| monica_site | prod | `monica_site` | `postgres://westfarn:<pw>@10.0.0.230:5432/monica_site` |
|
||||
| monica_site | beta | `monica_site_beta` | `postgres://westfarn:<pw>@10.0.0.230:5432/monica_site_beta` |
|
||||
| url_shortening_service | prod | `url_shortener` | `postgres://westfarn:<pw>@10.0.0.230:5432/url_shortener` |
|
||||
| url_shortening_service | beta | `url_shortener_beta` | `postgres://westfarn:<pw>@10.0.0.230:5432/url_shortener_beta` |
|
||||
| college_craft | prod | `college_craft` | `postgres://westfarn:<pw>@10.0.0.230:5432/college_craft` |
|
||||
| college_craft | beta | `college_craft_beta` | `postgres://westfarn:<pw>@10.0.0.230:5432/college_craft_beta` |
|
||||
| print_forge | prod | `print_forge` | `postgres://westfarn:<pw>@10.0.0.230:5432/print_forge` |
|
||||
| print_forge | beta | `print_forge_beta` | `postgres://westfarn:<pw>@10.0.0.230:5432/print_forge_beta` |
|
||||
| abc_be | beta | `abc_be_beta` | `postgres://westfarn:<pw>@10.0.0.230:5432/abc_be_beta` |
|
||||
| abc_worker | beta | *(same `abc_be_beta`)* | worker uses `DB_*` pointing at `abc_be_beta` — do **not** create a second DB |
|
||||
| abc_voice | beta | *(same `abc_be_beta`)* | voice container uses the worker DB settings — do **not** create a second DB |
|
||||
| disney_hotel_service | prod | `disney_hotel_service` | `postgres://westfarn:<pw>@10.0.0.230:5432/disney_hotel_service` |
|
||||
| disney_hotel_service | beta | `disney_hotel_service_beta` | `postgres://westfarn:<pw>@10.0.0.230:5432/disney_hotel_service_beta` |
|
||||
|
||||
Server prereqs on 10.0.0.230: create each DB + grant `westfarn`;
|
||||
`listen_addresses` covers LAN; `pg_hba.conf` allows `10.0.0.0/24`; firewall opens
|
||||
5432 to `10.0.0.0/24` only.
|
||||
|
||||
`disney_hotel_service` secrets (control node, never git, mode `600`) — [#47](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/47):
|
||||
|
||||
```text
|
||||
~/Documents/secrets/disney_hotel_service/disney_hotel_service_prod.env
|
||||
~/Documents/secrets/disney_hotel_service/disney_hotel_service_beta.env
|
||||
```
|
||||
|
||||
Ansible copies those to `/opt/apps/env/disney_hotel_service_<env>.env` on each
|
||||
app host. Create them on the machine that runs `deploy.sh` (ai-server-4080).
|
||||
Unix dotenv: `KEY=value`, one per line, no `export`. This app reads `POSTGRES_*`
|
||||
(`config/settings.py` and `scripts/entrypoint.sh`). It ignores `DATABASE_URL`.
|
||||
|
||||
Same keys in both files. These values differ:
|
||||
|
||||
| | prod | beta |
|
||||
|--|------|------|
|
||||
| `POSTGRES_DB` | `disney_hotel_service` | `disney_hotel_service_beta` |
|
||||
| `WEB_PORT` | `8023` | `8024` |
|
||||
| `DJANGO_ALLOWED_HOSTS` | `disneyhotel.app,www.disneyhotel.app` | `beta.disneyhotel.app` |
|
||||
| `DJANGO_CSRF_TRUSTED_ORIGINS` | `https://disneyhotel.app,https://www.disneyhotel.app` | `https://beta.disneyhotel.app` |
|
||||
| `SITE_BASE_URL` | `https://disneyhotel.app` | `https://beta.disneyhotel.app` |
|
||||
|
||||
Shared DB settings: `POSTGRES_USER=westfarn`, `POSTGRES_HOST=10.0.0.230`,
|
||||
`POSTGRES_PORT=5432`. `DJANGO_DEBUG=0`. Leave `RUN_MIGRATIONS=0` and
|
||||
`RUN_SEED=0` (deploy runs migrate once; seed parks by hand after the first
|
||||
deploy). Blank `STRIPE_SECRET_KEY` skips payment enforcement. Blank
|
||||
`SMTP2GO_API_KEY` logs email to the container console.
|
||||
|
||||
App repo still needs `docker-compose.prod.yml` on `master` (no bundled `db`,
|
||||
`ports: ["${WEB_PORT}:8000"]`, `worker` under profile `worker`) before the
|
||||
first deploy succeeds.
|
||||
|
||||
`url_shortening_service` extra env (control-node secrets, not in git):
|
||||
`SHORT_PUBLIC_HOSTS` / `SHORT_API_HOSTS` / `SHORT_ADMIN_HOSTS` (admin = `localhost,127.0.0.1` only),
|
||||
`SHORTENER_API_TOKENS` (`monica:<token>`), `SHORT_ALLOWED_HOSTS`, `CLICK_IP_PEPPER`
|
||||
(distinct from `DJANGO_SECRET_KEY`). `DJANGO_DEBUG=false` for prod and beta.
|
||||
Caller `monica_site` uses `SHORTENER_BASE_URL=https://<api-host>` and Bearer mint;
|
||||
do not mint via the short hostname.
|
||||
|
||||
`college_craft` extra env (control-node secrets, not in git):
|
||||
`~/Documents/secrets/college_craft/college_craft_prod.env` and
|
||||
`college_craft_beta.env`. Template: app repo `.env.prod.example`.
|
||||
`DJANGO_ENV` / `DJANGO_ALLOWED_HOSTS` / `DATABASE_URL` / `WEB_PORT` /
|
||||
`PUBLIC_SITE_URL` differ per env (prod `collegecraft.com` / `:8006`;
|
||||
beta `college-craft-preview.aimloperations.com` / `:8016`).
|
||||
`SITE_UNDER_CONSTRUCTION=true` on prod (holding page until launch), `false` on beta.
|
||||
`FEATURE_BLOG=true`; other `FEATURE_*` stay false until purchased. Prod also needs
|
||||
`TIANJI_WEBSITE_ID` + reCAPTCHA keys. If email/SMS or direct mail is turned on later,
|
||||
add a `college_craft:<token>` entry to `SHORTENER_API_TOKENS` on `url_shortening_service`.
|
||||
|
||||
App-repo companion (`college_craft`, not this repo): deploy workflows still call
|
||||
`--app client_site` (template leftover). They must become `--app college_craft`
|
||||
before CI deploy will hit this catalog entry. Default branch is `master` (not `main`).
|
||||
|
||||
`print_forge` extra env (control-node secrets, not in git) — **currently stopped**
|
||||
([#33](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/33)):
|
||||
`host_apps` rows stay with `enabled: false`. Bring back with `enabled` removed/true
|
||||
then `deploy.sh`. Secrets kept:
|
||||
`~/Documents/secrets/print_forge/print_forge_prod.env` and
|
||||
`print_forge_beta.env`. Template: app repo `.env.prod.example`.
|
||||
`DJANGO_ENV` / `DJANGO_ALLOWED_HOSTS` / `DATABASE_URL` / `WEB_PORT` /
|
||||
`PUBLIC_SITE_URL` differ per env (prod `printforgeprints.com` / `:8007`;
|
||||
beta `print-forge-preview.aimloperations.com` / `:8019`).
|
||||
`SITE_UNDER_CONSTRUCTION=true` on prod (holding page until launch), `false` on beta.
|
||||
Purchased flags: `FEATURE_EMAIL_SMS`, `FEATURE_PAYMENTS`, `FEATURE_SHOP`,
|
||||
`FEATURE_SHIPPING`. Other `FEATURE_*` stay false. Add a `print_forge:<token>`
|
||||
entry to `SHORTENER_API_TOKENS` on `url_shortening_service` (beta token first).
|
||||
`SHORTENER_BASE_URL` is prod shortener on prod, `https://shortener-beta.aimloperations.com`
|
||||
on beta. Default branch is `master`.
|
||||
|
||||
Companion app ticket: [print_forge#1](https://git.aimloperations.com/ai_ml_operations/print_forge/issues/1)
|
||||
([#27](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/27)).
|
||||
App Gitea workflow still calls `--app print_forge` on `master`; deploy **no-ops**
|
||||
while `enabled: false`.
|
||||
|
||||
`abc_be` / `abc_worker` / `abc_fe` (GIS org, not this repo) — **beta only** ([#26](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/26)):
|
||||
|
||||
Control-node secrets (never git, mode `600`):
|
||||
|
||||
```text
|
||||
~/Documents/secrets/abc_be/abc_be_beta.env
|
||||
~/Documents/secrets/abc_worker/abc_worker_beta.env
|
||||
~/Documents/secrets/abc_voice/abc_voice_beta.env
|
||||
```
|
||||
|
||||
Templates: `GIS/abc_be` `.env.beta.example`, `GIS/abc_worker` `.env.beta.example`, `GIS/abc_worker` `.env.voice.beta.example`.
|
||||
`abc_voice` is the same checkout as `abc_worker` with `docker-compose.voice.yml`. Copy the worker beta env, set `VOICE_ENABLED=true`, `CARTESIA_API_KEY`, and the LiveKit keys. `abc_be` then uses `WORKER_AV_URL=https://beta.abc.voice.aimloperations.com`.
|
||||
`abc_fe` has no secret file — public `VITE_*` URLs live in committed `.env.beta`.
|
||||
Create Postgres DB `abc_be_beta` and grant `westfarn` before first deploy.
|
||||
NPM: `beta.abc.aimloperations.com` → `:8085`, `beta.abc.be.aimloperations.com` → `:8017`,
|
||||
`beta.abc.worker.aimloperations.com` → `:8018` (enable WebSocket),
|
||||
`beta.abc.voice.aimloperations.com` → `:8020` on all five webservers. No prod ABC
|
||||
`host_apps` rows; ports **8009** / 8008 / 8084 reserved (`print_forge` took prod **8007**).
|
||||
|
||||
Companion workflows: [abc_be#22](https://git.aimloperations.com/GIS/abc_be/issues/22),
|
||||
[abc_worker#27](https://git.aimloperations.com/GIS/abc_worker/issues/27),
|
||||
[abc_fe#27](https://git.aimloperations.com/GIS/abc_fe/issues/27).
|
||||
|
||||
`livekit` (GIS org, SFU on webservers — [abc_worker#14](https://git.aimloperations.com/GIS/abc_worker/issues/14)–[#17](https://git.aimloperations.com/GIS/abc_worker/issues/17)):
|
||||
|
||||
Control-node secret (never git, mode `600`):
|
||||
|
||||
```text
|
||||
~/Documents/secrets/livekit/livekit_beta.env
|
||||
```
|
||||
|
||||
Template: `GIS/livekit` `.env.beta.example`. Same `LIVEKIT_API_KEY` /
|
||||
`LIVEKIT_API_SECRET` pair as `abc_be` / `abc_worker`; worker `LIVEKIT_URL=wss://<LIVEKIT_DOMAIN>`.
|
||||
|
||||
Deploy: `./scripts/deploy.sh --app livekit --env beta` (all webservers, like
|
||||
`abc_worker`). NPM: `LIVEKIT_DOMAIN` → **one** host `:7880` with WebSocket
|
||||
(adama). Router DNAT UDP 3478 + 50000–60000 and TCP 7881 to that same host.
|
||||
Do not active/active-balance LiveKit until Redis is in the compose.
|
||||
|
||||
`dta_blog` ([#29](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/29)) —
|
||||
Python SSG, **no secrets file**. `npm run build:<env>` writes
|
||||
`/var/www/<env>.blog.realpath.app/html`. Prod (`:8086`) omits `demo: true` seed
|
||||
posts; beta (`:8087`) includes them. Header/footer links are baked at build
|
||||
(`https://realpath.app` vs `https://beta.realpath.app`); Tianji website ids
|
||||
are likewise baked (`cmtvvmf562afjzqumwt1yh2y8` / `cmtvvn6z62agdzqumtk8xijpy`).
|
||||
Default branch is `main`.
|
||||
|
||||
NPM / DNS / Cloudflare (Ansible does not manage these): `blog.realpath.app` →
|
||||
`:8086`, `beta.blog.realpath.app` → `:8087`, TLS like the other RealPath names.
|
||||
Do **not** mount this on `realpath.app/blog`. Static ports stay LAN / NPM-only
|
||||
(UFW does not world-open 8086/8087).
|
||||
|
||||
Companion app ticket: [dta_blog#1](https://git.aimloperations.com/Ditch_The_Agent/dta_blog/issues/1).
|
||||
Gitea deploy can later call `./scripts/deploy.sh --app dta_blog --env beta|prod`.
|
||||
|
||||
### One-time host bootstrap (per target)
|
||||
|
||||
- [x] Gitea SSH key: the `gitea-key` role (in `site.yml`) generates a key per
|
||||
@@ -563,7 +338,7 @@ Gitea deploy can later call `./scripts/deploy.sh --app dta_blog --env beta|prod`
|
||||
(default `~/Documents/secrets/<app>/<app>_<env>.env`) with `DATABASE_URL`
|
||||
(see table), `DJANGO_ENV`, `DJANGO_SECRET_KEY`, `WEB_PORT` (matching the port
|
||||
table). Deploy pushes these to `/opt/apps/env/<app>_<env>.env` (mode 600) on
|
||||
adama + roslin + starbuck + apollo. Never committed to git.
|
||||
adama + roslin. Never committed to git.
|
||||
- [x] Node.js/npm/npx for the `dta_webapp` build — installed by the `nodejs`
|
||||
role in `site.yml` (NodeSource, `node_major` default 20).
|
||||
|
||||
@@ -572,7 +347,7 @@ Gitea deploy can later call `./scripts/deploy.sh --app dta_blog --env beta|prod`
|
||||
**Recommended:** Single self-hosted runner on ai-server-4080.
|
||||
|
||||
- One orchestration point.
|
||||
- App hosts (adama/roslin/starbuck/apollo) run the workloads; no runner needed on them for deploy fan-out.
|
||||
- App hosts (adama/roslin) run the workloads; no runner needed on them for deploy fan-out.
|
||||
- Runner needs: Ansible, this repo checked out, SSH key to all hosts, vault password (later).
|
||||
|
||||
### Runner requirements on ai-server-4080
|
||||
@@ -581,7 +356,7 @@ Gitea deploy can later call `./scripts/deploy.sh --app dta_blog --env beta|prod`
|
||||
|-------------|-----|
|
||||
| Ansible | Run `deploy-apps.yml` |
|
||||
| `server-infra` checkout | Playbooks + inventory |
|
||||
| SSH key to app hosts | Deploy fan-out |
|
||||
| SSH key to adama + roslin | Deploy fan-out |
|
||||
|
||||
On every push or merged PR to `master`, `.gitea/workflows/sync-checkout.yml`
|
||||
fast-forward pulls this repo at `~/Documents/repos/server-infra` on the Act
|
||||
@@ -612,8 +387,8 @@ Store vault password for CI in a file readable only by the Act runner (e.g. `~/.
|
||||
| # | Task | Status |
|
||||
|---|------|--------|
|
||||
| 1 | Create `server-infra` repo | Done |
|
||||
| 2 | Inventory with all 5 hosts | Done ([#20](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/20)) |
|
||||
| 3 | Bootstrap SSH to app hosts | Manual |
|
||||
| 2 | Inventory with all 3 hosts | Done |
|
||||
| 3 | Bootstrap SSH to adama + roslin | Manual |
|
||||
| 4 | `site.yml` → common, ufw, docker | Done |
|
||||
| 5 | Verify `ansible webservers -m ping` | Manual |
|
||||
| 6 | Test on single server: `./scripts/provision.sh adama` | Manual |
|
||||
@@ -626,21 +401,13 @@ Store vault password for CI in a file readable only by the Act runner (e.g. `~/.
|
||||
| 10b | Register + deploy `chat_web_app` (node-static, ports 8082/8083) | Done (prod); beta ([#7](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/7), [chat_web_app#35](https://git.aimloperations.com/ai_ml_operations/chat_web_app/issues/35)) |
|
||||
| 10c | Register + deploy `chat_backend` (django, ports 8003/8013) | Done (prod); beta ([#7](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/7), [chat_backend#26](https://git.aimloperations.com/ai_ml_operations/chat_backend/issues/26)) |
|
||||
| 10d | Register + deploy `monica_site` (django, ports 8004/8014) | Done ([#14](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/14)) |
|
||||
| 10e | Auto-start `monica_site` dj-queue worker on adama (`compose_profiles`) | Done ([#17](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/17)) |
|
||||
| 10f | Add starbuck + apollo as app hosts (same workloads as roslin) | Done ([#20](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/20)) |
|
||||
| 10g | Register + deploy `url_shortening_service` (django, ports 8005/8015) | Done ([#22](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/22)) |
|
||||
| 10h | Register + deploy `college_craft` (django, ports 8006/8016) | Done ([#24](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/24)) |
|
||||
| 10i | Register + deploy ABC beta (`abc_be` 8017, `abc_worker` 8018, `abc_fe` 8085) | Done ([#26](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/26)) |
|
||||
| 10j | Register + deploy `print_forge` (django, ports 8007/8019) | Done ([#27](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/27)) |
|
||||
| 10k | Register + deploy `dta_blog` (node-static, ports 8086/8087) | Done ([#29](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/29)) |
|
||||
| 10l | Register GIS LiveKit SFU (`abc_worker`-style docker on webservers, port 7880) | Done |
|
||||
| 10m | Register + deploy `disney_hotel_service` (django, ports 8023/8024) | This PR ([#47](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/47)) |
|
||||
| 10e | Auto-start `monica_site` dj-queue worker on adama (`compose_profiles`) | In progress ([#17](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/17)) |
|
||||
| 11 | Gitea container registry (optional) | Future |
|
||||
|
||||
## Open Decisions
|
||||
|
||||
1. **Deploy user** — `westfarn` vs dedicated `deploy` for CI.
|
||||
2. **NPM load balancing** — confirm jc21 NPM can express all app-host upstreams (Advanced tab), else active/active is just independent instances.
|
||||
2. **NPM load balancing** — confirm jc21 NPM can express adama+roslin upstreams (Advanced tab), else active/active is just two independent instances.
|
||||
3. **Secrets** — Ansible Vault vs per-host env files (currently per-host `/opt/apps/env/*.env`).
|
||||
|
||||
## Adding a New VM
|
||||
|
||||
@@ -11,8 +11,6 @@ ansible-galaxy collection install -r requirements.yml
|
||||
# Bootstrap SSH key to each host (one-time, before Ansible)
|
||||
ssh-copy-id westfarn@10.0.0.77
|
||||
ssh-copy-id westfarn@10.0.0.176
|
||||
ssh-copy-id westfarn@10.0.0.44
|
||||
ssh-copy-id westfarn@10.0.0.7
|
||||
|
||||
# First-time only: passwordless sudo on each new host (before first provision)
|
||||
ssh -t westfarn@10.0.0.176 # repeat for each host IP
|
||||
@@ -33,21 +31,12 @@ ansible adama -m ping
|
||||
|
||||
# Provision all hosts
|
||||
./scripts/provision.sh
|
||||
|
||||
# Stop a compose app (does not remove volumes/checkouts)
|
||||
./scripts/stop.sh --app print_forge --env prod
|
||||
./scripts/stop.sh --app print_forge --env beta
|
||||
```
|
||||
|
||||
See [IMPLEMENTATION.md](IMPLEMENTATION.md) for full architecture, CI/CD plan, and phase breakdown.
|
||||
|
||||
Observability (Alloy → Loki / Prometheus → Grafana): [docs/OBSERVABILITY.md](docs/OBSERVABILITY.md) · [docs/GRAFANA_USAGE.md](docs/GRAFANA_USAGE.md)
|
||||
|
||||
WordPress (MariaDB on TrueNAS): [docs/WORDPRESS.md](docs/WORDPRESS.md).
|
||||
Provision mounts each site's `wp-content` and writes its compose file from
|
||||
[inventory/wordpress_sites.yml](inventory/wordpress_sites.yml). Start the
|
||||
containers by hand (adama first).
|
||||
|
||||
Provision installs **Alloy** on every host. On **ai-server-4080** it also starts
|
||||
the central **Loki / Prometheus / Grafana** stack (`observability_stack: true`).
|
||||
|
||||
@@ -57,6 +46,4 @@ the central **Loki / Prometheus / Grafana** stack (`observability_stack: true`).
|
||||
|------|-----|------|
|
||||
| adama | 10.0.0.77 | app host |
|
||||
| roslin | 10.0.0.176 | app host |
|
||||
| starbuck | 10.0.0.44 | app host |
|
||||
| apollo | 10.0.0.7 | app host |
|
||||
| ai-server-4080 | 10.0.0.128 | control node + act runner |
|
||||
@@ -45,7 +45,7 @@ first; use text search (`|=`, `|~`) on logs second.
|
||||
|
||||
| Label | Meaning | Examples |
|
||||
|-------|---------|----------|
|
||||
| `host` | Inventory hostname | `adama`, `roslin`, `starbuck`, `apollo`, `ai-server-4080` |
|
||||
| `host` | Inventory hostname | `adama`, `roslin`, `ai-server-4080` |
|
||||
| `env` | Deploy environment | `beta`, `prod`, `host` (journal), `infra` (stack containers) |
|
||||
| `app` | App / service name | `company_site`, `dta_service`, `dta_webapp`, `system` |
|
||||
| `job` | Collector | `docker`, `systemd` |
|
||||
@@ -56,7 +56,7 @@ first; use text search (`|=`, `|~`) on logs second.
|
||||
|
||||
| Label | Meaning | Examples |
|
||||
|-------|---------|----------|
|
||||
| `host` | Inventory hostname (stamped by Alloy) | `adama`, `roslin`, `starbuck`, `apollo`, `ai-server-4080` |
|
||||
| `host` | Inventory hostname (stamped by Alloy) | `adama`, `roslin`, `ai-server-4080` |
|
||||
| `job` | Scrape job | `node` (host), `cadvisor` (containers) |
|
||||
| `env` / `app` | Parsed from Compose project `<app>_<env>` | `prod` / `dta_service` |
|
||||
| `name` | Container name (cAdvisor) | `company_site_prod-web-1` |
|
||||
|
||||
+7
-17
@@ -10,8 +10,6 @@ flowchart LR
|
||||
subgraph hosts ["All webservers"]
|
||||
A["adama\nAlloy"]
|
||||
R["roslin\nAlloy"]
|
||||
S["starbuck\nAlloy"]
|
||||
Ap["apollo\nAlloy"]
|
||||
C["ai-server-4080\nAlloy"]
|
||||
end
|
||||
|
||||
@@ -26,13 +24,9 @@ flowchart LR
|
||||
|
||||
A -->|logs| L
|
||||
R -->|logs| L
|
||||
S -->|logs| L
|
||||
Ap -->|logs| L
|
||||
C -->|logs| L
|
||||
A -->|metrics| P
|
||||
R -->|metrics| P
|
||||
S -->|metrics| P
|
||||
Ap -->|metrics| P
|
||||
C -->|metrics| P
|
||||
L --> G
|
||||
P --> G
|
||||
@@ -42,7 +36,7 @@ flowchart LR
|
||||
|
||||
| Piece | Where | Role |
|
||||
|-------|--------|------|
|
||||
| **Alloy** | every host (`adama`, `roslin`, `starbuck`, `apollo`, `ai-server-4080`) | Ship journald + Docker **logs** to Loki; scrape host + container **metrics** → Prometheus |
|
||||
| **Alloy** | every host (`adama`, `roslin`, `ai-server-4080`) | Ship journald + Docker **logs** to Loki; scrape host + container **metrics** → Prometheus |
|
||||
| **Loki** | `ai-server-4080` only | Store and index logs |
|
||||
| **Prometheus** | `ai-server-4080` only | Store metrics (CPU, RAM, disk, container health) |
|
||||
| **Grafana** | `ai-server-4080` only | Explore logs/metrics, dashboards, alerts |
|
||||
@@ -188,7 +182,7 @@ services:
|
||||
user: "0:0"
|
||||
command: -config.file=/etc/loki/loki-config.yml
|
||||
ports:
|
||||
# Bind to all interfaces so Alloy on app hosts can push.
|
||||
# Bind to all interfaces so Alloy on adama/roslin can push.
|
||||
# Firewall (UFW) should restrict who can connect — see 1.6.
|
||||
- "3100:3100"
|
||||
volumes:
|
||||
@@ -382,7 +376,7 @@ Alloy runs on **every** host in `webservers`. It:
|
||||
|
||||
| Label | Source | Example values |
|
||||
|-------|--------|----------------|
|
||||
| `host` | Ansible inventory hostname | `adama`, `roslin`, `starbuck`, `apollo`, `ai-server-4080` |
|
||||
| `host` | Ansible inventory hostname | `adama`, `roslin`, `ai-server-4080` |
|
||||
| `job` | collector name | `systemd`, `docker` |
|
||||
| `env` | Docker Compose project suffix | `beta`, `prod`, `host`, `infra` |
|
||||
| `app` | Compose project prefix | `company_site`, `dta_service`, `dta_webapp`, … |
|
||||
@@ -393,7 +387,7 @@ Alloy runs on **every** host in `webservers`. It:
|
||||
|
||||
| Label | Source | Example values |
|
||||
|-------|--------|----------------|
|
||||
| `host` | Added by Alloy relabel | `adama`, `roslin`, `starbuck`, `apollo`, `ai-server-4080` |
|
||||
| `host` | Added by Alloy relabel | `adama`, `roslin`, `ai-server-4080` |
|
||||
| `job` | scrape job name | `node`, `cadvisor` |
|
||||
| `name` | container name (cAdvisor) | `company_site_prod-web-1` |
|
||||
| `container_label_com_docker_compose_project` | Compose project | `dta_service_prod` |
|
||||
@@ -413,7 +407,7 @@ container_memory_usage_bytes{host="adama", env="prod", app="dta_service"}
|
||||
|
||||
### 3.1 Install Alloy (manual — one host)
|
||||
|
||||
Repeat on `adama`, `roslin`, `starbuck`, `apollo`, and `ai-server-4080`. Example for **adama**:
|
||||
Repeat on `adama`, `roslin`, and `ai-server-4080`. Example for **adama**:
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /opt/apps/observability/alloy
|
||||
@@ -421,7 +415,7 @@ sudo chown -R westfarn:westfarn /opt/apps/observability
|
||||
```
|
||||
|
||||
Create `/opt/apps/observability/alloy/config.alloy`. **Change every
|
||||
`host = "adama"`** on each machine (`adama` / `roslin` / `starbuck` / `apollo` / `ai-server-4080`):
|
||||
`host = "adama"`** on each machine (`adama` / `roslin` / `ai-server-4080`):
|
||||
|
||||
```river
|
||||
// Grafana Alloy — logs → Loki, metrics → Prometheus.
|
||||
@@ -711,8 +705,6 @@ Healthy Alloy logs mention connecting / sending without repeated
|
||||
```logql
|
||||
{host="adama"}
|
||||
{host="roslin"}
|
||||
{host="starbuck"}
|
||||
{host="apollo"}
|
||||
{host="ai-server-4080"}
|
||||
```
|
||||
|
||||
@@ -824,8 +816,6 @@ observability_stack: true
|
||||
./scripts/provision.sh ai-server-4080
|
||||
./scripts/provision.sh adama
|
||||
./scripts/provision.sh roslin
|
||||
./scripts/provision.sh starbuck
|
||||
./scripts/provision.sh apollo
|
||||
# or all (site.yml orders stack before Alloy):
|
||||
./scripts/provision.sh
|
||||
```
|
||||
@@ -926,7 +916,7 @@ du -sh /opt/apps/observability/loki/data \
|
||||
- [ ] `GF_SERVER_ROOT_URL` matches public URL
|
||||
- [ ] Both Loki and Prometheus datasources green in Grafana
|
||||
|
||||
### Alloy (each of adama, roslin, starbuck, apollo, ai-server-4080)
|
||||
### Alloy (each of adama, roslin, ai-server-4080)
|
||||
|
||||
- [ ] `config.alloy` has correct `host = "..."` (or Ansible `inventory_hostname`)
|
||||
- [ ] Alloy container running privileged with host `/proc` `/sys` mounts
|
||||
|
||||
@@ -1,393 +0,0 @@
|
||||
# Adding a WordPress site
|
||||
|
||||
Manual procedure for a WordPress site on all five web servers, one MariaDB on
|
||||
TrueNAS, and Nginx Proxy Manager (NPM) in front. Ansible does **not** start
|
||||
the WordPress containers. When you run `scripts/provision.sh` it mounts each
|
||||
site's `wp-content` and, once that entry has a public `url`, writes its
|
||||
compose file from
|
||||
[inventory/wordpress_sites.yml](../inventory/wordpress_sites.yml). Repeat
|
||||
[Per site](#per-site) for every new site. Do not install another MariaDB.
|
||||
|
||||
## Where things live
|
||||
|
||||
| Piece | Where |
|
||||
|-------|--------|
|
||||
| MariaDB | TrueNAS `10.0.0.230`, port **3306**, LAN only |
|
||||
| MariaDB login | `westfarn` @ `%` (the host the TrueNAS app created) |
|
||||
| NPM | `10.0.0.230` (not managed by Ansible) |
|
||||
| App hosts | adama `10.0.0.77`, roslin `10.0.0.176`, starbuck `10.0.0.44`, apollo `10.0.0.7`, ai-server-4080 `10.0.0.128` |
|
||||
| Secrets | `~/Documents/secrets/wordpress/` on the control node. Never commit these. |
|
||||
| Shared files | one NFS dataset per site per env, mounted on every app host |
|
||||
|
||||
Prod and beta never share a database, an NFS dataset, a port, or a domain.
|
||||
The five hosts of one env do share that env's database and `wp-content`.
|
||||
|
||||
## Already done
|
||||
|
||||
Leave this alone when adding the next site.
|
||||
|
||||
- MariaDB app is running. Its host-path dataset is owned by uid **999** (`netdata`). Without that, the container crash-loops with `Errcode: 13 "Permission denied"` on `/var/lib/mysql`.
|
||||
- Login `westfarn` exists at host `%`, not at `10.0.0.%`. `GRANT` to `'westfarn'@'10.0.0.%'` fails with `ERROR 1133` because that row is not in `mysql.user`.
|
||||
- Databases `relawfirm` and `relawfirm_beta` already exist.
|
||||
|
||||
Confirm from **Apps → mariadb → Shell**:
|
||||
|
||||
```bash
|
||||
mariadb -u root -p
|
||||
```
|
||||
|
||||
```sql
|
||||
SELECT user, host FROM mysql.user WHERE user = 'westfarn';
|
||||
SHOW DATABASES;
|
||||
SHOW GRANTS FOR 'westfarn'@'%';
|
||||
```
|
||||
|
||||
`relawfirm` still needs the grants below if `SHOW GRANTS` does not list both databases. New sites use the same `GRANT` shape with their own database names.
|
||||
|
||||
```sql
|
||||
GRANT ALL PRIVILEGES ON relawfirm.* TO 'westfarn'@'%';
|
||||
GRANT ALL PRIVILEGES ON relawfirm_beta.* TO 'westfarn'@'%';
|
||||
FLUSH PRIVILEGES;
|
||||
```
|
||||
|
||||
## Ports
|
||||
|
||||
Same host port on every app server for a given site and env. NPM balances
|
||||
`adama:PORT` + `roslin:PORT` + `starbuck:PORT` + `apollo:PORT` + `ai-server-4080:PORT`.
|
||||
|
||||
These are not in the Ansible `host_apps` tables. Do not reuse a port from
|
||||
[IMPLEMENTATION.md](../IMPLEMENTATION.md) (8000–8020, **8023–8024**, and 8080–8088
|
||||
are taken; **8020** is `abc_voice`, **8023/8024** are `disney_hotel_service`).
|
||||
Next WordPress site starts at **8025**.
|
||||
|
||||
| Site | prod port | beta port | prod database | beta database |
|
||||
|------|-----------|-----------|---------------|---------------|
|
||||
| relawfirm | **8021** | **8022** | `relawfirm` | `relawfirm_beta` |
|
||||
| *(next site)* | next free | next free | `<site>` | `<site>_beta` |
|
||||
|
||||
---
|
||||
|
||||
## Per site
|
||||
|
||||
Fill this in before touching a host. `relawfirm` is the first site.
|
||||
|
||||
| | prod | beta |
|
||||
|--|------|------|
|
||||
| Public URL | `https://<prod-domain>` | `https://<beta-domain>` |
|
||||
| Database | `relawfirm` | `relawfirm_beta` |
|
||||
| Host port | `8021` | `8022` |
|
||||
| NFS dataset | `/mnt/AIMLOperationsNAS/wordpress/relawfirm` | `/mnt/AIMLOperationsNAS/wordpress/relawfirm_beta` |
|
||||
| Mount on app hosts | `/mnt/wordpress/relawfirm/wp-content` | `/mnt/wordpress/relawfirm_beta/wp-content` |
|
||||
| Env file | `relawfirm_prod.env` | `relawfirm_beta.env` |
|
||||
| Compose project | `relawfirm_prod` | `relawfirm_beta` |
|
||||
|
||||
`<site>` below means that short name (`relawfirm`, then whatever the next site is). Use lowercase letters, numbers, and underscore only.
|
||||
|
||||
### 1. Databases
|
||||
|
||||
Skip this for `relawfirm` / `relawfirm_beta`. They already exist.
|
||||
|
||||
In the MariaDB shell (`mariadb -u root -p`):
|
||||
|
||||
```sql
|
||||
CREATE DATABASE <site> CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
|
||||
CREATE DATABASE <site>_beta CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
|
||||
GRANT ALL PRIVILEGES ON <site>.* TO 'westfarn'@'%';
|
||||
GRANT ALL PRIVILEGES ON <site>_beta.* TO 'westfarn'@'%';
|
||||
FLUSH PRIVILEGES;
|
||||
```
|
||||
|
||||
The account in `TO` must match `SELECT user, host FROM mysql.user`. Today that is `'westfarn'@'%'`. Do not create another MariaDB user unless you want a separate login; this matches the shared Postgres user.
|
||||
|
||||
### 2. Salts and env files
|
||||
|
||||
On the control node, generate salts **once per env**:
|
||||
|
||||
```bash
|
||||
curl -fsS https://api.wordpress.org/secret-key/1.1/salt/
|
||||
```
|
||||
|
||||
Copy the same eight values into the env file on every app host for that env. A different salt on one host logs people out as soon as NPM sends them to another host. Prod and beta should not share salts.
|
||||
|
||||
Create the file locally (mode `600`). Do not commit it.
|
||||
|
||||
`~/Documents/secrets/wordpress/<site>_prod.env`:
|
||||
|
||||
```bash
|
||||
WORDPRESS_DB_HOST=10.0.0.230:3306
|
||||
WORDPRESS_DB_USER=westfarn
|
||||
WORDPRESS_DB_PASSWORD=<mariadb password for westfarn>
|
||||
WORDPRESS_DB_NAME=<site>
|
||||
WORDPRESS_AUTH_KEY=<from salt generator>
|
||||
WORDPRESS_SECURE_AUTH_KEY=<from salt generator>
|
||||
WORDPRESS_LOGGED_IN_KEY=<from salt generator>
|
||||
WORDPRESS_NONCE_KEY=<from salt generator>
|
||||
WORDPRESS_AUTH_SALT=<from salt generator>
|
||||
WORDPRESS_SECURE_AUTH_SALT=<from salt generator>
|
||||
WORDPRESS_LOGGED_IN_SALT=<from salt generator>
|
||||
WORDPRESS_NONCE_SALT=<from salt generator>
|
||||
```
|
||||
|
||||
`<site>_beta.env` is the same except `WORDPRESS_DB_NAME=<site>_beta` and a second set of salts.
|
||||
|
||||
`WORDPRESS_DB_PASSWORD` is the MariaDB password. The WordPress admin password is chosen later in the browser, on the first host only.
|
||||
|
||||
### 3. Shared `wp-content` (TrueNAS NFS)
|
||||
|
||||
Uploads, plugins, and themes are files. With five hosts, those files have to live on one share. A private Docker volume on each host 404s media as soon as NPM moves the next request.
|
||||
|
||||
Create **one dataset per site per env**. Preset **Generic** (this dataset is an NFS share, not the MariaDB host path). Leave record size at the default. Case sensitive. Do not add an SMB share.
|
||||
|
||||
Example paths:
|
||||
|
||||
```text
|
||||
/mnt/AIMLOperationsNAS/wordpress/relawfirm
|
||||
/mnt/AIMLOperationsNAS/wordpress/relawfirm_beta
|
||||
```
|
||||
|
||||
**Shares → Unix Shares (NFS) → Add**, one share per dataset:
|
||||
|
||||
| Setting | Value |
|
||||
|---------|--------|
|
||||
| Path | the dataset |
|
||||
| Authorized Networks | `10.0.0.0/24` |
|
||||
| Maproot User | `root` |
|
||||
| Maproot Group | `root` |
|
||||
|
||||
Maproot root lets the WordPress entrypoint create files on an empty share. The export is limited to the LAN. Do not forward NFS to the internet.
|
||||
|
||||
Do not edit `/etc/fstab` by hand. Add one entry per env to
|
||||
[inventory/wordpress_sites.yml](../inventory/wordpress_sites.yml), then mount
|
||||
every host:
|
||||
|
||||
```bash
|
||||
./scripts/provision.sh
|
||||
```
|
||||
|
||||
`export` is the TrueNAS dataset. `mount` is the path on the app hosts. Prod
|
||||
and beta are separate entries (beta export ends in `_beta`, not `-beta`).
|
||||
|
||||
```yaml
|
||||
wordpress_sites:
|
||||
- name: <site>
|
||||
port: 8021
|
||||
url: https://<prod-domain>
|
||||
export: /mnt/AIMLOperationsNAS/wordpress/<site>
|
||||
mount: /mnt/wordpress/<site>/wp-content
|
||||
- name: <site>_beta
|
||||
port: 8022
|
||||
url: https://<beta-domain>
|
||||
export: /mnt/AIMLOperationsNAS/wordpress/<site>_beta
|
||||
mount: /mnt/wordpress/<site>_beta/wp-content
|
||||
```
|
||||
|
||||
`url` is the public NPM URL. Leave it empty and provision still mounts NFS,
|
||||
but it does not write a compose file. Set it, then provision again.
|
||||
|
||||
One host is enough to check the new lines:
|
||||
|
||||
```bash
|
||||
./scripts/provision.sh adama
|
||||
mount | grep wordpress
|
||||
touch /mnt/wordpress/<site>/wp-content/.write-test && rm /mnt/wordpress/<site>/wp-content/.write-test
|
||||
```
|
||||
|
||||
The first container to start copies the stock themes and plugins into this directory. Mount it on all five hosts **before** that start, and start only one host while the directory is still empty. Beta uses its own dataset and its own yaml entry.
|
||||
|
||||
### 4. Compose file
|
||||
|
||||
Do not copy a compose file by hand. The template is
|
||||
[roles/wordpress/templates/docker-compose.yml.j2](../roles/wordpress/templates/docker-compose.yml.j2).
|
||||
`scripts/provision.sh` renders one file per inventory entry on every app host.
|
||||
Secrets stay in the env file. `WP_HOME` and `WP_SITEURL` come from that
|
||||
entry's `url`, not from `http://10.0.0.77:8021`.
|
||||
|
||||
| `name` | Compose file | Project (`name:` in the file) | Env file |
|
||||
|--------|--------------|--------------------------------|----------|
|
||||
| `<site>` | `/opt/apps/wordpress/<site>/docker-compose.yml` | `<site>_prod` | `/opt/apps/env/<site>_prod.env` |
|
||||
| `<site>_beta` | `/opt/apps/wordpress/<site>-beta/docker-compose.yml` | `<site>_beta` | `/opt/apps/env/<site>_beta.env` |
|
||||
|
||||
Prod and beta do not share a directory. Image default is `wordpress:6-apache`
|
||||
(`wordpress_image` in the role). Override with `image:` on one entry. The
|
||||
bind mount is that entry's `mount`.
|
||||
|
||||
Copy the env file onto each host (mode `600`) before the first start.
|
||||
Provision creates `/opt/apps/env` (mode `700`) but does not copy secrets.
|
||||
|
||||
### 5. Install in the browser, one host only
|
||||
|
||||
Start **adama** first. Leave the other four stopped until the installer has finished. Five installers against one database will fight.
|
||||
|
||||
On adama:
|
||||
|
||||
```bash
|
||||
cd /opt/apps/wordpress/<site>
|
||||
docker compose -p <site>_prod up -d
|
||||
docker compose -p <site>_prod logs -f
|
||||
```
|
||||
|
||||
Wait until Apache is up. The log should not show a database connection error.
|
||||
|
||||
In NPM, add a normal Proxy Host (single forward target) **before** opening the installer:
|
||||
|
||||
| Field | Value |
|
||||
|-------|--------|
|
||||
| Domain | prod domain |
|
||||
| Scheme | `http` |
|
||||
| Forward Hostname | `10.0.0.77` |
|
||||
| Forward Port | `8021` |
|
||||
| SSL | request a certificate, force SSL |
|
||||
| Websockets | off |
|
||||
|
||||
Open `https://<prod-domain>` and finish the WordPress installer (site title, admin user, admin password). That admin password is only in WordPress. It is not the MariaDB password.
|
||||
|
||||
Then confirm a page refresh stays on `https://<prod-domain>` and does not bounce to `http://10.0.0.77:8021`.
|
||||
|
||||
### 6. Start the other four hosts
|
||||
|
||||
Copy the same compose file and the same env file to roslin, starbuck, apollo, and ai-server-4080. NFS must already be mounted. Then, on each:
|
||||
|
||||
```bash
|
||||
cd /opt/apps/wordpress/<site>
|
||||
docker compose -p <site>_prod up -d
|
||||
```
|
||||
|
||||
There is no second browser install. These containers attach to the database adama already populated.
|
||||
|
||||
### 7. Load balance in NPM
|
||||
|
||||
NPM's proxy host form accepts one forward address. The upstream list has to live in NPM's custom **http** config, which is included outside the server block. An `upstream` directive in the proxy host Advanced tab is rejected (`upstream directive is not allowed here`).
|
||||
|
||||
On the NPM host (TrueNAS shell, or the NPM app shell), edit the file the container sees as `/data/nginx/custom/http.conf`. On a TrueNAS install that path is under the NPM app's data dataset, usually `.../npm/data/nginx/custom/http.conf`. Create `custom/` if it is missing.
|
||||
|
||||
Append one block per site per env. Name the upstream after the compose project.
|
||||
|
||||
```nginx
|
||||
upstream relawfirm_prod {
|
||||
least_conn;
|
||||
server 10.0.0.77:8021 max_fails=3 fail_timeout=10s;
|
||||
server 10.0.0.176:8021 max_fails=3 fail_timeout=10s;
|
||||
server 10.0.0.44:8021 max_fails=3 fail_timeout=10s;
|
||||
server 10.0.0.7:8021 max_fails=3 fail_timeout=10s;
|
||||
server 10.0.0.128:8021 max_fails=3 fail_timeout=10s;
|
||||
}
|
||||
```
|
||||
|
||||
In the proxy host **Advanced** tab, set:
|
||||
|
||||
```nginx
|
||||
location ^~ / {
|
||||
proxy_pass http://relawfirm_prod;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_set_header X-Forwarded-Host $host;
|
||||
client_max_body_size 64m;
|
||||
}
|
||||
```
|
||||
|
||||
`location ^~ /` is what sends traffic at the upstream. The Details tab can keep `10.0.0.77:8021` as the saved forward target; the `^~` location is the one that matches.
|
||||
|
||||
Reload and check the config from the NPM container:
|
||||
|
||||
```bash
|
||||
docker exec <npm-container> nginx -t
|
||||
docker exec <npm-container> nginx -s reload
|
||||
```
|
||||
|
||||
`nginx -t` must succeed. A duplicate-location error means this Advanced snippet was pasted twice.
|
||||
|
||||
`client_max_body_size 64m` is the upload limit. Raise it in this block if the site needs larger media. A 413 from NPM is this limit, not MariaDB.
|
||||
|
||||
Details tab: SSL on, Force SSL on, HTTP/2 on. Block Common Exploits can stay on.
|
||||
|
||||
### 8. Cron, on adama only
|
||||
|
||||
`DISABLE_WP_CRON` is set, so page views do not run scheduled jobs. One host runs them. Same idea as the dj-queue worker that stays on adama.
|
||||
|
||||
On adama, `/etc/cron.d/wordpress-<site>`:
|
||||
|
||||
```cron
|
||||
* * * * * westfarn curl -fsS https://<prod-domain>/wp-cron.php?doing_wp_cron >/dev/null
|
||||
```
|
||||
|
||||
Beta gets its own cron file hitting the beta URL. Do not add this cron on the other four hosts.
|
||||
|
||||
### 9. Plugins, themes, and upgrades
|
||||
|
||||
Plugins and themes installed from the dashboard are written to the NFS share, so every host sees them. Run one update at a time.
|
||||
|
||||
WordPress core in this setup comes from the `wordpress:6-apache` image, not from the dashboard updater. The dashboard updater writes core into that container only. To move core, change `wordpress_image` (or that entry's `image`) and run `./scripts/provision.sh` so every host gets the new tag. Then, on each host:
|
||||
|
||||
```bash
|
||||
cd /opt/apps/wordpress/<site>
|
||||
docker compose pull
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
The project name is already in the compose file, so `-p` is optional. The database and `wp-content` stay on MariaDB and NFS.
|
||||
|
||||
---
|
||||
|
||||
## Beta
|
||||
|
||||
Repeat sections 2–8 with the beta column of the table:
|
||||
|
||||
- database `<site>_beta`
|
||||
- port **8022** for relawfirm (next free WordPress port is **8025**; **8023/8024** are `disney_hotel_service`)
|
||||
- its own env file, salts, NFS dataset, compose project, NPM host, and adama cron
|
||||
|
||||
Do not point beta `WORDPRESS_DB_NAME` at the prod database.
|
||||
|
||||
---
|
||||
|
||||
## Next site
|
||||
|
||||
MariaDB is already the shared server. For the next site, start at [Per site](#per-site) and do not redo [Already done](#already-done).
|
||||
|
||||
1. Pick a short name, a prod domain, a beta domain, and the next two free host ports. Add a row to the [Ports](#ports) table in this file.
|
||||
2. `CREATE DATABASE` / `GRANT` to `'westfarn'@'%'` for `<site>` and `<site>_beta`.
|
||||
3. New env files and a new salt set per env.
|
||||
4. New NFS datasets, then two entries in [inventory/wordpress_sites.yml](../inventory/wordpress_sites.yml) (`name`, `port`, `url`, `export`, `mount`). Run `./scripts/provision.sh` so every host mounts them and writes compose. Do not reuse another site's `wp-content`.
|
||||
5. Start adama first, browser install once, then the other four.
|
||||
6. New `upstream` block in `http.conf` and a new NPM proxy host.
|
||||
7. New cron file on adama only.
|
||||
|
||||
---
|
||||
|
||||
## Checks
|
||||
|
||||
On each app host:
|
||||
|
||||
```bash
|
||||
docker compose -p <site>_prod ps
|
||||
curl -fsS -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8021/
|
||||
mount | grep wordpress
|
||||
```
|
||||
|
||||
From the control node, each backend should answer:
|
||||
|
||||
```bash
|
||||
for ip in 10.0.0.77 10.0.0.176 10.0.0.44 10.0.0.7 10.0.0.128; do
|
||||
curl -fsS -o /dev/null -w "$ip %{http_code}\n" "http://$ip:8021/"
|
||||
done
|
||||
```
|
||||
|
||||
Then load `https://<prod-domain>` several times. A media file uploaded while you are on the site should still load after a refresh (that request may have been served by a different host).
|
||||
|
||||
## When something fails
|
||||
|
||||
| Symptom | Cause |
|
||||
|---------|--------|
|
||||
| MariaDB `Errcode: 13` on `/var/lib/mysql` | Dataset not writable by uid 999. See [Already done](#already-done). |
|
||||
| `ERROR 1133` on `GRANT` | User host does not match. Use the host from `SELECT user, host FROM mysql.user`. |
|
||||
| `Can't create database; database exists` | Database was created by the TrueNAS wizard or a previous run. Continue at `GRANT`. |
|
||||
| Container exits, cannot connect to MySQL | `WORDPRESS_DB_*` host, user, password, or database name. Port is `10.0.0.230:3306`. |
|
||||
| Installer or login redirects to `http://10.0.0.x:8021` | `WP_HOME` / `WP_SITEURL` are not the public URL, or NPM is not sending `X-Forwarded-Proto`. |
|
||||
| Logged out on refresh | Salts differ between hosts. Recreate with one shared env file. |
|
||||
| Upload succeeds, image 404s on refresh | That host does not have the NFS mount, or the mount is not the one in the compose file. |
|
||||
| `413 Request Entity Too Large` | `client_max_body_size` in the NPM `location ^~ /` block. |
|
||||
| Scheduled posts never publish | Cron is missing on adama, or `wp-cron.php` is not reachable at the public URL. |
|
||||
| Plugin shows on one host only | It was installed onto a local disk instead of the NFS `wp-content`. |
|
||||
@@ -12,19 +12,6 @@ ufw_allowed_tcp_ports:
|
||||
- 80
|
||||
- 443
|
||||
|
||||
# LiveKit on webservers. Signaling stays LAN (NPM → :7880). UDP must be
|
||||
# reachable from the internet via router DNAT to the NPM upstream host.
|
||||
# 8020 is abc_voice. That process uses host networking, so Docker does not
|
||||
# publish the port and UFW drops it unless it is listed here. NPM
|
||||
# (abc.voice.aimloperations.com) reaches it from the LAN.
|
||||
ufw_lan_tcp_ports:
|
||||
- 7880
|
||||
- 7881
|
||||
- 8020
|
||||
ufw_extra_udp_ports:
|
||||
- 3478
|
||||
- "50000:60000"
|
||||
|
||||
# Docker
|
||||
docker_users:
|
||||
- "{{ admin_user }}"
|
||||
@@ -63,7 +50,6 @@ secrets_dir: "{{ lookup('ansible.builtin.env', 'HOME') }}/Documents/secrets"
|
||||
web_static_root: /var/www
|
||||
|
||||
# Catalog of deployable apps. host_apps (per host_vars) references these by name.
|
||||
# Optional host_apps.enabled: false skips deploy (stop.sh still compose-downs).
|
||||
app_catalog:
|
||||
company_site:
|
||||
type: django
|
||||
@@ -94,15 +80,6 @@ app_catalog:
|
||||
webroot_pattern: "/var/www/{env}.realpath.app/html"
|
||||
# deploy runs `npm ci` then `npm run build:<env>`; that script writes to
|
||||
# {{ web_static_root }}/<env>_dta_webapp (beta/prod), served by web-static.
|
||||
dta_blog:
|
||||
type: node-static
|
||||
repo: "{{ git_base_url }}/Ditch_The_Agent/dta_blog.git"
|
||||
default_branch: main
|
||||
# Python SSG (stdlib). package.json build:<env> copies dist/ to this webroot
|
||||
# so the existing node-static path (`npm ci` then `npm run build:<env>`) works.
|
||||
webroot_pattern: "/var/www/{env}.blog.realpath.app/html"
|
||||
# Real 404 page (not SPA index fallback). See roles/web-static nginx.conf.j2.
|
||||
error_page_404: /404.html
|
||||
scha:
|
||||
type: django
|
||||
repo: "{{ git_base_url }}/ai_ml_operations/scha.git"
|
||||
@@ -119,25 +96,6 @@ app_catalog:
|
||||
# Document root served by nginx and written by `npm run build:<env>`.
|
||||
# {env} is replaced with the entry's env (beta/prod).
|
||||
webroot_pattern: "/var/www/{env}.chat.aimloperations/html"
|
||||
# hesychia.ai Stripe/RevenueCat webhooks hit this vhost. Proxy the prefix
|
||||
# to chat_backend so POST is not answered 405 by the SPA index.html.
|
||||
api_backend: chat_backend
|
||||
api_proxy_prefix: /api/finance/webhooks/
|
||||
# /robots.txt, /sitemap.xml, and /llms.txt must not fall through to index.html.
|
||||
exact_root_files: true
|
||||
# Homepage was missing these. Enforcing CSP stays loose so Tianji, the API,
|
||||
# websockets, and inline styles keep working. Report-Only is the strict policy.
|
||||
security_headers: true
|
||||
# Unknown URLs are HTTP 404 with public/404.html, not 200 + the SPA shell.
|
||||
# Slash routes are directory index.html files from scripts/stampCanonicals.js.
|
||||
# These extensionless routes are the matching files (keep in sync with
|
||||
# llm-fe/src/llm-fe/utils/canonicals.js paths that do not end in /).
|
||||
error_page_404: /404.html
|
||||
html_exact_paths:
|
||||
- /legal/terms-of-service
|
||||
- /billing/success
|
||||
- /billing/cancel
|
||||
- /document_storage
|
||||
chat_backend:
|
||||
type: django
|
||||
repo: "{{ git_base_url }}/ai_ml_operations/chat_backend.git"
|
||||
@@ -145,11 +103,6 @@ app_catalog:
|
||||
compose_file: docker-compose.prod.yml
|
||||
web_service: web
|
||||
migrate_cmd: "uv run python manage.py migrate --noinput"
|
||||
# Host header the static-site webhook proxy sends. Must be in Django
|
||||
# ALLOWED_HOSTS (see chat_backend DJANGO_ALLOWED_HOSTS).
|
||||
public_hosts:
|
||||
prod: chatbackend.aimloperations.com
|
||||
beta: beta.chatbackend.aimloperations.com
|
||||
monica_site:
|
||||
type: django
|
||||
repo: "{{ git_base_url }}/ai_ml_operations/monica_site.git"
|
||||
@@ -159,77 +112,6 @@ app_catalog:
|
||||
migrate_cmd: "uv run python manage.py migrate --noinput"
|
||||
# dj-queue worker = compose profile `worker` (singleton). Deploy starts it
|
||||
# via host_apps.compose_profiles on adama only (see host_vars/adama.yml).
|
||||
url_shortening_service:
|
||||
type: django
|
||||
repo: "{{ git_base_url }}/ai_ml_operations/url_shortening_service.git"
|
||||
default_branch: master
|
||||
compose_file: docker-compose.prod.yml
|
||||
web_service: web
|
||||
migrate_cmd: "uv run python manage.py migrate --noinput"
|
||||
# No compose worker profile. Two public NPM hosts share one container/port.
|
||||
college_craft:
|
||||
type: django
|
||||
repo: "{{ git_base_url }}/ai_ml_operations/college_craft.git"
|
||||
default_branch: master
|
||||
compose_file: docker-compose.prod.yml
|
||||
web_service: web
|
||||
migrate_cmd: "uv run python manage.py migrate --noinput"
|
||||
# dj-queue worker = compose profile `worker` (singleton). Deploy starts it
|
||||
# via host_apps.compose_profiles on adama only (see host_vars/adama.yml).
|
||||
print_forge:
|
||||
type: django
|
||||
repo: "{{ git_base_url }}/ai_ml_operations/print_forge.git"
|
||||
default_branch: master
|
||||
compose_file: docker-compose.prod.yml
|
||||
web_service: web
|
||||
migrate_cmd: "uv run python manage.py migrate --noinput"
|
||||
# dj-queue worker = compose profile `worker` (singleton). Deploy starts it
|
||||
# via host_apps.compose_profiles on adama only (see host_vars/adama.yml).
|
||||
abc_be:
|
||||
type: django
|
||||
repo: "{{ git_base_url }}/GIS/abc_be.git"
|
||||
default_branch: Development
|
||||
compose_file: docker-compose.prod.yml
|
||||
web_service: web
|
||||
migrate_cmd: "uv run python manage.py migrate --noinput"
|
||||
abc_worker:
|
||||
type: django
|
||||
repo: "{{ git_base_url }}/GIS/abc_worker.git"
|
||||
default_branch: Development
|
||||
compose_file: docker-compose.prod.yml
|
||||
web_service: web
|
||||
# FastAPI — no Django migrations. `true` keeps the django deploy path happy.
|
||||
migrate_cmd: "true"
|
||||
# Same git repo as abc_worker, second container (voice join). Not the text worker.
|
||||
abc_voice:
|
||||
type: django
|
||||
repo: "{{ git_base_url }}/GIS/abc_worker.git"
|
||||
default_branch: Development
|
||||
compose_file: docker-compose.voice.yml
|
||||
web_service: web
|
||||
migrate_cmd: "true"
|
||||
abc_fe:
|
||||
type: node-static
|
||||
repo: "{{ git_base_url }}/GIS/abc_fe.git"
|
||||
default_branch: Development
|
||||
webroot_pattern: "/var/www/{env}.abc.aimloperations/html"
|
||||
livekit:
|
||||
type: django
|
||||
repo: "{{ git_base_url }}/GIS/livekit.git"
|
||||
default_branch: master
|
||||
compose_file: docker-compose.prod.yml
|
||||
web_service: web
|
||||
# SFU — no Django migrations. `true` keeps the django deploy path happy.
|
||||
migrate_cmd: "true"
|
||||
disney_hotel_service:
|
||||
type: django
|
||||
repo: "{{ git_base_url }}/DisneyHotel/disney_hotel_service.git"
|
||||
default_branch: master
|
||||
compose_file: docker-compose.prod.yml
|
||||
web_service: web
|
||||
migrate_cmd: "uv run python manage.py migrate --noinput"
|
||||
# Alert worker (db_worker + enqueue_due_alerts) = compose profile `worker`
|
||||
# (singleton). Deploy starts it via host_apps.compose_profiles on adama only.
|
||||
|
||||
# Deploy filter vars. CI passes these; manual runs may leave them undefined
|
||||
# to (re)deploy every app listed in the host's host_apps.
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
---
|
||||
# Django services run active/active here and on roslin/starbuck/apollo
|
||||
# (shared external DB). dta_webapp static also runs active/active (built into
|
||||
# /var/www, served by the web-static nginx container). Ports MUST match the
|
||||
# other app hosts so NPM can balance.
|
||||
# Django services run active/active here and on roslin (shared external DB).
|
||||
# dta_webapp static also runs active/active (built into /var/www, served by
|
||||
# the web-static nginx container). Ports MUST match roslin so NPM can balance.
|
||||
# Each entry is one workload: django -> compose project <name>_<env> on port;
|
||||
# node-static -> /var/www/<env>_dta_webapp served on port.
|
||||
host_apps:
|
||||
@@ -11,29 +10,12 @@ host_apps:
|
||||
- { name: dta_service, env: beta, port: 8011 }
|
||||
- { name: dta_webapp, env: prod, port: 8080 }
|
||||
- { name: dta_webapp, env: beta, port: 8081 }
|
||||
- { name: dta_blog, env: prod, port: 8086 }
|
||||
- { name: dta_blog, env: beta, port: 8087 }
|
||||
- { name: scha, env: prod, port: 8002 }
|
||||
# optional: - { name: scha, env: beta, port: 8012 }
|
||||
- { name: chat_web_app, env: prod, port: 8082 }
|
||||
- { name: chat_web_app, env: beta, port: 8083 }
|
||||
- { name: chat_backend, env: prod, port: 8003 }
|
||||
- { name: chat_backend, env: beta, port: 8013 }
|
||||
# compose_profiles: worker → dj-queue singleton (not on other app hosts).
|
||||
# compose_profiles: worker → dj-queue singleton (not on roslin / ai-server-4080).
|
||||
- { name: monica_site, env: prod, port: 8004, compose_profiles: [worker] }
|
||||
- { name: monica_site, env: beta, port: 8014, compose_profiles: [worker] }
|
||||
- { name: url_shortening_service, env: prod, port: 8005 }
|
||||
- { name: url_shortening_service, env: beta, port: 8015 }
|
||||
- { name: college_craft, env: prod, port: 8006, compose_profiles: [worker] }
|
||||
- { name: college_craft, env: beta, port: 8016, compose_profiles: [worker] }
|
||||
# print_forge stopped (#33) — enabled: false keeps ports; stop.sh still matches.
|
||||
- { name: print_forge, env: prod, port: 8007, compose_profiles: [worker], enabled: false }
|
||||
- { name: print_forge, env: beta, port: 8019, compose_profiles: [worker], enabled: false }
|
||||
- { name: abc_be, env: beta, port: 8017 }
|
||||
- { name: abc_worker, env: beta, port: 8018 }
|
||||
- { name: abc_voice, env: beta, port: 8020 }
|
||||
- { name: abc_fe, env: beta, port: 8085 }
|
||||
- { name: livekit, env: beta, port: 7880 }
|
||||
# Alert worker singleton (db_worker). Other hosts omit compose_profiles.
|
||||
- { name: disney_hotel_service, env: prod, port: 8023, compose_profiles: [worker] }
|
||||
- { name: disney_hotel_service, env: beta, port: 8024, compose_profiles: [worker] }
|
||||
@@ -22,13 +22,11 @@ searxng_stack: true
|
||||
gitea_key_path: "/home/{{ admin_user }}/.ssh/gitea_deploy"
|
||||
|
||||
# company_site + dta_webapp + dta_service for side testing and active/active.
|
||||
# Ports MUST match adama/roslin/starbuck/apollo so NPM can balance all upstreams.
|
||||
# Ports MUST match adama/roslin so NPM can balance all three upstreams.
|
||||
host_apps:
|
||||
- { name: company_site, env: prod, port: 8000 }
|
||||
- { name: dta_webapp, env: prod, port: 8080 }
|
||||
- { name: dta_webapp, env: beta, port: 8081 }
|
||||
- { name: dta_blog, env: prod, port: 8086 }
|
||||
- { name: dta_blog, env: beta, port: 8087 }
|
||||
- { name: dta_service, env: prod, port: 8001 }
|
||||
- { name: dta_service, env: beta, port: 8011 }
|
||||
- { name: scha, env: prod, port: 8002 }
|
||||
@@ -36,19 +34,6 @@ host_apps:
|
||||
- { name: chat_web_app, env: beta, port: 8083 }
|
||||
- { name: chat_backend, env: prod, port: 8003 }
|
||||
- { name: chat_backend, env: beta, port: 8013 }
|
||||
# monica_site / college_craft / print_forge / disney_hotel_service workers NOT here — adama sets compose_profiles: [worker].
|
||||
# monica_site worker NOT here — adama host_apps sets compose_profiles: [worker].
|
||||
- { name: monica_site, env: prod, port: 8004 }
|
||||
- { name: monica_site, env: beta, port: 8014 }
|
||||
- { name: url_shortening_service, env: prod, port: 8005 }
|
||||
- { name: url_shortening_service, env: beta, port: 8015 }
|
||||
- { name: college_craft, env: prod, port: 8006 }
|
||||
- { name: college_craft, env: beta, port: 8016 }
|
||||
- { name: print_forge, env: prod, port: 8007, enabled: false }
|
||||
- { name: print_forge, env: beta, port: 8019, enabled: false }
|
||||
- { name: abc_be, env: beta, port: 8017 }
|
||||
- { name: abc_worker, env: beta, port: 8018 }
|
||||
- { name: abc_voice, env: beta, port: 8020 }
|
||||
- { name: abc_fe, env: beta, port: 8085 }
|
||||
- { name: livekit, env: beta, port: 7880 }
|
||||
- { name: disney_hotel_service, env: prod, port: 8023 }
|
||||
- { name: disney_hotel_service, env: beta, port: 8024 }
|
||||
@@ -1,33 +0,0 @@
|
||||
---
|
||||
# Mirrors roslin for active/active. Ports MUST match adama/roslin so NPM
|
||||
# upstreams can balance apollo:PORT with the other app hosts.
|
||||
host_apps:
|
||||
- { name: company_site, env: prod, port: 8000 }
|
||||
- { name: dta_service, env: prod, port: 8001 }
|
||||
- { name: dta_service, env: beta, port: 8011 }
|
||||
- { name: dta_webapp, env: prod, port: 8080 }
|
||||
- { name: dta_webapp, env: beta, port: 8081 }
|
||||
- { name: dta_blog, env: prod, port: 8086 }
|
||||
- { name: dta_blog, env: beta, port: 8087 }
|
||||
- { name: scha, env: prod, port: 8002 }
|
||||
# optional: - { name: scha, env: beta, port: 8012 }
|
||||
- { name: chat_web_app, env: prod, port: 8082 }
|
||||
- { name: chat_web_app, env: beta, port: 8083 }
|
||||
- { name: chat_backend, env: prod, port: 8003 }
|
||||
- { name: chat_backend, env: beta, port: 8013 }
|
||||
# monica_site / college_craft / print_forge / disney_hotel_service workers NOT here — adama sets compose_profiles: [worker].
|
||||
- { name: monica_site, env: prod, port: 8004 }
|
||||
- { name: monica_site, env: beta, port: 8014 }
|
||||
- { name: url_shortening_service, env: prod, port: 8005 }
|
||||
- { name: url_shortening_service, env: beta, port: 8015 }
|
||||
- { name: college_craft, env: prod, port: 8006 }
|
||||
- { name: college_craft, env: beta, port: 8016 }
|
||||
- { name: print_forge, env: prod, port: 8007, enabled: false }
|
||||
- { name: print_forge, env: beta, port: 8019, enabled: false }
|
||||
- { name: abc_be, env: beta, port: 8017 }
|
||||
- { name: abc_worker, env: beta, port: 8018 }
|
||||
- { name: abc_voice, env: beta, port: 8020 }
|
||||
- { name: abc_fe, env: beta, port: 8085 }
|
||||
- { name: livekit, env: beta, port: 7880 }
|
||||
- { name: disney_hotel_service, env: prod, port: 8023 }
|
||||
- { name: disney_hotel_service, env: beta, port: 8024 }
|
||||
@@ -1,33 +1,18 @@
|
||||
---
|
||||
# Mirrors adama for active/active. Ports MUST match adama/starbuck/apollo so
|
||||
# NPM upstreams can balance roslin:PORT with the other app hosts.
|
||||
# Mirrors adama for active/active. Ports MUST match adama so NPM upstreams
|
||||
# can balance adama:PORT and roslin:PORT for the same workload.
|
||||
host_apps:
|
||||
- { name: company_site, env: prod, port: 8000 }
|
||||
- { name: dta_service, env: prod, port: 8001 }
|
||||
- { name: dta_service, env: beta, port: 8011 }
|
||||
- { name: dta_webapp, env: prod, port: 8080 }
|
||||
- { name: dta_webapp, env: beta, port: 8081 }
|
||||
- { name: dta_blog, env: prod, port: 8086 }
|
||||
- { name: dta_blog, env: beta, port: 8087 }
|
||||
- { name: scha, env: prod, port: 8002 }
|
||||
# optional: - { name: scha, env: beta, port: 8012 }
|
||||
- { name: chat_web_app, env: prod, port: 8082 }
|
||||
- { name: chat_web_app, env: beta, port: 8083 }
|
||||
- { name: chat_backend, env: prod, port: 8003 }
|
||||
- { name: chat_backend, env: beta, port: 8013 }
|
||||
# monica_site / college_craft / print_forge / disney_hotel_service workers NOT here — adama sets compose_profiles: [worker].
|
||||
# monica_site worker NOT here — adama host_apps sets compose_profiles: [worker].
|
||||
- { name: monica_site, env: prod, port: 8004 }
|
||||
- { name: monica_site, env: beta, port: 8014 }
|
||||
- { name: url_shortening_service, env: prod, port: 8005 }
|
||||
- { name: url_shortening_service, env: beta, port: 8015 }
|
||||
- { name: college_craft, env: prod, port: 8006 }
|
||||
- { name: college_craft, env: beta, port: 8016 }
|
||||
- { name: print_forge, env: prod, port: 8007, enabled: false }
|
||||
- { name: print_forge, env: beta, port: 8019, enabled: false }
|
||||
- { name: abc_be, env: beta, port: 8017 }
|
||||
- { name: abc_worker, env: beta, port: 8018 }
|
||||
- { name: abc_voice, env: beta, port: 8020 }
|
||||
- { name: abc_fe, env: beta, port: 8085 }
|
||||
- { name: livekit, env: beta, port: 7880 }
|
||||
- { name: disney_hotel_service, env: prod, port: 8023 }
|
||||
- { name: disney_hotel_service, env: beta, port: 8024 }
|
||||
@@ -1,33 +0,0 @@
|
||||
---
|
||||
# Mirrors roslin for active/active. Ports MUST match adama/roslin so NPM
|
||||
# upstreams can balance starbuck:PORT with the other app hosts.
|
||||
host_apps:
|
||||
- { name: company_site, env: prod, port: 8000 }
|
||||
- { name: dta_service, env: prod, port: 8001 }
|
||||
- { name: dta_service, env: beta, port: 8011 }
|
||||
- { name: dta_webapp, env: prod, port: 8080 }
|
||||
- { name: dta_webapp, env: beta, port: 8081 }
|
||||
- { name: dta_blog, env: prod, port: 8086 }
|
||||
- { name: dta_blog, env: beta, port: 8087 }
|
||||
- { name: scha, env: prod, port: 8002 }
|
||||
# optional: - { name: scha, env: beta, port: 8012 }
|
||||
- { name: chat_web_app, env: prod, port: 8082 }
|
||||
- { name: chat_web_app, env: beta, port: 8083 }
|
||||
- { name: chat_backend, env: prod, port: 8003 }
|
||||
- { name: chat_backend, env: beta, port: 8013 }
|
||||
# monica_site / college_craft / print_forge / disney_hotel_service workers NOT here — adama sets compose_profiles: [worker].
|
||||
- { name: monica_site, env: prod, port: 8004 }
|
||||
- { name: monica_site, env: beta, port: 8014 }
|
||||
- { name: url_shortening_service, env: prod, port: 8005 }
|
||||
- { name: url_shortening_service, env: beta, port: 8015 }
|
||||
- { name: college_craft, env: prod, port: 8006 }
|
||||
- { name: college_craft, env: beta, port: 8016 }
|
||||
- { name: print_forge, env: prod, port: 8007, enabled: false }
|
||||
- { name: print_forge, env: beta, port: 8019, enabled: false }
|
||||
- { name: abc_be, env: beta, port: 8017 }
|
||||
- { name: abc_worker, env: beta, port: 8018 }
|
||||
- { name: abc_voice, env: beta, port: 8020 }
|
||||
- { name: abc_fe, env: beta, port: 8085 }
|
||||
- { name: livekit, env: beta, port: 7880 }
|
||||
- { name: disney_hotel_service, env: prod, port: 8023 }
|
||||
- { name: disney_hotel_service, env: beta, port: 8024 }
|
||||
@@ -7,9 +7,5 @@ all:
|
||||
ansible_host: 10.0.0.77
|
||||
roslin:
|
||||
ansible_host: 10.0.0.176
|
||||
starbuck:
|
||||
ansible_host: 10.0.0.44
|
||||
apollo:
|
||||
ansible_host: 10.0.0.7
|
||||
ai-server-4080:
|
||||
ansible_host: 10.0.0.128
|
||||
@@ -1,32 +0,0 @@
|
||||
---
|
||||
# WordPress sites on every webserver. scripts/provision.sh mounts wp-content
|
||||
# and writes one compose file per entry. Do not hand-edit /etc/fstab or the
|
||||
# generated compose file on the hosts.
|
||||
#
|
||||
# name: site key. Prod is the short name (relawfirm). Beta is that name
|
||||
# plus _beta (relawfirm_beta). Names use lowercase, numbers, underscore.
|
||||
# export: dataset path on the TrueNAS NFS server
|
||||
# mount: local path compose bind-mounts to /var/www/html/wp-content
|
||||
# port: host port published to container port 80
|
||||
# url: public https URL (WP_HOME and WP_SITEURL). Empty skips compose.
|
||||
#
|
||||
# From name, provision derives:
|
||||
# relawfirm -> /opt/apps/wordpress/relawfirm/docker-compose.yml
|
||||
# project relawfirm_prod, env /opt/apps/env/relawfirm_prod.env
|
||||
# relawfirm_beta -> /opt/apps/wordpress/relawfirm-beta/docker-compose.yml
|
||||
# project relawfirm_beta, env /opt/apps/env/relawfirm_beta.env
|
||||
|
||||
wordpress_nfs_server: 10.0.0.230
|
||||
wordpress_nfs_opts: "nfsvers=4,tcp,_netdev"
|
||||
|
||||
wordpress_sites:
|
||||
- name: relawfirm
|
||||
port: 8021
|
||||
url: ""
|
||||
export: /mnt/AIMLOperationsNAS/wordpress/relawfirm
|
||||
mount: /mnt/wordpress/relawfirm/wp-content
|
||||
- name: relawfirm_beta
|
||||
port: 8022
|
||||
url: http://relawfirm-preview.aimloperations.com
|
||||
export: /mnt/AIMLOperationsNAS/wordpress/relawfirm_beta
|
||||
mount: /mnt/wordpress/relawfirm_beta/wp-content
|
||||
@@ -1,9 +1,8 @@
|
||||
---
|
||||
# Phase 2: CI-triggered app deployment.
|
||||
#
|
||||
# Runs against all webservers; each host deploys only enabled apps listed in
|
||||
# its host_apps (enabled: false is skipped). Deploy exactly one app+env at a
|
||||
# pinned ref via extra-vars:
|
||||
# Runs against all webservers; each host deploys only the apps listed in its
|
||||
# host_apps. Deploy exactly one app+env at a pinned ref via extra-vars:
|
||||
#
|
||||
# ansible-playbook playbooks/deploy-apps.yml \
|
||||
# -e app=company_site -e app_env=prod -e app_ref=<sha>
|
||||
|
||||
@@ -13,9 +13,6 @@
|
||||
- nodejs
|
||||
- gitea-key
|
||||
- tianji
|
||||
# NFS wp-content and generated compose for inventory/wordpress_sites.yml.
|
||||
- role: wordpress
|
||||
tags: [wordpress]
|
||||
|
||||
- name: Provision central observability stack
|
||||
hosts: ai-server-4080
|
||||
|
||||
@@ -1,15 +0,0 @@
|
||||
---
|
||||
# Stop one docker-compose app+env. Inverse of deploy-apps.yml start.
|
||||
#
|
||||
# ansible-playbook playbooks/stop-apps.yml -e app=print_forge -e app_env=prod
|
||||
#
|
||||
# host_apps rows with enabled: false are still valid stop targets (so a
|
||||
# disabled app can be taken down). Node-static apps are skipped.
|
||||
- name: Stop applications
|
||||
hosts: webservers
|
||||
become: true
|
||||
tasks:
|
||||
- name: Stop requested compose apps
|
||||
ansible.builtin.include_role:
|
||||
name: app-deploy
|
||||
tasks_from: stop.yml
|
||||
@@ -2,5 +2,3 @@
|
||||
collections:
|
||||
- name: community.general
|
||||
version: ">=8.0.0"
|
||||
- name: ansible.posix
|
||||
version: ">=1.5.0"
|
||||
@@ -104,9 +104,8 @@
|
||||
- _legacy_systemd_probe.stdout | default('') | length > 0
|
||||
|
||||
# Optional host_apps.compose_profiles (e.g. [worker]) activates compose profiles
|
||||
# on this host only. Used for monica_site / college_craft / print_forge /
|
||||
# disney_hotel_service worker singleton on adama — without it, plain up
|
||||
# --remove-orphans can drop a manually started worker.
|
||||
# on this host only. Used for monica_site dj-queue singleton on adama — without
|
||||
# it, plain up --remove-orphans can drop a manually started worker.
|
||||
- name: "django[{{ _project }}] start containers"
|
||||
ansible.builtin.command:
|
||||
cmd: "docker compose -f {{ _spec.compose_file }} --env-file .env up -d --remove-orphans"
|
||||
|
||||
@@ -6,21 +6,24 @@
|
||||
# at branch master.
|
||||
#
|
||||
# host_apps (host_vars) lists what runs on THIS host; app_catalog (group_vars)
|
||||
# describes how each app is built. host_apps.enabled: false (omit = true)
|
||||
# keeps the row for ports/stop.sh but skips deploy so CI cannot resurrect it.
|
||||
# describes how each app is built.
|
||||
|
||||
- name: Classify catalog by type
|
||||
ansible.builtin.set_fact:
|
||||
django_names: "{{ app_catalog | dict2items | selectattr('value.type', 'equalto', 'django') | map(attribute='key') | list }}"
|
||||
node_names: "{{ app_catalog | dict2items | selectattr('value.type', 'equalto', 'node-static') | map(attribute='key') | list }}"
|
||||
|
||||
- name: Resolve matching host_apps for {{ inventory_hostname }}
|
||||
- name: Resolve deploy targets for {{ inventory_hostname }}
|
||||
ansible.builtin.set_fact:
|
||||
matching_host_apps: >-
|
||||
deploy_targets: >-
|
||||
{{ (host_apps | default([]) | selectattr('name', 'equalto', app | default('')) | selectattr('env', 'equalto', app_env | default('')) | list)
|
||||
if (app is defined and app_env is defined)
|
||||
else (host_apps | default([])) }}
|
||||
|
||||
- name: Show deploy targets
|
||||
ansible.builtin.debug:
|
||||
msg: "ref={{ app_ref | default('(per-app default branch)') }} targets={{ deploy_targets | map(attribute='name') | zip(deploy_targets | map(attribute='env')) | list }}"
|
||||
|
||||
# Explicit --app/--env with no host_apps match used to silently no-op Django
|
||||
# and still refresh web-static (looked like a "static-only" deploy). Fail loud.
|
||||
- name: Fail when requested app+env is not on this host
|
||||
@@ -33,32 +36,7 @@
|
||||
when:
|
||||
- app is defined
|
||||
- app_env is defined
|
||||
- matching_host_apps | length == 0
|
||||
|
||||
# rejectattr('enabled', …) raises when the key is omitted. Omit = enabled (#35).
|
||||
- name: Reset deploy targets
|
||||
ansible.builtin.set_fact:
|
||||
deploy_targets: []
|
||||
|
||||
- name: "Drop disabled host_apps (enabled: false)"
|
||||
ansible.builtin.set_fact:
|
||||
deploy_targets: "{{ deploy_targets + [item] }}"
|
||||
loop: "{{ matching_host_apps }}"
|
||||
when: item.enabled | default(true)
|
||||
loop_control:
|
||||
label: "{{ item.name }}/{{ item.env }}"
|
||||
|
||||
- name: Show skipped disabled apps
|
||||
ansible.builtin.debug:
|
||||
msg: "skip disabled {{ item.name }}/{{ item.env }} on {{ inventory_hostname }}"
|
||||
loop: "{{ matching_host_apps }}"
|
||||
when: not (item.enabled | default(true))
|
||||
loop_control:
|
||||
label: "{{ item.name }}/{{ item.env }}"
|
||||
|
||||
- name: Show deploy targets
|
||||
ansible.builtin.debug:
|
||||
msg: "ref={{ app_ref | default('(per-app default branch)') }} targets={{ deploy_targets | map(attribute='name') | zip(deploy_targets | map(attribute='env')) | list }}"
|
||||
- deploy_targets | length == 0
|
||||
|
||||
- name: Fail when requested app missing from app_catalog
|
||||
ansible.builtin.fail:
|
||||
|
||||
@@ -1,47 +0,0 @@
|
||||
---
|
||||
# Stop one app+env. CI/manual must pass: app=<name> app_env=<beta|prod>
|
||||
#
|
||||
# Matches host_apps including enabled: false. Hosts without that row skip.
|
||||
# Django/compose → docker compose down (no -v). Node-static → message only.
|
||||
|
||||
- name: Fail unless app and app_env extra-vars set
|
||||
ansible.builtin.fail:
|
||||
msg: "stop-apps.yml requires -e app=<name> -e app_env=<beta|prod>"
|
||||
when: app is not defined or app_env is not defined
|
||||
|
||||
- name: Classify catalog by type
|
||||
ansible.builtin.set_fact:
|
||||
django_names: "{{ app_catalog | dict2items | selectattr('value.type', 'equalto', 'django') | map(attribute='key') | list }}"
|
||||
node_names: "{{ app_catalog | dict2items | selectattr('value.type', 'equalto', 'node-static') | map(attribute='key') | list }}"
|
||||
|
||||
- name: Fail when requested app missing from app_catalog
|
||||
ansible.builtin.fail:
|
||||
msg: >-
|
||||
app={{ app }} is not in app_catalog. Known: {{ app_catalog.keys() | list }}.
|
||||
when: app not in app_catalog
|
||||
|
||||
- name: Resolve stop targets for {{ inventory_hostname }}
|
||||
ansible.builtin.set_fact:
|
||||
stop_targets: "{{ host_apps | default([]) | selectattr('name', 'equalto', app) | selectattr('env', 'equalto', app_env) | list }}"
|
||||
|
||||
- name: Skip {{ inventory_hostname }} — no host_apps row for {{ app }}/{{ app_env }}
|
||||
ansible.builtin.debug:
|
||||
msg: "skip {{ inventory_hostname }}: no host_apps entry for app={{ app }} env={{ app_env }}"
|
||||
when: stop_targets | length == 0
|
||||
|
||||
- name: Skip node-static {{ app }}/{{ app_env }} (no compose project)
|
||||
ansible.builtin.debug:
|
||||
msg: >-
|
||||
{{ app }} is node-static. stop.sh only compose-downs Django apps.
|
||||
Drop the host_apps row (or set enabled: false) and re-run deploy.sh
|
||||
so web-static regenerates nginx without that vhost.
|
||||
when:
|
||||
- stop_targets | length > 0
|
||||
- app in node_names
|
||||
|
||||
- name: Stop Django (docker compose) apps
|
||||
ansible.builtin.include_tasks: stop_django.yml
|
||||
loop: "{{ stop_targets | selectattr('name', 'in', django_names) | list }}"
|
||||
loop_control:
|
||||
loop_var: app_item
|
||||
label: "{{ app_item.name }}/{{ app_item.env }}"
|
||||
@@ -1,72 +0,0 @@
|
||||
---
|
||||
# Compose-down one Django app+env. Called per item with loop_var app_item.
|
||||
# No `down -v` — volumes stay. Checkout and secrets stay.
|
||||
|
||||
- name: "stop[{{ app_item.name }}/{{ app_item.env }}] locals"
|
||||
ansible.builtin.set_fact:
|
||||
_spec: "{{ app_catalog[app_item.name] }}"
|
||||
_src: "{{ apps_src_dir }}/{{ app_item.name }}_{{ app_item.env }}"
|
||||
_project: "{{ app_item.name }}_{{ app_item.env }}"
|
||||
|
||||
- name: "stop[{{ _project }}] checkout dir"
|
||||
ansible.builtin.stat:
|
||||
path: "{{ _src }}"
|
||||
register: _src_stat
|
||||
|
||||
- name: "stop[{{ _project }}] compose file"
|
||||
ansible.builtin.stat:
|
||||
path: "{{ _src }}/{{ _spec.compose_file }}"
|
||||
register: _compose_stat
|
||||
when: _src_stat.stat.exists
|
||||
|
||||
- name: "stop[{{ _project }}] env file"
|
||||
ansible.builtin.stat:
|
||||
path: "{{ _src }}/.env"
|
||||
register: _env_stat
|
||||
when: _src_stat.stat.exists
|
||||
|
||||
- name: "stop[{{ _project }}] use compose down"
|
||||
ansible.builtin.set_fact:
|
||||
_use_compose: "{{ _src_stat.stat.exists and _compose_stat is defined and _compose_stat.stat.exists }}"
|
||||
|
||||
- name: "stop[{{ _project }}] compose down"
|
||||
ansible.builtin.command:
|
||||
cmd: >-
|
||||
docker compose -f {{ _spec.compose_file }}
|
||||
{{ '--env-file .env' if (_env_stat is defined and _env_stat.stat.exists) else '' }}
|
||||
down
|
||||
chdir: "{{ _src }}"
|
||||
environment:
|
||||
COMPOSE_PROJECT_NAME: "{{ _project }}"
|
||||
WEB_PORT: "{{ app_item.port }}"
|
||||
COMPOSE_PROFILES: "{{ (app_item.compose_profiles | default([])) | join(',') }}"
|
||||
become: true
|
||||
become_user: "{{ admin_user }}"
|
||||
when: _use_compose | bool
|
||||
changed_when: true
|
||||
|
||||
- name: "stop[{{ _project }}] find leftover project containers"
|
||||
ansible.builtin.command:
|
||||
cmd: docker ps -aq --filter label=com.docker.compose.project={{ _project }}
|
||||
register: _leftovers
|
||||
changed_when: false
|
||||
become: true
|
||||
when: not (_use_compose | bool)
|
||||
|
||||
- name: "stop[{{ _project }}] remove leftover project containers"
|
||||
ansible.builtin.command:
|
||||
cmd: "docker rm -f {{ _leftovers.stdout_lines | join(' ') }}"
|
||||
become: true
|
||||
when:
|
||||
- not (_use_compose | bool)
|
||||
- _leftovers is defined
|
||||
- _leftovers.stdout_lines | length > 0
|
||||
changed_when: true
|
||||
|
||||
- name: "stop[{{ _project }}] nothing to stop (no checkout)"
|
||||
ansible.builtin.debug:
|
||||
msg: "no checkout at {{ _src }} and no containers for project {{ _project }}"
|
||||
when:
|
||||
- not (_use_compose | bool)
|
||||
- _leftovers is defined
|
||||
- _leftovers.stdout_lines | length == 0
|
||||
@@ -23,5 +23,4 @@
|
||||
- gnupg
|
||||
- apt-transport-https
|
||||
- software-properties-common
|
||||
- nfs-common
|
||||
state: present
|
||||
@@ -28,28 +28,6 @@
|
||||
proto: tcp
|
||||
loop: "{{ ufw_allowed_tcp_ports }}"
|
||||
|
||||
- name: Allow extra TCP ports
|
||||
community.general.ufw:
|
||||
rule: allow
|
||||
port: "{{ item }}"
|
||||
proto: tcp
|
||||
loop: "{{ ufw_extra_tcp_ports | default([]) }}"
|
||||
|
||||
- name: Allow extra TCP ports from LAN
|
||||
community.general.ufw:
|
||||
rule: allow
|
||||
port: "{{ item }}"
|
||||
proto: tcp
|
||||
from_ip: "{{ ufw_ssh_allowed_network }}"
|
||||
loop: "{{ ufw_lan_tcp_ports | default([]) }}"
|
||||
|
||||
- name: Allow extra UDP ports or ranges
|
||||
community.general.ufw:
|
||||
rule: allow
|
||||
port: "{{ item }}"
|
||||
proto: udp
|
||||
loop: "{{ ufw_extra_udp_ports | default([]) }}"
|
||||
|
||||
- name: Enable UFW
|
||||
community.general.ufw:
|
||||
state: enabled
|
||||
@@ -6,20 +6,9 @@
|
||||
ansible.builtin.set_fact:
|
||||
_node_names: "{{ app_catalog | dict2items | selectattr('value.type', 'equalto', 'node-static') | map(attribute='key') | list }}"
|
||||
|
||||
# rejectattr('enabled', …) raises when the key is omitted. Omit = enabled (#35).
|
||||
- name: web-static | reset static apps
|
||||
ansible.builtin.set_fact:
|
||||
_static_apps: []
|
||||
|
||||
- name: web-static | this host's static apps
|
||||
ansible.builtin.set_fact:
|
||||
_static_apps: "{{ _static_apps + [item] }}"
|
||||
loop: "{{ host_apps | default([]) }}"
|
||||
when:
|
||||
- item.name in _node_names
|
||||
- item.enabled | default(true)
|
||||
loop_control:
|
||||
label: "{{ item.name }}/{{ item.env }}"
|
||||
_static_apps: "{{ host_apps | default([]) | selectattr('name', 'in', _node_names) | list }}"
|
||||
|
||||
- name: web-static | configure and run
|
||||
when: _static_apps | length > 0
|
||||
|
||||
@@ -3,9 +3,6 @@ services:
|
||||
web-static:
|
||||
image: {{ web_static_image }}
|
||||
restart: unless-stopped
|
||||
# Webhook locations proxy to chat_backend on the host-published port.
|
||||
extra_hosts:
|
||||
- "host.docker.internal:host-gateway"
|
||||
logging:
|
||||
driver: json-file
|
||||
options:
|
||||
|
||||
@@ -17,14 +17,6 @@ log_format observability escape=json
|
||||
'"user_agent":"$http_user_agent"'
|
||||
'}';
|
||||
|
||||
{% macro security_headers() %}
|
||||
add_header X-Content-Type-Options "nosniff" always;
|
||||
add_header X-Frame-Options "DENY" always;
|
||||
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
|
||||
add_header Content-Security-Policy "default-src 'self' https: wss: data: blob: 'unsafe-inline'; frame-ancestors 'none'; base-uri 'self'; object-src 'none'" always;
|
||||
add_header Content-Security-Policy-Report-Only "default-src 'self'; style-src 'self' 'unsafe-inline'; img-src 'self' data: blob:; font-src 'self' data:; connect-src 'self'; frame-ancestors 'none'; base-uri 'self'; object-src 'none'; form-action 'self'" always;
|
||||
{% endmacro %}
|
||||
|
||||
{% for a in _static_apps %}
|
||||
server {
|
||||
listen {{ a.port }};
|
||||
@@ -37,92 +29,14 @@ server {
|
||||
|
||||
root {{ (app_catalog[a.name].webroot_pattern | default(web_static_root ~ '/{env}_' ~ a.name)) | replace('{env}', a.env) }};
|
||||
index index.html;
|
||||
{% if app_catalog[a.name].security_headers | default(false) %}
|
||||
{{ security_headers() }}
|
||||
{% endif %}
|
||||
|
||||
{%- set api_backend_name = app_catalog[a.name].api_backend | default('') %}
|
||||
{%- if api_backend_name %}
|
||||
{%- set api_proxy_prefix = app_catalog[a.name].api_proxy_prefix | default('/api/') %}
|
||||
{%- set api_match = namespace(port=0, ok=false) %}
|
||||
{%- for item in host_apps | default([]) %}
|
||||
{%- if item.name == api_backend_name and item.env == a.env and (item.enabled | default(true)) %}
|
||||
{%- set api_match.port = item.port %}
|
||||
{%- set api_match.ok = true %}
|
||||
{%- endif %}
|
||||
{%- endfor %}
|
||||
{%- if api_match.ok %}
|
||||
{%- set api_host = (app_catalog[api_backend_name].public_hosts | default({}))[a.env] | default('') %}
|
||||
|
||||
# Finance webhooks (Stripe) arrive on the public product host. The SPA
|
||||
# try_files fallback is a static file and returns 405 for POST.
|
||||
location {{ api_proxy_prefix }} {
|
||||
proxy_pass http://host.docker.internal:{{ api_match.port }};
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Host {{ api_host if api_host else '$host' }};
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_read_timeout 60s;
|
||||
client_max_body_size 2m;
|
||||
}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
|
||||
{% if app_catalog[a.name].exact_root_files | default(false) %}
|
||||
# Crawlers must get these files, never the SPA index.html fallback.
|
||||
location = /robots.txt {
|
||||
default_type text/plain;
|
||||
try_files /robots.txt =404;
|
||||
}
|
||||
location = /sitemap.xml {
|
||||
default_type application/xml;
|
||||
try_files /sitemap.xml =404;
|
||||
}
|
||||
location = /llms.txt {
|
||||
default_type text/plain;
|
||||
try_files /llms.txt =404;
|
||||
}
|
||||
{% endif %}
|
||||
|
||||
{% if app_catalog[a.name].html_exact_paths | default([]) %}
|
||||
{% for p in app_catalog[a.name].html_exact_paths %}
|
||||
location = {{ p }} {
|
||||
default_type text/html;
|
||||
try_files {{ p }} =404;
|
||||
}
|
||||
{% endfor %}
|
||||
{% endif %}
|
||||
|
||||
{% if a.name == 'chat_web_app' and a.env == 'prod' %}
|
||||
# HeyCatch short links.
|
||||
# f Facebook, h Threads, i Instagram, l LinkedIn, r Reddit, x X, y YouTube.
|
||||
# Relative Location: this vhost listens on the container port. An absolute
|
||||
# redirect would publish that port (hesychia.ai:8082).
|
||||
location ~ ^/([a-z0-9])/?$ {
|
||||
absolute_redirect off;
|
||||
return 302 /?utm_source=heycatch&utm_campaign=$1;
|
||||
}
|
||||
{% endif %}
|
||||
|
||||
{% if app_catalog[a.name].error_page_404 | default('') %}
|
||||
error_page 404 {{ app_catalog[a.name].error_page_404 }};
|
||||
|
||||
location / {
|
||||
try_files $uri $uri/ =404;
|
||||
}
|
||||
{% else %}
|
||||
location / {
|
||||
try_files $uri $uri/ /index.html;
|
||||
}
|
||||
{% endif %}
|
||||
|
||||
location ~* \.(?:js|css|woff2?|png|jpg|jpeg|gif|svg|ico)$ {
|
||||
expires 7d;
|
||||
add_header Cache-Control "public";
|
||||
{% if app_catalog[a.name].security_headers | default(false) %}
|
||||
{{ security_headers() }}
|
||||
{% endif %}
|
||||
}
|
||||
}
|
||||
{% endfor %}
|
||||
@@ -1,3 +0,0 @@
|
||||
---
|
||||
wordpress_image: "wordpress:6-apache"
|
||||
wordpress_root: "{{ apps_base_dir }}/wordpress"
|
||||
@@ -1,77 +0,0 @@
|
||||
---
|
||||
# Mount each site's shared wp-content and write its compose file.
|
||||
# List lives in inventory/wordpress_sites.yml. Containers are not started
|
||||
# here: the first browser install must run on one host only.
|
||||
|
||||
- name: Load WordPress site list
|
||||
ansible.builtin.include_vars:
|
||||
file: "{{ inventory_dir }}/wordpress_sites.yml"
|
||||
|
||||
- name: Install NFS client
|
||||
ansible.builtin.apt:
|
||||
name: nfs-common
|
||||
state: present
|
||||
|
||||
- name: Create WordPress mount points
|
||||
ansible.builtin.file:
|
||||
path: "{{ item.mount }}"
|
||||
state: directory
|
||||
mode: "0755"
|
||||
loop: "{{ wordpress_sites }}"
|
||||
loop_control:
|
||||
label: "{{ item.name }}"
|
||||
|
||||
- name: Mount WordPress wp-content
|
||||
ansible.posix.mount:
|
||||
src: "{{ wordpress_nfs_server }}:{{ item.export }}"
|
||||
path: "{{ item.mount }}"
|
||||
fstype: nfs
|
||||
opts: "{{ wordpress_nfs_opts }}"
|
||||
dump: "0"
|
||||
passno: "0"
|
||||
state: mounted
|
||||
loop: "{{ wordpress_sites }}"
|
||||
loop_control:
|
||||
label: "{{ item.name }}"
|
||||
|
||||
- name: Ensure WordPress env directory
|
||||
ansible.builtin.file:
|
||||
path: "{{ apps_env_dir }}"
|
||||
state: directory
|
||||
owner: "{{ admin_user }}"
|
||||
group: "{{ admin_user }}"
|
||||
mode: "0700"
|
||||
|
||||
- name: Ensure WordPress compose directories
|
||||
ansible.builtin.file:
|
||||
path: "{{ wordpress_root }}/{{ item.name | regex_replace('_beta$', '-beta') }}"
|
||||
state: directory
|
||||
owner: "{{ admin_user }}"
|
||||
group: "{{ admin_user }}"
|
||||
mode: "0755"
|
||||
loop: "{{ wordpress_sites }}"
|
||||
when: (item.url | default('') | length) > 0
|
||||
loop_control:
|
||||
label: "{{ item.name }}"
|
||||
|
||||
- name: Note WordPress sites waiting on a public URL
|
||||
ansible.builtin.debug:
|
||||
msg: >-
|
||||
WordPress {{ item.name }} has no url. NFS is mounted.
|
||||
Compose is not written until url is set in inventory/wordpress_sites.yml.
|
||||
loop: "{{ wordpress_sites }}"
|
||||
when: (item.url | default('') | length) == 0
|
||||
loop_control:
|
||||
label: "{{ item.name }}"
|
||||
|
||||
- name: Write WordPress compose file
|
||||
ansible.builtin.template:
|
||||
src: docker-compose.yml.j2
|
||||
dest: "{{ wordpress_root }}/{{ item.name | regex_replace('_beta$', '-beta') }}/docker-compose.yml"
|
||||
owner: "{{ admin_user }}"
|
||||
group: "{{ admin_user }}"
|
||||
mode: "0644"
|
||||
loop: "{{ wordpress_sites }}"
|
||||
when: (item.url | default('') | length) > 0
|
||||
loop_control:
|
||||
label: "{{ item.name }}"
|
||||
@@ -1,21 +0,0 @@
|
||||
# Managed by Ansible (roles/wordpress). Do not edit by hand.
|
||||
name: {{ item.name if (item.name is match('.*_beta$')) else (item.name ~ '_prod') }}
|
||||
services:
|
||||
wordpress:
|
||||
image: {{ item.image | default(wordpress_image) }}
|
||||
restart: unless-stopped
|
||||
env_file:
|
||||
- {{ apps_env_dir }}/{{ item.name if (item.name is match('.*_beta$')) else (item.name ~ '_prod') }}.env
|
||||
ports:
|
||||
- "{{ item.port }}:80"
|
||||
environment:
|
||||
WORDPRESS_CONFIG_EXTRA: |
|
||||
define('WP_HOME', '{{ item.url }}');
|
||||
define('WP_SITEURL', '{{ item.url }}');
|
||||
define('DISABLE_WP_CRON', true);
|
||||
if (isset($_SERVER['HTTP_X_FORWARDED_PROTO']) && strpos($_SERVER['HTTP_X_FORWARDED_PROTO'], 'https') !== false) {
|
||||
$_SERVER['HTTPS'] = 'on';
|
||||
}
|
||||
define('FORCE_SSL_ADMIN', true);
|
||||
volumes:
|
||||
- {{ item.mount }}:/var/www/html/wp-content
|
||||
+3
-3
@@ -14,10 +14,10 @@ Usage: $(basename "$0") [HOST] [OPTIONS]
|
||||
|
||||
Deploy applications with deploy-apps.yml.
|
||||
|
||||
HOST Optional. Limit to one host: adama, roslin, starbuck, apollo, or ai-server-4080.
|
||||
HOST Optional. Limit to one host: adama, roslin, or ai-server-4080.
|
||||
|
||||
Options:
|
||||
--app NAME App to deploy (company_site, dta_service, dta_webapp, dta_blog, scha, chat_web_app, chat_backend, monica_site, url_shortening_service, college_craft, print_forge, abc_be, abc_worker, abc_fe, livekit, disney_hotel_service)
|
||||
--app NAME App to deploy (company_site, dta_service, dta_webapp, scha, chat_web_app, chat_backend, monica_site)
|
||||
--env ENV Environment: beta or prod
|
||||
--ref REF Git ref/sha to deploy (default: master)
|
||||
--check Dry run
|
||||
@@ -67,7 +67,7 @@ while [[ $# -gt 0 ]]; do
|
||||
EXTRA_VARS+=(-e "$2")
|
||||
shift 2
|
||||
;;
|
||||
adama|roslin|starbuck|apollo|ai-server-4080)
|
||||
adama|roslin|ai-server-4080)
|
||||
LIMIT="$1"
|
||||
shift
|
||||
;;
|
||||
|
||||
@@ -14,16 +14,11 @@ Usage: $(basename "$0") [HOST] [OPTIONS]
|
||||
Provision server(s) with site.yml.
|
||||
|
||||
Applies: common, ufw, docker, nodejs, gitea-key, tianji, alloy (every host).
|
||||
common installs nfs-common with the other base packages.
|
||||
Also mounts WordPress wp-content (NFS) and, when that entry has a public
|
||||
url, writes its compose file. Site list is inventory/wordpress_sites.yml.
|
||||
Add a site there; the next provision mounts it on every webserver.
|
||||
Containers are not started.
|
||||
On ai-server-4080 also: observability (Loki + Prometheus + Grafana) when
|
||||
observability_stack is true, and SearxNG when searxng_stack is true
|
||||
(chat_backend grounded search on :8088).
|
||||
|
||||
HOST Optional. Limit to one host: adama, roslin, starbuck, apollo, or ai-server-4080.
|
||||
HOST Optional. Limit to one host: adama, roslin, or ai-server-4080.
|
||||
Omit to run against all webservers.
|
||||
|
||||
Options:
|
||||
@@ -64,7 +59,7 @@ while [[ $# -gt 0 ]]; do
|
||||
EXTRA_ARGS+=(--ask-become-pass)
|
||||
shift
|
||||
;;
|
||||
adama|roslin|starbuck|apollo|ai-server-4080)
|
||||
adama|roslin|ai-server-4080)
|
||||
LIMIT="$1"
|
||||
shift
|
||||
;;
|
||||
|
||||
@@ -1,95 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
cd "$REPO_ROOT"
|
||||
|
||||
LIMIT=""
|
||||
EXTRA_ARGS=()
|
||||
EXTRA_VARS=()
|
||||
APP=""
|
||||
APP_ENV=""
|
||||
|
||||
usage() {
|
||||
cat <<EOF
|
||||
Usage: $(basename "$0") [HOST] --app NAME --env ENV [OPTIONS]
|
||||
|
||||
Stop a docker-compose app (compose down). Inverse of deploy.sh.
|
||||
|
||||
Does not remove volumes, checkouts, or secrets. Next deploy.sh of the same
|
||||
app+env will start it again unless host_apps sets enabled: false.
|
||||
|
||||
Node-static apps have no compose project — stop.sh skips them.
|
||||
|
||||
HOST Optional. Limit to one host: adama, roslin, starbuck, apollo, or ai-server-4080.
|
||||
|
||||
Options:
|
||||
--app NAME Required. App to stop (same names as deploy.sh)
|
||||
--env ENV Required. Environment: beta or prod
|
||||
--check Dry run
|
||||
--diff Show diffs
|
||||
--extra-vars V Pass raw extra vars
|
||||
-h, --help Show this help
|
||||
|
||||
Examples:
|
||||
$(basename "$0") --app print_forge --env prod
|
||||
$(basename "$0") --app print_forge --env beta
|
||||
$(basename "$0") adama --app chat_backend --env beta --check
|
||||
EOF
|
||||
}
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
-h|--help)
|
||||
usage
|
||||
exit 0
|
||||
;;
|
||||
--app)
|
||||
APP="$2"
|
||||
EXTRA_VARS+=(-e "app=$2")
|
||||
shift 2
|
||||
;;
|
||||
--env)
|
||||
APP_ENV="$2"
|
||||
EXTRA_VARS+=(-e "app_env=$2")
|
||||
shift 2
|
||||
;;
|
||||
--check)
|
||||
EXTRA_ARGS+=(--check)
|
||||
shift
|
||||
;;
|
||||
--diff)
|
||||
EXTRA_ARGS+=(--diff)
|
||||
shift
|
||||
;;
|
||||
--extra-vars)
|
||||
EXTRA_VARS+=(-e "$2")
|
||||
shift 2
|
||||
;;
|
||||
adama|roslin|starbuck|apollo|ai-server-4080)
|
||||
LIMIT="$1"
|
||||
shift
|
||||
;;
|
||||
*)
|
||||
echo "Unknown argument: $1" >&2
|
||||
usage >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [[ -z "$APP" || -z "$APP_ENV" ]]; then
|
||||
echo "error: --app and --env are required" >&2
|
||||
usage >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
CMD=(ansible-playbook playbooks/stop-apps.yml "${EXTRA_ARGS[@]}" "${EXTRA_VARS[@]}")
|
||||
if [[ -n "$LIMIT" ]]; then
|
||||
CMD+=(--limit "$LIMIT")
|
||||
echo "==> Stopping ${APP}/${APP_ENV} on: $LIMIT"
|
||||
else
|
||||
echo "==> Stopping ${APP}/${APP_ENV} on all webservers"
|
||||
fi
|
||||
|
||||
exec "${CMD[@]}"
|
||||
Reference in new issue
Block a user