Add starbuck and apollo as active/active app hosts #21
+49
-32
@@ -10,6 +10,8 @@ flowchart TB
|
||||
Control1["ai-server-4080\n(control node)"]
|
||||
Control1 -->|ansible-playbook site.yml| Adama
|
||||
Control1 -->|ansible-playbook site.yml| Roslin
|
||||
Control1 -->|ansible-playbook site.yml| Starbuck
|
||||
Control1 -->|ansible-playbook site.yml| Apollo
|
||||
end
|
||||
|
||||
subgraph cicd ["CI/CD (every merge to master)"]
|
||||
@@ -19,6 +21,8 @@ flowchart TB
|
||||
Deploy --> AnsibleDeploy["ansible-playbook deploy-apps.yml"]
|
||||
AnsibleDeploy --> Adama2["adama"]
|
||||
AnsibleDeploy --> Roslin2["roslin"]
|
||||
AnsibleDeploy --> Starbuck2["starbuck"]
|
||||
AnsibleDeploy --> Apollo2["apollo"]
|
||||
end
|
||||
```
|
||||
|
||||
@@ -35,6 +39,8 @@ Both pipelines share the same inventory (`inventory/hosts.yml`).
|
||||
|------|-----|------|
|
||||
| adama | 10.0.0.77 | Ubuntu Server VM (Proxmox) — app host |
|
||||
| roslin | 10.0.0.176 | Ubuntu Server VM (Proxmox) — app host |
|
||||
| starbuck | 10.0.0.44 | Ubuntu Server VM (Proxmox) — app host |
|
||||
| apollo | 10.0.0.7 | Ubuntu Server VM (Proxmox) — app host |
|
||||
| ai-server-4080 | 10.0.0.128 | Control node + Gitea act runner + Ollama + SearxNG + observability; also runs app replicas |
|
||||
|
||||
Hostname on this machine: `ryan-development-1`
|
||||
@@ -52,8 +58,10 @@ server-infra/
|
||||
│ ├── group_vars/
|
||||
│ │ └── all.yml # vars + app_catalog
|
||||
│ └── host_vars/
|
||||
│ ├── adama.yml # host_apps (django + dta_webapp)
|
||||
│ ├── roslin.yml # host_apps (mirrors adama)
|
||||
│ ├── adama.yml # host_apps (django + static; monica worker)
|
||||
│ ├── roslin.yml # host_apps (mirrors adama, no worker)
|
||||
│ ├── starbuck.yml # host_apps (mirrors roslin)
|
||||
│ ├── apollo.yml # host_apps (mirrors roslin)
|
||||
│ └── ai-server-4080.yml # control node / act runner / SearxNG / observability
|
||||
├── playbooks/
|
||||
│ ├── site.yml # Phase 1: provision
|
||||
@@ -84,11 +92,15 @@ Ansible needs SSH + sudo on each target before playbooks work.
|
||||
```bash
|
||||
ssh-copy-id westfarn@10.0.0.77
|
||||
ssh-copy-id westfarn@10.0.0.176
|
||||
ssh-copy-id westfarn@10.0.0.44
|
||||
ssh-copy-id westfarn@10.0.0.7
|
||||
```
|
||||
3. Confirm passwordless SSH:
|
||||
```bash
|
||||
ssh westfarn@10.0.0.77
|
||||
ssh westfarn@10.0.0.176
|
||||
ssh westfarn@10.0.0.44
|
||||
ssh westfarn@10.0.0.7
|
||||
```
|
||||
4. **First-time only** — grant passwordless sudo on each new host before the first
|
||||
`provision.sh` run. Ubuntu 26.04 ships `sudo-rs` by default; Ansible's
|
||||
@@ -142,6 +154,8 @@ New hosts need the one-time passwordless sudo bootstrap in
|
||||
|
||||
# Same for other hosts
|
||||
./scripts/provision.sh roslin
|
||||
./scripts/provision.sh starbuck
|
||||
./scripts/provision.sh apollo
|
||||
./scripts/provision.sh ai-server-4080
|
||||
```
|
||||
|
||||
@@ -156,6 +170,8 @@ New hosts need the one-time passwordless sudo bootstrap in
|
||||
```bash
|
||||
./scripts/deploy.sh adama
|
||||
./scripts/deploy.sh --check roslin
|
||||
./scripts/deploy.sh starbuck
|
||||
./scripts/deploy.sh apollo
|
||||
```
|
||||
|
||||
Under the hood, scripts pass `--limit <hostname>` to `ansible-playbook`.
|
||||
@@ -192,13 +208,13 @@ After Docker install, re-SSH so the `docker` group membership takes effect.
|
||||
|
||||
| App | Type | Hosts | Envs | Notes |
|
||||
|-----|------|-------|------|-------|
|
||||
| `company_site` | django (docker) | adama + roslin (+ ai-server-4080) | prod | active/active behind NPM; beta port reserved |
|
||||
| `dta_service` | django (docker) | adama + roslin + ai-server-4080 | beta + prod | active/active behind NPM |
|
||||
| `dta_webapp` | node/vite static | adama + roslin (+ ai-server-4080) | beta + prod | active/active; built to `/var/www/<env>.realpath.app/html`, served by web-static nginx |
|
||||
| `scha` | django (docker) | adama + roslin + ai-server-4080 | prod | active/active behind NPM; beta port reserved |
|
||||
| `chat_web_app` | node-static (CRA) | adama + roslin + ai-server-4080 | beta + prod | active/active; built to `/var/www/<env>.chat.aimloperations/html`, served by web-static nginx |
|
||||
| `chat_backend` | django (docker) | adama + roslin + ai-server-4080 | beta + prod | active/active behind NPM; Ollama `http://10.0.0.128:11434`; SearxNG `http://10.0.0.128:8088` (`SEARXNG_BASE_URL`) |
|
||||
| `monica_site` | django (docker) | adama + roslin + ai-server-4080 | beta + prod | active/active behind NPM; no bundled Postgres (like `scha`); dj-queue **worker singleton on adama** only (`compose --profile worker`); Ollama social drafting via `10.0.0.128:11434` |
|
||||
| `company_site` | django (docker) | all webservers | prod | active/active behind NPM; beta port reserved |
|
||||
| `dta_service` | django (docker) | all webservers | beta + prod | active/active behind NPM |
|
||||
| `dta_webapp` | node/vite static | all webservers | beta + prod | active/active; built to `/var/www/<env>.realpath.app/html`, served by web-static nginx |
|
||||
| `scha` | django (docker) | all webservers | prod | active/active behind NPM; beta port reserved |
|
||||
| `chat_web_app` | node-static (CRA) | all webservers | beta + prod | active/active; built to `/var/www/<env>.chat.aimloperations/html`, served by web-static nginx |
|
||||
| `chat_backend` | django (docker) | all webservers | beta + prod | active/active behind NPM; Ollama `http://10.0.0.128:11434`; SearxNG `http://10.0.0.128:8088` (`SEARXNG_BASE_URL`) |
|
||||
| `monica_site` | django (docker) | all webservers | beta + prod | active/active behind NPM; no bundled Postgres (like `scha`); dj-queue **worker singleton on adama** only (`compose --profile worker`); Ollama social drafting via `10.0.0.128:11434` |
|
||||
|
||||
Django apps use a **shared external Postgres** (via `DATABASE_URL` in each host's
|
||||
env file) so active/active replicas share one database. Beta and prod never share
|
||||
@@ -211,7 +227,7 @@ a DB.
|
||||
Optional `compose_profiles: [worker]` activates docker compose profiles on that
|
||||
host only (used for `monica_site` dj-queue singleton on adama).
|
||||
- Django app = one compose project per env: project name `<app>_<env>`, host port from `host_apps`.
|
||||
Ports match across adama/roslin so NPM can balance `adama:PORT` + `roslin:PORT`.
|
||||
Ports match across app hosts so NPM can balance `adama:PORT` + `roslin:PORT` + `starbuck:PORT` + `apollo:PORT`.
|
||||
|
||||
### Ports
|
||||
|
||||
@@ -221,13 +237,13 @@ future beta replica.
|
||||
|
||||
| App | beta | prod | Deployed on |
|
||||
|-----|------|------|-------------|
|
||||
| company_site | 8010 (*not deployed*) | 8000 | adama, roslin, ai-server-4080 |
|
||||
| dta_service | 8011 | 8001 | adama, roslin, ai-server-4080 |
|
||||
| scha | 8012 (*not deployed*) | 8002 | adama, roslin, ai-server-4080 |
|
||||
| chat_backend | 8013 | 8003 | adama, roslin, ai-server-4080 |
|
||||
| monica_site | 8014 | 8004 | adama, roslin, ai-server-4080 |
|
||||
| dta_webapp (nginx) | 8081 | 8080 | adama, roslin, ai-server-4080 |
|
||||
| chat_web_app (nginx) | 8083 | 8082 | adama, roslin, ai-server-4080 |
|
||||
| company_site | 8010 (*not deployed*) | 8000 | all webservers |
|
||||
| dta_service | 8011 | 8001 | all webservers |
|
||||
| scha | 8012 (*not deployed*) | 8002 | all webservers |
|
||||
| chat_backend | 8013 | 8003 | all webservers |
|
||||
| monica_site | 8014 | 8004 | all webservers |
|
||||
| dta_webapp (nginx) | 8081 | 8080 | all webservers |
|
||||
| chat_web_app (nginx) | 8083 | 8082 | all webservers |
|
||||
| SearxNG (LAN only) | — | **8088** | ai-server-4080 only (`searxng_stack`); not an NPM upstream |
|
||||
|
||||
Host-local services on ai-server-4080 (not balanced by NPM):
|
||||
@@ -277,18 +293,18 @@ point each domain at the backend(s):
|
||||
|
||||
- Single host: standard Proxy Host → `adama:PORT`.
|
||||
- Active/active: jc21 NPM's UI Proxy Host is single-target. To balance
|
||||
adama+roslin you need the **Advanced** tab with a custom `upstream {}` block
|
||||
app hosts you need the **Advanced** tab with a custom `upstream {}` block
|
||||
(or a real LB). Confirm this before relying on active/active.
|
||||
|
||||
| App | Domains | Backends |
|
||||
|-----|---------|----------|
|
||||
| company_site | aimloperations.com (+ www) | `adama:8000` + `roslin:8000` |
|
||||
| dta_service | (see DTA NPM hosts) | `adama:8001` / `8011` + same on roslin / ai-server-4080 |
|
||||
| dta_webapp | (see DTA NPM hosts) | `adama:8080` / `8081` + same on roslin |
|
||||
| scha | `schawheaton.aimloperations.com`, `schawheaton.com` (+ www) | `adama:8002` + `roslin:8002` (+ `ai-server-4080:8002`) |
|
||||
| chat_web_app | `chat.aimloperations.com` (+ www); `beta.chat.aimloperations.com` | `adama:8082` / `8083` + same on roslin / ai-server-4080 |
|
||||
| chat_backend | `chatbackend.aimloperations.com`; `beta.chatbackend.aimloperations.com` | `adama:8003` / `8013` + same on roslin / ai-server-4080 |
|
||||
| monica_site | `mkdrealtor.com` (+ www); `monica-preview.aimloperations.com` (beta) | `adama:8004` / `8014` + same on roslin / ai-server-4080 |
|
||||
| company_site | aimloperations.com (+ www) | `adama:8000` + `roslin:8000` + `starbuck:8000` + `apollo:8000` |
|
||||
| dta_service | (see DTA NPM hosts) | `adama:8001` / `8011` + same on roslin / starbuck / apollo / ai-server-4080 |
|
||||
| dta_webapp | (see DTA NPM hosts) | `adama:8080` / `8081` + same on roslin / starbuck / apollo |
|
||||
| scha | `schawheaton.aimloperations.com`, `schawheaton.com` (+ www) | `adama:8002` + `roslin:8002` + `starbuck:8002` + `apollo:8002` (+ `ai-server-4080:8002`) |
|
||||
| chat_web_app | `chat.aimloperations.com` (+ www); `beta.chat.aimloperations.com` | `adama:8082` / `8083` + same on roslin / starbuck / apollo / ai-server-4080 |
|
||||
| chat_backend | `chatbackend.aimloperations.com`; `beta.chatbackend.aimloperations.com` | `adama:8003` / `8013` + same on roslin / starbuck / apollo / ai-server-4080 |
|
||||
| monica_site | `mkdrealtor.com` (+ www); `monica-preview.aimloperations.com` (beta) | `adama:8004` / `8014` + same on roslin / starbuck / apollo / ai-server-4080 |
|
||||
|
||||
### Required changes IN each app repo (owned separately)
|
||||
|
||||
@@ -338,7 +354,7 @@ Server prereqs on 10.0.0.230: create each DB + grant `westfarn`;
|
||||
(default `~/Documents/secrets/<app>/<app>_<env>.env`) with `DATABASE_URL`
|
||||
(see table), `DJANGO_ENV`, `DJANGO_SECRET_KEY`, `WEB_PORT` (matching the port
|
||||
table). Deploy pushes these to `/opt/apps/env/<app>_<env>.env` (mode 600) on
|
||||
adama + roslin. Never committed to git.
|
||||
adama + roslin + starbuck + apollo. Never committed to git.
|
||||
- [x] Node.js/npm/npx for the `dta_webapp` build — installed by the `nodejs`
|
||||
role in `site.yml` (NodeSource, `node_major` default 20).
|
||||
|
||||
@@ -347,7 +363,7 @@ Server prereqs on 10.0.0.230: create each DB + grant `westfarn`;
|
||||
**Recommended:** Single self-hosted runner on ai-server-4080.
|
||||
|
||||
- One orchestration point.
|
||||
- App hosts (adama/roslin) run the workloads; no runner needed on them for deploy fan-out.
|
||||
- App hosts (adama/roslin/starbuck/apollo) run the workloads; no runner needed on them for deploy fan-out.
|
||||
- Runner needs: Ansible, this repo checked out, SSH key to all hosts, vault password (later).
|
||||
|
||||
### Runner requirements on ai-server-4080
|
||||
@@ -356,7 +372,7 @@ Server prereqs on 10.0.0.230: create each DB + grant `westfarn`;
|
||||
|-------------|-----|
|
||||
| Ansible | Run `deploy-apps.yml` |
|
||||
| `server-infra` checkout | Playbooks + inventory |
|
||||
| SSH key to adama + roslin | Deploy fan-out |
|
||||
| SSH key to app hosts | Deploy fan-out |
|
||||
|
||||
On every push or merged PR to `master`, `.gitea/workflows/sync-checkout.yml`
|
||||
fast-forward pulls this repo at `~/Documents/repos/server-infra` on the Act
|
||||
@@ -387,8 +403,8 @@ Store vault password for CI in a file readable only by the Act runner (e.g. `~/.
|
||||
| # | Task | Status |
|
||||
|---|------|--------|
|
||||
| 1 | Create `server-infra` repo | Done |
|
||||
| 2 | Inventory with all 3 hosts | Done |
|
||||
| 3 | Bootstrap SSH to adama + roslin | Manual |
|
||||
| 2 | Inventory with all 5 hosts | Done ([#20](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/20)) |
|
||||
| 3 | Bootstrap SSH to app hosts | Manual |
|
||||
| 4 | `site.yml` → common, ufw, docker | Done |
|
||||
| 5 | Verify `ansible webservers -m ping` | Manual |
|
||||
| 6 | Test on single server: `./scripts/provision.sh adama` | Manual |
|
||||
@@ -401,13 +417,14 @@ Store vault password for CI in a file readable only by the Act runner (e.g. `~/.
|
||||
| 10b | Register + deploy `chat_web_app` (node-static, ports 8082/8083) | Done (prod); beta ([#7](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/7), [chat_web_app#35](https://git.aimloperations.com/ai_ml_operations/chat_web_app/issues/35)) |
|
||||
| 10c | Register + deploy `chat_backend` (django, ports 8003/8013) | Done (prod); beta ([#7](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/7), [chat_backend#26](https://git.aimloperations.com/ai_ml_operations/chat_backend/issues/26)) |
|
||||
| 10d | Register + deploy `monica_site` (django, ports 8004/8014) | Done ([#14](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/14)) |
|
||||
| 10e | Auto-start `monica_site` dj-queue worker on adama (`compose_profiles`) | In progress ([#17](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/17)) |
|
||||
| 10e | Auto-start `monica_site` dj-queue worker on adama (`compose_profiles`) | Done ([#17](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/17)) |
|
||||
| 10f | Add starbuck + apollo as app hosts (same workloads as roslin) | This PR ([#20](https://git.aimloperations.com/ai_ml_operations/server-infra/issues/20)) |
|
||||
| 11 | Gitea container registry (optional) | Future |
|
||||
|
||||
## Open Decisions
|
||||
|
||||
1. **Deploy user** — `westfarn` vs dedicated `deploy` for CI.
|
||||
2. **NPM load balancing** — confirm jc21 NPM can express adama+roslin upstreams (Advanced tab), else active/active is just two independent instances.
|
||||
2. **NPM load balancing** — confirm jc21 NPM can express all app-host upstreams (Advanced tab), else active/active is just independent instances.
|
||||
3. **Secrets** — Ansible Vault vs per-host env files (currently per-host `/opt/apps/env/*.env`).
|
||||
|
||||
## Adding a New VM
|
||||
|
||||
@@ -11,6 +11,8 @@ ansible-galaxy collection install -r requirements.yml
|
||||
# Bootstrap SSH key to each host (one-time, before Ansible)
|
||||
ssh-copy-id westfarn@10.0.0.77
|
||||
ssh-copy-id westfarn@10.0.0.176
|
||||
ssh-copy-id westfarn@10.0.0.44
|
||||
ssh-copy-id westfarn@10.0.0.7
|
||||
|
||||
# First-time only: passwordless sudo on each new host (before first provision)
|
||||
ssh -t westfarn@10.0.0.176 # repeat for each host IP
|
||||
@@ -46,4 +48,6 @@ the central **Loki / Prometheus / Grafana** stack (`observability_stack: true`).
|
||||
|------|-----|------|
|
||||
| adama | 10.0.0.77 | app host |
|
||||
| roslin | 10.0.0.176 | app host |
|
||||
| starbuck | 10.0.0.44 | app host |
|
||||
| apollo | 10.0.0.7 | app host |
|
||||
| ai-server-4080 | 10.0.0.128 | control node + act runner |
|
||||
|
||||
@@ -45,7 +45,7 @@ first; use text search (`|=`, `|~`) on logs second.
|
||||
|
||||
| Label | Meaning | Examples |
|
||||
|-------|---------|----------|
|
||||
| `host` | Inventory hostname | `adama`, `roslin`, `ai-server-4080` |
|
||||
| `host` | Inventory hostname | `adama`, `roslin`, `starbuck`, `apollo`, `ai-server-4080` |
|
||||
| `env` | Deploy environment | `beta`, `prod`, `host` (journal), `infra` (stack containers) |
|
||||
| `app` | App / service name | `company_site`, `dta_service`, `dta_webapp`, `system` |
|
||||
| `job` | Collector | `docker`, `systemd` |
|
||||
@@ -56,7 +56,7 @@ first; use text search (`|=`, `|~`) on logs second.
|
||||
|
||||
| Label | Meaning | Examples |
|
||||
|-------|---------|----------|
|
||||
| `host` | Inventory hostname (stamped by Alloy) | `adama`, `roslin`, `ai-server-4080` |
|
||||
| `host` | Inventory hostname (stamped by Alloy) | `adama`, `roslin`, `starbuck`, `apollo`, `ai-server-4080` |
|
||||
| `job` | Scrape job | `node` (host), `cadvisor` (containers) |
|
||||
| `env` / `app` | Parsed from Compose project `<app>_<env>` | `prod` / `dta_service` |
|
||||
| `name` | Container name (cAdvisor) | `company_site_prod-web-1` |
|
||||
|
||||
+17
-7
@@ -10,6 +10,8 @@ flowchart LR
|
||||
subgraph hosts ["All webservers"]
|
||||
A["adama\nAlloy"]
|
||||
R["roslin\nAlloy"]
|
||||
S["starbuck\nAlloy"]
|
||||
Ap["apollo\nAlloy"]
|
||||
C["ai-server-4080\nAlloy"]
|
||||
end
|
||||
|
||||
@@ -24,9 +26,13 @@ flowchart LR
|
||||
|
||||
A -->|logs| L
|
||||
R -->|logs| L
|
||||
S -->|logs| L
|
||||
Ap -->|logs| L
|
||||
C -->|logs| L
|
||||
A -->|metrics| P
|
||||
R -->|metrics| P
|
||||
S -->|metrics| P
|
||||
Ap -->|metrics| P
|
||||
C -->|metrics| P
|
||||
L --> G
|
||||
P --> G
|
||||
@@ -36,7 +42,7 @@ flowchart LR
|
||||
|
||||
| Piece | Where | Role |
|
||||
|-------|--------|------|
|
||||
| **Alloy** | every host (`adama`, `roslin`, `ai-server-4080`) | Ship journald + Docker **logs** to Loki; scrape host + container **metrics** → Prometheus |
|
||||
| **Alloy** | every host (`adama`, `roslin`, `starbuck`, `apollo`, `ai-server-4080`) | Ship journald + Docker **logs** to Loki; scrape host + container **metrics** → Prometheus |
|
||||
| **Loki** | `ai-server-4080` only | Store and index logs |
|
||||
| **Prometheus** | `ai-server-4080` only | Store metrics (CPU, RAM, disk, container health) |
|
||||
| **Grafana** | `ai-server-4080` only | Explore logs/metrics, dashboards, alerts |
|
||||
@@ -182,7 +188,7 @@ services:
|
||||
user: "0:0"
|
||||
command: -config.file=/etc/loki/loki-config.yml
|
||||
ports:
|
||||
# Bind to all interfaces so Alloy on adama/roslin can push.
|
||||
# Bind to all interfaces so Alloy on app hosts can push.
|
||||
# Firewall (UFW) should restrict who can connect — see 1.6.
|
||||
- "3100:3100"
|
||||
volumes:
|
||||
@@ -376,7 +382,7 @@ Alloy runs on **every** host in `webservers`. It:
|
||||
|
||||
| Label | Source | Example values |
|
||||
|-------|--------|----------------|
|
||||
| `host` | Ansible inventory hostname | `adama`, `roslin`, `ai-server-4080` |
|
||||
| `host` | Ansible inventory hostname | `adama`, `roslin`, `starbuck`, `apollo`, `ai-server-4080` |
|
||||
| `job` | collector name | `systemd`, `docker` |
|
||||
| `env` | Docker Compose project suffix | `beta`, `prod`, `host`, `infra` |
|
||||
| `app` | Compose project prefix | `company_site`, `dta_service`, `dta_webapp`, … |
|
||||
@@ -387,7 +393,7 @@ Alloy runs on **every** host in `webservers`. It:
|
||||
|
||||
| Label | Source | Example values |
|
||||
|-------|--------|----------------|
|
||||
| `host` | Added by Alloy relabel | `adama`, `roslin`, `ai-server-4080` |
|
||||
| `host` | Added by Alloy relabel | `adama`, `roslin`, `starbuck`, `apollo`, `ai-server-4080` |
|
||||
| `job` | scrape job name | `node`, `cadvisor` |
|
||||
| `name` | container name (cAdvisor) | `company_site_prod-web-1` |
|
||||
| `container_label_com_docker_compose_project` | Compose project | `dta_service_prod` |
|
||||
@@ -407,7 +413,7 @@ container_memory_usage_bytes{host="adama", env="prod", app="dta_service"}
|
||||
|
||||
### 3.1 Install Alloy (manual — one host)
|
||||
|
||||
Repeat on `adama`, `roslin`, and `ai-server-4080`. Example for **adama**:
|
||||
Repeat on `adama`, `roslin`, `starbuck`, `apollo`, and `ai-server-4080`. Example for **adama**:
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /opt/apps/observability/alloy
|
||||
@@ -415,7 +421,7 @@ sudo chown -R westfarn:westfarn /opt/apps/observability
|
||||
```
|
||||
|
||||
Create `/opt/apps/observability/alloy/config.alloy`. **Change every
|
||||
`host = "adama"`** on each machine (`adama` / `roslin` / `ai-server-4080`):
|
||||
`host = "adama"`** on each machine (`adama` / `roslin` / `starbuck` / `apollo` / `ai-server-4080`):
|
||||
|
||||
```river
|
||||
// Grafana Alloy — logs → Loki, metrics → Prometheus.
|
||||
@@ -705,6 +711,8 @@ Healthy Alloy logs mention connecting / sending without repeated
|
||||
```logql
|
||||
{host="adama"}
|
||||
{host="roslin"}
|
||||
{host="starbuck"}
|
||||
{host="apollo"}
|
||||
{host="ai-server-4080"}
|
||||
```
|
||||
|
||||
@@ -816,6 +824,8 @@ observability_stack: true
|
||||
./scripts/provision.sh ai-server-4080
|
||||
./scripts/provision.sh adama
|
||||
./scripts/provision.sh roslin
|
||||
./scripts/provision.sh starbuck
|
||||
./scripts/provision.sh apollo
|
||||
# or all (site.yml orders stack before Alloy):
|
||||
./scripts/provision.sh
|
||||
```
|
||||
@@ -916,7 +926,7 @@ du -sh /opt/apps/observability/loki/data \
|
||||
- [ ] `GF_SERVER_ROOT_URL` matches public URL
|
||||
- [ ] Both Loki and Prometheus datasources green in Grafana
|
||||
|
||||
### Alloy (each of adama, roslin, ai-server-4080)
|
||||
### Alloy (each of adama, roslin, starbuck, apollo, ai-server-4080)
|
||||
|
||||
- [ ] `config.alloy` has correct `host = "..."` (or Ansible `inventory_hostname`)
|
||||
- [ ] Alloy container running privileged with host `/proc` `/sys` mounts
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
---
|
||||
# Django services run active/active here and on roslin (shared external DB).
|
||||
# dta_webapp static also runs active/active (built into /var/www, served by
|
||||
# the web-static nginx container). Ports MUST match roslin so NPM can balance.
|
||||
# Django services run active/active here and on roslin/starbuck/apollo
|
||||
# (shared external DB). dta_webapp static also runs active/active (built into
|
||||
# /var/www, served by the web-static nginx container). Ports MUST match the
|
||||
# other app hosts so NPM can balance.
|
||||
# Each entry is one workload: django -> compose project <name>_<env> on port;
|
||||
# node-static -> /var/www/<env>_dta_webapp served on port.
|
||||
host_apps:
|
||||
@@ -16,6 +17,6 @@ host_apps:
|
||||
- { name: chat_web_app, env: beta, port: 8083 }
|
||||
- { name: chat_backend, env: prod, port: 8003 }
|
||||
- { name: chat_backend, env: beta, port: 8013 }
|
||||
# compose_profiles: worker → dj-queue singleton (not on roslin / ai-server-4080).
|
||||
# compose_profiles: worker → dj-queue singleton (not on other app hosts).
|
||||
- { name: monica_site, env: prod, port: 8004, compose_profiles: [worker] }
|
||||
- { name: monica_site, env: beta, port: 8014, compose_profiles: [worker] }
|
||||
|
||||
@@ -22,7 +22,7 @@ searxng_stack: true
|
||||
gitea_key_path: "/home/{{ admin_user }}/.ssh/gitea_deploy"
|
||||
|
||||
# company_site + dta_webapp + dta_service for side testing and active/active.
|
||||
# Ports MUST match adama/roslin so NPM can balance all three upstreams.
|
||||
# Ports MUST match adama/roslin/starbuck/apollo so NPM can balance all upstreams.
|
||||
host_apps:
|
||||
- { name: company_site, env: prod, port: 8000 }
|
||||
- { name: dta_webapp, env: prod, port: 8080 }
|
||||
|
||||
@@ -0,0 +1,18 @@
|
||||
---
|
||||
# Mirrors roslin for active/active. Ports MUST match adama/roslin so NPM
|
||||
# upstreams can balance apollo:PORT with the other app hosts.
|
||||
host_apps:
|
||||
- { name: company_site, env: prod, port: 8000 }
|
||||
- { name: dta_service, env: prod, port: 8001 }
|
||||
- { name: dta_service, env: beta, port: 8011 }
|
||||
- { name: dta_webapp, env: prod, port: 8080 }
|
||||
- { name: dta_webapp, env: beta, port: 8081 }
|
||||
- { name: scha, env: prod, port: 8002 }
|
||||
# optional: - { name: scha, env: beta, port: 8012 }
|
||||
- { name: chat_web_app, env: prod, port: 8082 }
|
||||
- { name: chat_web_app, env: beta, port: 8083 }
|
||||
- { name: chat_backend, env: prod, port: 8003 }
|
||||
- { name: chat_backend, env: beta, port: 8013 }
|
||||
# monica_site worker NOT here — adama host_apps sets compose_profiles: [worker].
|
||||
- { name: monica_site, env: prod, port: 8004 }
|
||||
- { name: monica_site, env: beta, port: 8014 }
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
# Mirrors adama for active/active. Ports MUST match adama so NPM upstreams
|
||||
# can balance adama:PORT and roslin:PORT for the same workload.
|
||||
# Mirrors adama for active/active. Ports MUST match adama/starbuck/apollo so
|
||||
# NPM upstreams can balance roslin:PORT with the other app hosts.
|
||||
host_apps:
|
||||
- { name: company_site, env: prod, port: 8000 }
|
||||
- { name: dta_service, env: prod, port: 8001 }
|
||||
|
||||
@@ -0,0 +1,18 @@
|
||||
---
|
||||
# Mirrors roslin for active/active. Ports MUST match adama/roslin so NPM
|
||||
# upstreams can balance starbuck:PORT with the other app hosts.
|
||||
host_apps:
|
||||
- { name: company_site, env: prod, port: 8000 }
|
||||
- { name: dta_service, env: prod, port: 8001 }
|
||||
- { name: dta_service, env: beta, port: 8011 }
|
||||
- { name: dta_webapp, env: prod, port: 8080 }
|
||||
- { name: dta_webapp, env: beta, port: 8081 }
|
||||
- { name: scha, env: prod, port: 8002 }
|
||||
# optional: - { name: scha, env: beta, port: 8012 }
|
||||
- { name: chat_web_app, env: prod, port: 8082 }
|
||||
- { name: chat_web_app, env: beta, port: 8083 }
|
||||
- { name: chat_backend, env: prod, port: 8003 }
|
||||
- { name: chat_backend, env: beta, port: 8013 }
|
||||
# monica_site worker NOT here — adama host_apps sets compose_profiles: [worker].
|
||||
- { name: monica_site, env: prod, port: 8004 }
|
||||
- { name: monica_site, env: beta, port: 8014 }
|
||||
@@ -7,5 +7,9 @@ all:
|
||||
ansible_host: 10.0.0.77
|
||||
roslin:
|
||||
ansible_host: 10.0.0.176
|
||||
starbuck:
|
||||
ansible_host: 10.0.0.44
|
||||
apollo:
|
||||
ansible_host: 10.0.0.7
|
||||
ai-server-4080:
|
||||
ansible_host: 10.0.0.128
|
||||
|
||||
+2
-2
@@ -14,7 +14,7 @@ Usage: $(basename "$0") [HOST] [OPTIONS]
|
||||
|
||||
Deploy applications with deploy-apps.yml.
|
||||
|
||||
HOST Optional. Limit to one host: adama, roslin, or ai-server-4080.
|
||||
HOST Optional. Limit to one host: adama, roslin, starbuck, apollo, or ai-server-4080.
|
||||
|
||||
Options:
|
||||
--app NAME App to deploy (company_site, dta_service, dta_webapp, scha, chat_web_app, chat_backend, monica_site)
|
||||
@@ -67,7 +67,7 @@ while [[ $# -gt 0 ]]; do
|
||||
EXTRA_VARS+=(-e "$2")
|
||||
shift 2
|
||||
;;
|
||||
adama|roslin|ai-server-4080)
|
||||
adama|roslin|starbuck|apollo|ai-server-4080)
|
||||
LIMIT="$1"
|
||||
shift
|
||||
;;
|
||||
|
||||
@@ -18,7 +18,7 @@ On ai-server-4080 also: observability (Loki + Prometheus + Grafana) when
|
||||
observability_stack is true, and SearxNG when searxng_stack is true
|
||||
(chat_backend grounded search on :8088).
|
||||
|
||||
HOST Optional. Limit to one host: adama, roslin, or ai-server-4080.
|
||||
HOST Optional. Limit to one host: adama, roslin, starbuck, apollo, or ai-server-4080.
|
||||
Omit to run against all webservers.
|
||||
|
||||
Options:
|
||||
@@ -59,7 +59,7 @@ while [[ $# -gt 0 ]]; do
|
||||
EXTRA_ARGS+=(--ask-become-pass)
|
||||
shift
|
||||
;;
|
||||
adama|roslin|ai-server-4080)
|
||||
adama|roslin|starbuck|apollo|ai-server-4080)
|
||||
LIMIT="$1"
|
||||
shift
|
||||
;;
|
||||
|
||||
Reference in New Issue
Block a user