Commit Graph
25 Commits
Author SHA1 Message Date
westfarnandCursor d0c8b3f419 Serve Hesychia public routes as real files with security headers.
Unknown URLs were returning the app shell with status 200. Return 404 for missing paths, keep the extensionless pages, and send the security headers. Production short links redirect for HeyCatch.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-19 09:56:58 -05:00
westfarnandCursor 3174c34a6f Merge master and keep both the webhook proxy and crawler file locations.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-17 07:20:48 -05:00
westfarnandCursor dce0879e62 Serve Hesychia crawler files as static text instead of the app shell.
Exact locations return robots.txt, sitemap.xml, and llms.txt, or 404.
Closes #43.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-17 07:19:05 -05:00
westfarn a84350fe60 Proxy Hesychia Stripe webhooks off the static site (#42)
Sync runner checkout / sync (push) Successful in 7s
## Summary
- Closes #41.
- Stripe live endpoint `we_1TzDNHDV0RPXOyxGCOftavHV` posts to `https://hesychia.ai/api/finance/webhooks/stripe/`. On 2026-10-09 the `chat_web_app` prod vhost returned HTTP 405 fifteen times (`request_time` 0.000, User-Agent Stripe). `pending_webhooks` stayed 1.
- `try_files` falls through to `index.html`, and nginx rejects POST on that static file. The Django handler on `chat_backend` never sees the body.
- `chat_web_app` now proxies `/api/finance/webhooks/` to the same-env `chat_backend` port (`8003` prod, `8013` beta) via `host.docker.internal`. `Host` is the backend public hostname so `ALLOWED_HOSTS` matches. Other static apps are unchanged.

## Test plan
- [ ] `ansible-playbook playbooks/site.yml --syntax-check` (or the web-static portion) renders without error
- [ ] Redeploy web-static (`scripts/deploy.sh` path that runs the web-static role) so `nginx.conf` reloads
- [ ] `curl -i -X POST http://127.0.0.1:8082/api/finance/webhooks/stripe/` from an app host reaches Django (400 invalid signature or 503 missing secret), not 405
- [ ] Stripe Dashboard: resend a failed event to the Hesychia.ai endpoint and confirm 2xx

Reviewed-on: #42
2026-10-10 04:54:04 -07:00
westfarn 6f05242ca7 Generate WordPress compose from the site list during provision.
Sync runner checkout / sync (push) Successful in 7s
Each host mounts wp-content and gets a compose file from inventory instead of a hand-copied one. Also allow LAN TCP 8020 so abc_voice is reachable through UFW.
2026-10-08 05:08:16 -05:00
westfarn f4219e2dca Add abc_voice beta container on port 8020 (#39)
Sync runner checkout / sync (push) Successful in 6s
## Summary
- Register `abc_voice` as a second container from `GIS/abc_worker` (`docker-compose.voice.yml`), beta port **8020** on adama, roslin, starbuck, apollo, and ai-server-4080.
- Document `https://beta.abc.voice.aimloperations.com` and `~/Documents/secrets/abc_voice/abc_voice_beta.env`. Ansible does not create the NPM host.
- Closes #38.

## Test plan
- [ ] Create the secret file (copy the worker beta env, set `VOICE_ENABLED=true`, Cartesia, and LiveKit) before the first deploy
- [ ] `deploy.sh --app abc_voice --env beta` after the worker compose file is on `Development`
- [ ] NPM: `beta.abc.voice.aimloperations.com` → `:8020` on all five hosts
- [ ] `abc_be` `WORKER_AV_URL=https://beta.abc.voice.aimloperations.com`

Merge this before the abc_worker workflow PR, or that deploy step fails looking up `abc_voice`.

Reviewed-on: #39
2026-09-27 06:12:43 -07:00
westfarn d9621a984b Deploy ABC beta from Development instead of master (#37)
Sync runner checkout / sync (push) Successful in 7s
## Summary
- A no-ref beta deploy of `abc_be`, `abc_worker`, and `abc_fe` checks out `Development` instead of `master`.
- `livekit` and every other app still use their current default branch.
- CI that passes `--ref` is unchanged; this only affects the catalog fallback.

## Test plan
- [ ] `scripts/deploy.sh --app abc_be --env beta` resolves ref `Development`
- [ ] Same for `abc_worker` and `abc_fe`
- [ ] `livekit` still resolves `master`

Reviewed-on: #37
2026-09-23 17:31:06 -07:00
westfarn 726ad971cb Add stop.sh (compose down) and disable print_forge (#34)
Sync runner checkout / sync (push) Successful in 6s
## Summary
- Closes [#33](#33).
- Add `scripts/stop.sh` → `playbooks/stop-apps.yml` → `docker compose down` for one `--app` + `--env` (no `-v`; checkouts/secrets stay).
- `host_apps.enabled: false` skips deploy (including CI `--app`) so a stopped app cannot come back. Rows stay for ports and `stop.sh`.
- Disable `print_forge` beta + prod on all app hosts. After merge (or from this branch): `./scripts/stop.sh --app print_forge --env prod` and `--env beta`.

## Test plan
- [ ] `./scripts/stop.sh --help` shows required `--app` / `--env`
- [ ] `ansible-playbook playbooks/stop-apps.yml --syntax-check -e app=print_forge -e app_env=prod`
- [ ] `./scripts/stop.sh --app print_forge --env prod` compose-downs `print_forge_prod` on all webservers
- [ ] `./scripts/stop.sh --app print_forge --env beta` compose-downs `print_forge_beta` (including adama worker)
- [ ] `./scripts/deploy.sh --app print_forge --env prod --check` skips disabled rows (does not start containers)
- [ ] NPM: `print-forge-preview.aimloperations.com` will 502 until the proxy host is disabled (out of scope)

Reviewed-on: #34
2026-09-16 03:11:23 -07:00
westfarn 209b27142c Register GIS LiveKit as docker compose on webservers.
Sync runner checkout / sync (pull_request) Successful in 7s
Same deploy path as abc_worker: host_apps on adama/starbuck/etc., NPM for wss://. UDP/7881 still forwarded to one upstream host.
2026-09-15 14:39:10 -05:00
westfarn 1988def1a6 Register dta_blog for node-static deploy (closes #29) (#30)
Sync runner checkout / sync (push) Successful in 7s
## Summary
- Closes [#29](#29).
- Register `dta_blog` in `app_catalog` (`type: node-static`, repo `Ditch_The_Agent/dta_blog`, default branch `main`, webroot `/var/www/{env}.blog.realpath.app/html`). Reuses `roles/app-deploy/tasks/node_static.yml` (`npm ci` then `npm run build:<env>`); the blog's `package.json` copies `dist/` to that webroot after `python3 build.py --env <env>`.
- Add prod **8086** / beta **8087** `host_apps` on adama, roslin, starbuck, apollo, and ai-server-4080 (those ports were free vs 8080–8085 / 8088).
- Optional nginx `error_page 404 /404.html` for this app only (`error_page_404` catalog field); other static apps keep the SPA `try_files` fallback.
- Document NPM/DNS (`blog.realpath.app` → `:8086`, `beta.blog.realpath.app` → `:8087`), no `realpath.app/blog` mount, and UFW staying LAN/NPM-only.

## Test plan
- [ ] Confirm 8086/8087 unused on app hosts before first deploy.
- [ ] `./scripts/deploy.sh --app dta_blog --env beta --ref main` publishes `/var/www/beta.blog.realpath.app/html`.
- [ ] `./scripts/deploy.sh --app dta_blog --env prod --ref main` publishes `/var/www/prod.blog.realpath.app/html`.
- [ ] **Until [dta_blog#1](Ditch_The_Agent/dta_blog#1) is on `main`**, use `--ref issue-1-static-blog` so the SSG (not the stub README) is built.
- [ ] NPM + DNS + TLS: `blog.realpath.app` → `:8086`, `beta.blog.realpath.app` → `:8087`. Do not reverse-proxy onto `realpath.app/blog`.
- [ ] Prod HTML has article text, Tianji id `cmtvvmf562afjzqumwt1yh2y8`, links to `https://realpath.app/`.
- [ ] Beta HTML has Tianji id `cmtvvn6z62agdzqumtk8xijpy`, links to `https://beta.realpath.app/`, demo posts present.
- [ ] `https://blog.realpath.app/sitemap.xml`, `robots.txt`, `llms.txt` return 200.
- [ ] Unknown slug returns 404.html (not the index SPA fallback).

Reviewed-on: #30
2026-09-10 18:34:25 -07:00
westfarn 2abcdd7c58 Register print_forge for django deploy (closes #27) (#28)
Sync runner checkout / sync (push) Successful in 6s
## Summary
- Closes [#27](#27).
- Register `print_forge` in `app_catalog` (`type: django`, repo `ai_ml_operations/print_forge`, default branch `master`).
- Add `host_apps` prod `:8007` / beta `:8019` on **adama**, **roslin**, **starbuck**, **apollo**, and **ai-server-4080**.
- dj-queue **worker singleton on adama** only (`compose_profiles: [worker]`); other hosts web-only.
- Document ports, NPM hosts, Postgres DBs, secrets, Nominatim (`10.0.0.128:8089`), and the app-repo companion ([print_forge#1](ai_ml_operations/print_forge#1)).
- Update `scripts/deploy.sh` `--app` help.
- Move unused `abc_be` prod port reserve **8007 → 8009** so it does not collide with `print_forge`.

## Test plan
- [ ] Confirm `print_forge` appears in `app_catalog` and `--app print_forge` is listed in `deploy.sh --help`
- [ ] Confirm beta `host_apps` on all five hosts (port **8019**); adama has `compose_profiles: [worker]`
- [ ] Confirm prod port **8007** reserved on those hosts (no NPM until launch)
- [ ] After merge (ops, not this PR): create Postgres DBs `print_forge_beta` + `print_forge` on `10.0.0.230`, grant `westfarn`
- [ ] After merge (ops): write `~/Documents/secrets/print_forge/print_forge_{beta,prod}.env` from app `.env.prod.example`; add `print_forge:<token>` to `SHORTENER_API_TOKENS` (beta first)
- [ ] After merge (ops): NPM `print-forge-preview.aimloperations.com` → `:8019` active/active; do **not** NPM-route `printforgeprints.com` until launch
- [ ] After secrets + DBs: `~/Documents/repos/server-infra/scripts/deploy.sh --app print_forge --env beta --ref master` and hit `/healthz/`

Reviewed-on: #28
2026-09-06 06:35:48 -07:00
westfarnandCursor aac815314d Register abc_fe / abc_be / abc_worker for beta-only deploy.
Sync runner checkout / sync (push) Successful in 6s
Closes #26. GIS Benefits Coach gets beta host_apps (8017/8018/8085) and catalog entries; prod ports stay reserved.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-04 16:21:23 -05:00
westfarn 95fbae8db4 Register college_craft for django deploy (closes #24) (#25)
Sync runner checkout / sync (push) Successful in 6s
## Summary
- Closes [#24](#24).
- Register `college_craft` in `app_catalog` (`type: django`, repo `ai_ml_operations/college_craft`, default branch `master`).
- Add `host_apps` prod `:8006` / beta `:8016` on **adama**, **roslin**, **starbuck**, **apollo**, and **ai-server-4080**.
- dj-queue **worker singleton on adama** only (`compose_profiles: [worker]`); other hosts web-only.
- Document ports, NPM hosts, Postgres DBs, secrets, Nominatim (`10.0.0.128:8089`), and the app-repo `--app client_site` leftover.
- Add `college_craft` to `scripts/deploy.sh` `--app` help.

## Out of scope (ops, after merge)
- Create Postgres DBs `college_craft` / `college_craft_beta` on `10.0.0.230` and grant `westfarn`.
- Write control-node secrets `~/Documents/secrets/college_craft/college_craft_{prod,beta}.env`.
- NPM: `collegecraft.com` (+ www) → `:8006`; `college-craft-preview.aimloperations.com` → `:8016`.
- Smoke deploy after secrets + DBs exist:
  ```bash
  ~/Documents/repos/server-infra/scripts/deploy.sh --app college_craft --env beta --ref master
  ~/Documents/repos/server-infra/scripts/deploy.sh --app college_craft --env prod --ref master
  ```

## Test plan
- [ ] Confirm `app_catalog.college_craft` matches other django apps (`compose_file`, `migrate_cmd`).
- [ ] Confirm ports 8006/8016 unused elsewhere and match across all five hosts.
- [ ] Confirm only adama has `compose_profiles: [worker]` for college_craft.
- [ ] After secrets + DBs: deploy beta then prod; `/healthz/` up on both; prod holding page, beta full site.

Reviewed-on: #25
2026-09-02 07:39:03 -07:00
westfarn 9208e63491 Register url_shortening_service for django deploy (closes #22) (#23)
Sync runner checkout / sync (push) Successful in 6s
## Summary
- Closes [#22](#22).
- Register `url_shortening_service` in `app_catalog` (`type: django`, repo `ai_ml_operations/url_shortening_service`, default branch `master`) and `host_apps` on **adama**, **roslin**, **starbuck**, **apollo**, and **ai-server-4080** (prod **8005**, beta **8015**).
- Document NPM (short host vs API host, same container), Postgres DBs (`url_shortener` / `url_shortener_beta`), and extra env keys. No compose worker profile.
- List `url_shortening_service` in `scripts/deploy.sh` `--app` help.

## Out of scope (manual ops)
- Create Postgres DBs `url_shortener` / `url_shortener_beta` + grant `westfarn` on `10.0.0.230`
- Control-node secrets under `~/Documents/secrets/url_shortening_service/`
- NPM proxy hosts (Ryan): short host `/` + `/[a-z0-9]{4,8}` (404 `/api/`, `/admin/`, `/debug/`); API host `/api/` only (404 `/admin/`)
- Deploy smoke — [url_shortening_service](https://git.aimloperations.com/ai_ml_operations/url_shortening_service) `master` is still a stub (`README.md` only)

## Test plan
- [x] Inventory YAML: `url_shortening_service` in `app_catalog`; every host `host_apps` has prod **8005** / beta **8015**
- [ ] After app compose + secrets + DBs exist: `./scripts/deploy.sh --app url_shortening_service --env beta --ref master`
- [ ] Same for prod
- [ ] `GET /healthz/` → `{"status":"ok"}` on both ports
- [ ] Short host `GET /` landing page; `POST /api/links/` without Bearer → 401; public `/admin/` → 404

Reviewed-on: #23
2026-08-30 04:51:31 -07:00
westfarn b07e8cb4c3 Auto-start monica_site dj-queue worker on adama (#18)
Sync runner checkout / sync (push) Successful in 7s
## Summary
- Closes [#17](#17).
- Django deploy passes optional `host_apps.compose_profiles` as `COMPOSE_PROFILES` so compose profiles survive `up -d --remove-orphans`.
- Adama `monica_site` beta+prod set `compose_profiles: [worker]` (dj-queue singleton). Roslin / ai-server-4080 stay web-only.

## Why
Real campaign sends enqueue dj-queue tasks; without the worker they stay **Queued**. Test email worked because it is synchronous SMTP.

## Test plan
- [ ] Merge + deploy `monica_site` beta (and/or prod) via `deploy.sh`
- [ ] On **adama**, confirm worker container up for `monica_site_beta` / `monica_site_prod`
- [ ] On roslin / ai-server-4080, confirm **no** worker
- [ ] Portal: send campaign → recipient leaves Queued → SentReviewed-on: #18
2026-08-08 12:01:18 -07:00
westfarn cc503988c0 Register monica_site for django deploy (closes #14) (#15)
Sync runner checkout / sync (push) Successful in 6s
## Summary
- Closes [#14](#14).
- Register `monica_site` in `app_catalog` (`type: django`, repo `ai_ml_operations/monica_site`, default branch `main`) and `host_apps` on **adama**, **roslin**, and **ai-server-4080** (prod **8004**, beta **8014**).
- Document NPM (`mkdrealtor.com` / `monica-preview.aimloperations.com`), Postgres DBs (`monica_site` / `monica_site_beta`), and **dj-queue worker singleton on adama** only (`compose --profile worker`).
- List `monica_site` in `scripts/deploy.sh` `--app` help.

## Out of scope (manual ops)
- Create Postgres DBs + grants on `10.0.0.230`
- Control-node secrets under `~/Documents/secrets/monica_site/`
- NPM proxy hosts (Ryan)
- First deploy smoke + worker start on adama

## Test plan
- [ ] Confirm inventory YAML parses (`ansible-inventory --list` or dry deploy)
- [ ] Create DBs `monica_site` / `monica_site_beta` + secrets before smoke deploy
- [ ] `./scripts/deploy.sh --app monica_site --env beta --ref main`
- [ ] `./scripts/deploy.sh --app monica_site --env prod --ref main`
- [ ] On **adama** only: `docker compose -f docker-compose.prod.yml --profile worker up -d`
- [ ] Hit `/healthz/` on beta + prod; prod under-construction holding pageReviewed-on: #15
2026-08-06 13:46:00 -07:00
westfarn 829a63cbf2 Point dta_webapp webroot at realpath.app paths (closes #12) (#13)
Sync runner checkout / sync (push) Successful in 7s
## Summary
- Change `dta_webapp` `webroot_pattern` from `/var/www/{env}.app.ditchtheagent/html` → `/var/www/{env}.realpath.app/html`
- Update `IMPLEMENTATION.md` references to match
- **web-static:** before `compose up`, `docker rm -f` any container publishing the static host ports (`docker ps --filter publish=<port>`), then force-recreate when cleanup ran — clears orphan/legacy binders while keeping the same ports

Closes #12

Coordinates with [dta_webapp#48](Ditch_The_Agent/dta_webapp#48) / [PR #49](Ditch_The_Agent/dta_webapp#49) (`build:beta`/`build:prod` write to the same dirs; prod uses `prod.realpath.app`).

## Test plan
- [ ] Confirm `app_catalog.dta_webapp.webroot_pattern` expands to beta/prod realpath paths
- [ ] On a host with a leftover container on 8080/8081, run deploy and confirm it is removed and web-static rebinds
- [ ] After merge, redeploy `dta_webapp` beta (and prod when ready) and verify files under `/var/www/{env}.realpath.app/html`
- [ ] Hit web-static ports and confirm new build content servedReviewed-on: #13
2026-08-06 08:46:33 -07:00
westfarnandCursor 39b0a9f475 Register chat_backend for django deploy (chat_backend#6 Part B).
Sync runner checkout / sync (pull_request) Successful in 7s
Add app_catalog entry, host_apps on adama/roslin/ai-server-4080 (prod
:8003), and document ports/NPM/Postgres/Ollama for active/active.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-25 07:03:36 -05:00
westfarn 71d96839dc Register chat_web_app for node-static deploy (chat_web_app#13 Part B). (#4)
Sync runner checkout / sync (push) Successful in 6s
Add app_catalog entry, host_apps on adama/roslin/ai-server-4080 (prod :8082),
and document ports/NPM backends for the CRA static frontend.Reviewed-on: #4
2026-07-25 04:28:26 -07:00
westfarn 04ccbecbd7 Register scha for deploy on all webservers (scha#19 Part B) (#3)
Sync runner checkout / sync (push) Successful in 6s
## Summary
- Register `scha` in `app_catalog` and `host_apps` on adama, roslin, and ai-server-4080 (prod port **8002**; beta **8012** reserved).
- Update `IMPLEMENTATION.md` ports, apps, Postgres, and NPM tables so every app lists beta + prod.
- Include `scha` in `deploy.sh` help.

Part A (dockerize / CI in the `scha` repo) is tracked separately in [scha#19](ai_ml_operations/scha#19).

## Test plan
- [ ] Confirm inventory YAML loads: `ansible-inventory --list` shows `scha` under each host
- [ ] After Part A merges + `~/Documents/secrets/scha/scha_prod.env` exists: dry-run `./scripts/deploy.sh --app scha --env prod --check`
- [ ] Create DB `scha` on shared Postgres (`10.0.0.230`) and point NPM at `adama:8002` + `roslin:8002` (+ optional `ai-server-4080:8002`)

Reviewed-on: #3
2026-07-14 03:06:46 -07:00
westfarn f37ec41530 Updates for grafana stack 2026-07-10 06:50:52 -05:00
westfarnandCursor b1b9b3bafe Stop legacy dta_service systemd units before Docker deploy (#52).
ai-server-4080 runs dta_service active/active with adama and roslin.
Disable beta_dta_wsgi/prod_dta_wsgi so port 8001/8011 are free for compose.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-09 11:45:03 -05:00
westfarn c53112bfd7 more updates 2026-07-08 13:30:33 -05:00
westfarn 589462e6d0 updates 2026-07-08 05:59:11 -05:00
westfarn f848420d8f Initial server-infra setup 2026-07-06 15:34:49 -05:00