westfarn
093e5462a5
Eval harness ( #62 P4), status frames, and agentic runs ( #63 ) ( #71 )
...
Deploy Beta / unit-tests (push) Successful in 11s
Unit Tests / test (push) Successful in 11s
Deploy Beta / docker (push) Successful in 30s
Deploy Beta / deploy-beta (push) Successful in 7m19s
## Summary
- Closes Phase 4 of [#62 ](#62 ): `evals/suite.json` (≥40 graded questions), `run_evals` management command, and manually-triggered `.gitea/workflows/run-evals.yml`.
- Emits versioned WS `status` frames during grounded chat (evaluating / searching / reading_sources / refining / writing) for [chat_web_app#96](ai_ml_operations/chat_web_app#96 ).
- Implements [#63 ](#63 ): Redis/Celery optional infra, `AgentRun`/`AgentStep`, tool registry (SSRF-safe `fetch_url`, tenant-scoped docs), LangGraph orchestrator, progress frames, REST `GET/POST /api/agent_runs/…`, gated by `ALLOW_AGENTIC_TASKS` (default off).
## Test plan
- [x] `SKIP_RAG_INIT=1 uv run python manage.py test` for evals, ws frames, agent tools, consumers, grounding
- [ ] Manual: with `ALLOW_AGENTIC_TASKS=false`, chat identical to today
- [ ] Manual: status frames visible in FE with #96 branch
- [ ] Manual (GPU): `python manage.py run_evals --runs 3`
- [ ] Manual: `ALLOW_AGENTIC_TASKS=true` multi-step research prompt creates AgentRun + framesReviewed-on: #71
2026-08-04 04:08:50 -07:00
westfarn
e1e086a474
Monetization app + RevenueCat webhooks (store IAP ledger) ( #69 )
...
Deploy Beta / unit-tests (push) Successful in 11s
Unit Tests / test (push) Successful in 11s
Deploy Beta / docker (push) Successful in 21s
Deploy Beta / deploy-beta (push) Successful in 50s
## Summary
- Rename `finance` → **`monetization`** Django app (keep `finance_*` tables via `label = "finance"`)
- Add `services/stripe.py` + `services/revenuecat.py`; RevenueCat webhook upserts **subscription + Invoice/Payment** (billing history parity with Stripe)
- Mount `/api/monetization/` + keep `/api/finance/` alias
- Extend `Source`/`Provider` with `revenuecat`; product→plan mapping via `revenuecat_product_id` / `REVENUECAT_PRODUCT_PLAN_MAP`
Closes #68 . Companion to [chat_web_app#100](ai_ml_operations/chat_web_app#100 ).
## Test plan
- [x] `manage.py test monetization.tests` (54 OK)
- [x] Smoke `chat_backend.tests.test_views_documents` + `test_oauth`
- [ ] Deploy: set `REVENUECAT_WEBHOOK_SECRET`; point RC webhook at `/api/finance/webhooks/revenuecat/`
- [ ] Map store product IDs on `SubscriptionPlan.revenuecat_product_id` (or env JSON map)
- [ ] Sandbox INITIAL_PURCHASE → subscription `source=revenuecat` + invoice in `/finance/invoices/`Reviewed-on: #69
2026-08-04 03:40:15 -07:00
westfarn
d8f5b8ebf2
Always-on grounded retrieval + role-scoped Ollama models ( #62 Phases 1–3) ( #65 )
...
Deploy Beta / unit-tests (push) Successful in 11s
Unit Tests / test (push) Successful in 10s
Deploy Beta / docker (push) Successful in 30s
Deploy Beta / deploy-beta (push) Successful in 6m49s
## Summary
- Closes Phases 1–3 of [#62 ](#62 ) (Phase 4 eval harness left for a follow-up).
- **Accuracy:** Retrieval is decided every turn (`GroundingDecider`, fails open). `FAST` no longer skips search — it only selects `OLLAMA_MODEL_FAST`. Search failures surface an explicit error instead of hallucinating from parametric memory.
- **Search:** Pluggable `services/search/` with **SearxNG primary** + DDGS failover, ranking/dedupe/rumour filtering, numbered dated source blocks, citations persisted on `Prompt.citations` and emitted as `{"v":1,"type":"citations",...}` after stream end.
- **Models:** Role-scoped `OLLAMA_MODEL_THINKING` / `_FAST` / `_UTILITY` / `OLLAMA_EMBED_MODEL=nomic-embed-text`, configurable `num_ctx`, real model name on `PromptMetric`, `reindex_embeddings` management command + loud embedding-dimension mismatch.
## SearxNG (ops)
See README **SearxNG** section. Short version: run `searxng/searxng` on the GPU host, enable `json` in `settings.yml`, set `SEARXNG_BASE_URL=http://10.0.0.128:8080 ` in prod/beta secrets, open `:8080` on the LAN firewall like Ollama.
## Test plan
- [x] `SKIP_RAG_INIT=1 python manage.py test chat_backend.tests` — 442 OK (6 skipped)
- [ ] Deploy beta with updated secrets (`OLLAMA_MODEL_*`, `OLLAMA_EMBED_MODEL=nomic-embed-text`, `SEARXNG_BASE_URL`)
- [ ] After embed change: `python manage.py reindex_embeddings`
- [ ] Verify `did Taylor Swift get married` in FAST and THINKING returns grounded answer with citations frame
- [ ] Kill SearxNG and confirm factual turns return search_unavailable (not Joe Alwyn hallucination); non-factual chat still worksReviewed-on: #65
2026-08-02 11:46:02 -07:00
westfarn
d54094f5e0
Tier-gated RAG + Drive document sources ( #42 ) ( #54 )
...
Unit Tests / test (push) Successful in 10s
Deploy Beta / unit-tests (push) Successful in 10s
Deploy Beta / docker (push) Successful in 21s
Deploy Beta / deploy-beta (push) Successful in 40s
## Summary
Implements epic [#42 ](#42 ) (children #43–#53) and advances [#11 ](#11 ).
- **Entitlement:** `allows_rag` on plans (founders / backer / pro / business; not standard); exposed as `features.rag`
- **Gates:** document REST + WS `PromptType.RAG` use `assert_feature_allowed(..., "rag")`
- **Lifecycle:** dedupe ingest, delete vectors by `document_id`, honor `active`, fix document detail PATCH/DELETE
- **Workspaces:** auto-create default company workspace; fail-closed scoping
- **Drive:** personal + company Google/Microsoft connect (`link_drive` / `link_company_drive`), resource selection, sync, webhooks stubs, `sync_drive_connections` management command
- **Docs/env:** README + `.env*.example` updated
Companion FE: `chat_web_app` branch `feature/rag-epic-42-ui` (#81–#85).
## Test plan
- [x] `SKIP_RAG_INIT=1 uv run python manage.py test` (457 OK)
- [ ] Migrate finance `0004` + chat_backend `0028` on beta
- [ ] Verify Standard user: Documents API 403 + no RAG retrieval
- [ ] Verify Founders/Pro: upload + list + active toggle
- [ ] Connect Google/Microsoft Drive (incremental scopes) and Sync
- [ ] Company manager: `link_company_drive`; non-manager 403
- [ ] Run `manage.py sync_drive_connections`Reviewed-on: #54
2026-08-01 14:02:36 -07:00
westfarn
67f16565e9
Add Stripe Customer Portal session API for account billing ( #35 )
...
Deploy Beta / unit-tests (push) Successful in 9s
Unit Tests / test (push) Successful in 10s
Deploy Beta / docker (push) Successful in 26s
Deploy Beta / deploy-beta (push) Successful in 6m49s
## Summary
- Companion to [chat_web_app#33](ai_ml_operations/chat_web_app#33 ) (Account billing + Customer Portal)
- Follow-on from finance MVP [#21 ](#21 ): add authenticated `POST /api/finance/portal/` that creates a Stripe Billing Portal session and returns `portal_url`
- Resolve Stripe customer from the user's latest `Invoice.stripe_customer_id`; return `400` when missing (user must complete Checkout first)
- Document `STRIPE_PORTAL_RETURN_URL` (default `{FRONTEND_BASE_URL}/account/`) in settings + env examples
## Test plan
- [ ] `manage.py test finance.tests.test_portal finance.tests.test_checkout`
- [ ] Authenticated portal create with invoice that has `stripe_customer_id` → `201` + `portal_url`
- [ ] No customer / unpaid user → `400` with clear detail
- [ ] Missing `STRIPE_SECRET_KEY` → `503`
- [ ] Unauthenticated → `401`
- [ ] Custom `return_url` in body overrides default portal return URLReviewed-on: #35
2026-07-31 03:54:28 -07:00
westfarn
5d5b448868
Enable beta auto-deploy on master; manual prod button ( #26 ) ( #30 )
...
Unit Tests / test (push) Successful in 9s
Deploy Beta / unit-tests (push) Successful in 10s
Deploy Beta / docker (push) Successful in 17s
Deploy Beta / deploy-beta (push) Successful in 1m3s
## Summary
Closes #26 .
- Replace auto-prod `deploy.yml` with `deploy-beta.yml` (push to `master` → tests → `--env beta`) and `deploy-prod.yml` (`workflow_dispatch` only → `--env prod`), matching `dta_service` / `chat_web_app`
- Default `ALLOWED_HOSTS` / `CORS_ALLOWED_ORIGINS` include beta API + SPA hosts so beta frontend (and Tianji-instrumented pages) can call the API
- Expand `.env.prod.example` beta block (CORS, CSRF, OAuth callbacks, `FRONTEND_BASE_URL`, Tianji note)
- Harden `validate-env.sh` for beta secrets; README documents beta auto / prod button
## Tianji
Backend does not load `tracker.js`. FE already owns wiring (`chat_web_app#35`, closed).
- Beta SPA origin: `https://beta.chat.aimloperations.com `
- Beta Tianji website ID (FE `.env.beta`): `cms38bw671mf9n5jjw3xp1j3q`
## Coordination
- Infra: [server-infra#7](ai_ml_operations/server-infra#7 ) (`host_apps` beta `:8013`, `chat_backend_beta` DB, secret, NPM)
- FE companion: [chat_web_app#35](ai_ml_operations/chat_web_app#35 )
## Test plan
- [ ] PR CI / unit tests green
- [ ] Merge to `master` triggers **Deploy Beta** only (not prod)
- [ ] Manual **Deploy Prod** `workflow_dispatch` still deploys `--env prod`
- [ ] After infra#7: beta container healthy on **8013** with `chat_backend_beta`
- [ ] Beta hosts / CSRF / CORS allow `https://beta.chat.aimloperations.com `Reviewed-on: #30
2026-07-27 08:35:07 -07:00
westfarn
acb3a51618
Add Google/Microsoft SSO OAuth for register and sign-in ( #24 ) ( #29 )
...
Unit Tests / test (push) Successful in 10s
## Summary
- Closes #24 (backend half)
- Add `OAuthIdentity` model (provider + `sub`, access/refresh tokens) for SSO now and Drive reuse later (#11 )
- Endpoints: `GET /api/auth/oauth/<google|microsoft>/start/` and `/callback/`
- Create or link `CustomUser` by verified email; issue same JWT access/refresh; redirect FE to `/auth/callback/`
- Document `GOOGLE_OAUTH_*` / `MICROSOFT_OAUTH_*` / `OAUTH_CALLBACK_BASE_URL` in `.env.example` and `.env.prod.example`
- Expose configured providers on `GET /api/public/settings/` as `oauth.google` / `oauth.microsoft`
## Pair with
- Frontend PR: `chat_web_app` branch `feature/sso-oauth-24`
## Test plan
- [ ] `python manage.py test chat_backend.tests.test_oauth`
- [ ] With local Google/Microsoft client IDs set, complete start → IdP → callback → JWT redirect
- [ ] Existing password user with same email links identity (no duplicate)
- [ ] Unverified / missing email redirects with error code
- [ ] Registration disabled: signup start 403; login without account → `account_not_found`
- [ ] Secrets not committed; env examples onlyReviewed-on: #29
2026-07-27 05:13:32 -07:00
westfarn
30ce3d048d
Gate self-serve registration behind ENABLE_ACCOUNT_REGISTRATION ( #25 )
...
Unit Tests / test (push) Successful in 9s
## Summary
- Add `ENABLE_ACCOUNT_REGISTRATION` (default `false`) for self-serve sign-up, set via env / control-node secret (`chat_backend_<env>.env`)
- Harden `POST /api/user/create/` to create company + user, return JWTs, and reject when the flag is off
- Expose `GET /api/public/settings/` with `enable_account_registration` for the SPA
Supports frontend: [chat_web_app#31](ai_ml_operations/chat_web_app#31 )
## Test plan
- [ ] `cd llm_be && uv run python manage.py test chat_backend.tests.test_views_users`
- [ ] Confirm `POST /api/user/create/` returns 403 when flag unset/false
- [ ] With `ENABLE_ACCOUNT_REGISTRATION=true`, register returns access/refresh + creates company
- [ ] `GET /api/public/settings/` returns the flag
- [ ] Add `ENABLE_ACCOUNT_REGISTRATION=false` to control-node `chat_backend_prod.env` / `chat_backend_beta.env` before deploy (see `.env.prod.example`)Reviewed-on: #25
2026-07-27 04:19:59 -07:00
westfarn
ad44359804
Add finance app with Stripe Checkout subscriptions ( #21 ) ( #23 )
...
Unit Tests / test (push) Successful in 10s
## Summary
- Closes #21 — new Django `finance` app with Stripe as payment provider
- Subscription price defaults to **$10 USD / month** via `SUBSCRIPTION_PRICE_AMOUNT_CENTS = 1000` in `settings.py` (env-overridable)
- Persists **Invoice** and **Payment** rows; both registered in Django admin (with payment inline on invoices)
- Checkout Session API redirects users to Stripe hosted payment; webhook verifies signatures and upserts ledger idempotently
## API
- `POST /api/finance/checkout/` — JWT auth → `{ checkout_url, session_id }`
- `GET /api/finance/invoices/` / `GET /api/finance/payments/` — own records
- `POST /api/finance/webhooks/stripe/` — Stripe signature-verified webhook
## Config
Documented in `.env.example` / `.env.prod.example`:
`STRIPE_SECRET_KEY`, `STRIPE_PUBLISHABLE_KEY`, `STRIPE_WEBHOOK_SECRET`, optional `STRIPE_PRICE_ID`, `FRONTEND_BASE_URL`
## Test plan
- [x] `uv run python manage.py test finance` (17 tests)
- [ ] Set Stripe test keys locally; create checkout session; complete payment in Stripe test mode
- [ ] Confirm Invoice/Payment appear in `/admin/`
- [ ] Point Stripe webhook to `/api/finance/webhooks/stripe/` and verify `checkout.session.completed` / `invoice.paid`Reviewed-on: #23
2026-07-26 17:35:06 -07:00
westfarn
92aa277a37
Allow Capacitor WebView origins in CORS/CSRF ( #22 ) ( #19 )
...
Unit Tests / test (push) Successful in 9s
## Summary
- Supports [chat_web_app#22](ai_ml_operations/chat_web_app#22 )
- Always merge Capacitor WebView origins (`https://localhost `, `capacitor://localhost`, `http://localhost `) into `CORS_ALLOWED_ORIGINS` and `CSRF_TRUSTED_ORIGINS`
- Keep `CORS_ALLOW_CREDENTIALS = False` (JWT header-only; no cookie credentials)
## Test plan
- [x] `uv run python manage.py test chat_backend.tests.test_settings_cors`
- [ ] OPTIONS preflight from `https://localhost ` / `capacitor://localhost` against prod/beta with `CORS_ORIGIN_ALLOW_ALL=false`
- [ ] Authenticated JWT GET/POST from Capacitor shell after FE #22 landsReviewed-on: #19
2026-07-26 14:04:09 -07:00
westfarn
ad81257ecb
fixing deploymennt ( #8 )
...
Unit Tests / test (push) Successful in 14s
Reviewed-on: #8
2026-07-25 06:17:46 -07:00
westfarn
d1660792ad
Dockerize chat_backend + Ollama LAN + DB file storage ( #6 ) ( #7 )
...
Unit Tests / test (push) Successful in 13s
## Summary
Implements [chat_backend#6](#6 ) Part A:
- **uv** packaging (`pyproject.toml` + `uv.lock`), Docker/compose (dev + prod), entrypoint/validate-env, Gitea unit-test + auto-deploy workflows (mirror `scha`)
- Env-driven Django settings (`DJANGO_*`, `DATABASE_URL`, CSRF/CORS)
- **`OLLAMA_BASE_URL`** wired through all Ollama/LangChain clients (prod → `http://10.0.0.128:11434 `)
- **DatabaseStorage** — prompt/document file blobs in Postgres (`StoredFile`), not container FS; RAG materializes temp paths for loaders
- ASGI via `gunicorn` + `UvicornWorker` (HTTP + WebSockets)
Companion server-infra PR registers `app_catalog` / `host_apps` (port **8003**).
## Test plan
- [ ] `uv sync && cd llm_be && SKIP_RAG_INIT=1 uv run python manage.py test`
- [ ] `docker compose build && docker compose up` against bundled Postgres
- [ ] Confirm Ollama calls use `OLLAMA_BASE_URL` (not hardcoded localhost)
- [ ] Upload a document / prompt file → row in `chat_backend_storedfile`, no disk under `media/`
- [ ] After server-infra merge + secret/Postgres/NPM: deploy via `deploy.sh --app chat_backend --env prod`Reviewed-on: #7
2026-07-25 05:23:33 -07:00
westfarn