d8f5b8ebf2200564bfd5cfe1dacd6ac2f654f684
8
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 westfarn
|
d8f5b8ebf2 |
Always-on grounded retrieval + role-scoped Ollama models (#62 Phases 1–3) (#65)
## Summary - Closes Phases 1–3 of [#62](#62) (Phase 4 eval harness left for a follow-up). - **Accuracy:** Retrieval is decided every turn (`GroundingDecider`, fails open). `FAST` no longer skips search — it only selects `OLLAMA_MODEL_FAST`. Search failures surface an explicit error instead of hallucinating from parametric memory. - **Search:** Pluggable `services/search/` with **SearxNG primary** + DDGS failover, ranking/dedupe/rumour filtering, numbered dated source blocks, citations persisted on `Prompt.citations` and emitted as `{"v":1,"type":"citations",...}` after stream end. - **Models:** Role-scoped `OLLAMA_MODEL_THINKING` / `_FAST` / `_UTILITY` / `OLLAMA_EMBED_MODEL=nomic-embed-text`, configurable `num_ctx`, real model name on `PromptMetric`, `reindex_embeddings` management command + loud embedding-dimension mismatch. ## SearxNG (ops) See README **SearxNG** section. Short version: run `searxng/searxng` on the GPU host, enable `json` in `settings.yml`, set `SEARXNG_BASE_URL=http://10.0.0.128:8080` in prod/beta secrets, open `:8080` on the LAN firewall like Ollama. ## Test plan - [x] `SKIP_RAG_INIT=1 python manage.py test chat_backend.tests` — 442 OK (6 skipped) - [ ] Deploy beta with updated secrets (`OLLAMA_MODEL_*`, `OLLAMA_EMBED_MODEL=nomic-embed-text`, `SEARXNG_BASE_URL`) - [ ] After embed change: `python manage.py reindex_embeddings` - [ ] Verify `did Taylor Swift get married` in FAST and THINKING returns grounded answer with citations frame - [ ] Kill SearxNG and confirm factual turns return search_unavailable (not Joe Alwyn hallucination); non-factual chat still worksReviewed-on: #65 |
||
|
westfarn
|
d54094f5e0 |
Tier-gated RAG + Drive document sources (#42) (#54)
## Summary Implements epic [#42](#42) (children #43–#53) and advances [#11](#11). - **Entitlement:** `allows_rag` on plans (founders / backer / pro / business; not standard); exposed as `features.rag` - **Gates:** document REST + WS `PromptType.RAG` use `assert_feature_allowed(..., "rag")` - **Lifecycle:** dedupe ingest, delete vectors by `document_id`, honor `active`, fix document detail PATCH/DELETE - **Workspaces:** auto-create default company workspace; fail-closed scoping - **Drive:** personal + company Google/Microsoft connect (`link_drive` / `link_company_drive`), resource selection, sync, webhooks stubs, `sync_drive_connections` management command - **Docs/env:** README + `.env*.example` updated Companion FE: `chat_web_app` branch `feature/rag-epic-42-ui` (#81–#85). ## Test plan - [x] `SKIP_RAG_INIT=1 uv run python manage.py test` (457 OK) - [ ] Migrate finance `0004` + chat_backend `0028` on beta - [ ] Verify Standard user: Documents API 403 + no RAG retrieval - [ ] Verify Founders/Pro: upload + list + active toggle - [ ] Connect Google/Microsoft Drive (incremental scopes) and Sync - [ ] Company manager: `link_company_drive`; non-manager 403 - [ ] Run `manage.py sync_drive_connections`Reviewed-on: #54 |
||
|
westfarn
|
841c0962d9 |
Multi-plan subscriptions, quotas, and token usage APIs (#16 #17 #36) (#37)
## Summary Implements [#16](#16), [#17](#17), and [#36](#36) in one backend PR. - **#36 Multi-plan catalog**: Founders ($10, public), Standard ($15), Pro ($40), Business ($99), Backer ($0). Future tiers seeded but hidden/`is_selectable=false`. Backer email whitelist auto-assigns Founders-level access with no checkout. - **#36 Feature + prompt gating**: plan feature flags (text vs image); rolling **6h** prompt windows (100 / 200 / 300 / 300 / 300). Enforced in both chat consumers when `ENFORCE_SUBSCRIPTION_GATES=true`. - **#17 Token-period quotas**: optional `monthly_token_quota` on plans + per-user override; calendar-month aggregation from `PromptMetric`; warn/block when reported token totals exceed cap. Null provider usage never fabricated as 0; tracked via `turns_missing_token_usage`. - **#16 Token API exposure**: `tokens_in` / `tokens_out` on conversation + prompt serializers (null when unknown). `GET /api/finance/subscription/` returns plan + usage snapshot for the FE. - Checkout defaults to **Founders**; Stripe paid webhooks assign Founders. Registration/OAuth redeem Backer whitelist and return `needs_checkout`. Companion FE PR: `chat_web_app` branch `feature/plans-quotas-token-usage`. ## Test plan - [ ] `manage.py migrate` seeds five plans; admin can add Backer emails - [ ] Public `GET /api/finance/plans/` returns only Founders - [ ] Register with Backer email → active Backer, `needs_checkout=false`, checkout rejected - [ ] Founders checkout + paid webhook → active Founders subscription - [ ] Chat turn blocked without subscription / when prompt window exceeded / when token period exceeded - [ ] Standard plan denies image feature; Pro/Founders/Backer allow - [ ] Conversation/prompt API returns `null` tokens when unreported, sums when present - [ ] `finance.tests.test_plans_quotas` + existing finance/checkout tests passReviewed-on: #37 |
||
|
westfarn
|
67f16565e9 |
Add Stripe Customer Portal session API for account billing (#35)
## Summary - Companion to [chat_web_app#33](ai_ml_operations/chat_web_app#33) (Account billing + Customer Portal) - Follow-on from finance MVP [#21](#21): add authenticated `POST /api/finance/portal/` that creates a Stripe Billing Portal session and returns `portal_url` - Resolve Stripe customer from the user's latest `Invoice.stripe_customer_id`; return `400` when missing (user must complete Checkout first) - Document `STRIPE_PORTAL_RETURN_URL` (default `{FRONTEND_BASE_URL}/account/`) in settings + env examples ## Test plan - [ ] `manage.py test finance.tests.test_portal finance.tests.test_checkout` - [ ] Authenticated portal create with invoice that has `stripe_customer_id` → `201` + `portal_url` - [ ] No customer / unpaid user → `400` with clear detail - [ ] Missing `STRIPE_SECRET_KEY` → `503` - [ ] Unauthenticated → `401` - [ ] Custom `return_url` in body overrides default portal return URLReviewed-on: #35 |
||
|
westfarn
|
acb3a51618 |
Add Google/Microsoft SSO OAuth for register and sign-in (#24) (#29)
Unit Tests / test (push) Successful in 10s
## Summary - Closes #24 (backend half) - Add `OAuthIdentity` model (provider + `sub`, access/refresh tokens) for SSO now and Drive reuse later (#11) - Endpoints: `GET /api/auth/oauth/<google|microsoft>/start/` and `/callback/` - Create or link `CustomUser` by verified email; issue same JWT access/refresh; redirect FE to `/auth/callback/` - Document `GOOGLE_OAUTH_*` / `MICROSOFT_OAUTH_*` / `OAUTH_CALLBACK_BASE_URL` in `.env.example` and `.env.prod.example` - Expose configured providers on `GET /api/public/settings/` as `oauth.google` / `oauth.microsoft` ## Pair with - Frontend PR: `chat_web_app` branch `feature/sso-oauth-24` ## Test plan - [ ] `python manage.py test chat_backend.tests.test_oauth` - [ ] With local Google/Microsoft client IDs set, complete start → IdP → callback → JWT redirect - [ ] Existing password user with same email links identity (no duplicate) - [ ] Unverified / missing email redirects with error code - [ ] Registration disabled: signup start 403; login without account → `account_not_found` - [ ] Secrets not committed; env examples onlyReviewed-on: #29 |
||
|
westfarn
|
30ce3d048d |
Gate self-serve registration behind ENABLE_ACCOUNT_REGISTRATION (#25)
Unit Tests / test (push) Successful in 9s
## Summary - Add `ENABLE_ACCOUNT_REGISTRATION` (default `false`) for self-serve sign-up, set via env / control-node secret (`chat_backend_<env>.env`) - Harden `POST /api/user/create/` to create company + user, return JWTs, and reject when the flag is off - Expose `GET /api/public/settings/` with `enable_account_registration` for the SPA Supports frontend: [chat_web_app#31](ai_ml_operations/chat_web_app#31) ## Test plan - [ ] `cd llm_be && uv run python manage.py test chat_backend.tests.test_views_users` - [ ] Confirm `POST /api/user/create/` returns 403 when flag unset/false - [ ] With `ENABLE_ACCOUNT_REGISTRATION=true`, register returns access/refresh + creates company - [ ] `GET /api/public/settings/` returns the flag - [ ] Add `ENABLE_ACCOUNT_REGISTRATION=false` to control-node `chat_backend_prod.env` / `chat_backend_beta.env` before deploy (see `.env.prod.example`)Reviewed-on: #25 |
||
|
westfarn
|
ad44359804 |
Add finance app with Stripe Checkout subscriptions (#21) (#23)
Unit Tests / test (push) Successful in 10s
## Summary - Closes #21 — new Django `finance` app with Stripe as payment provider - Subscription price defaults to **$10 USD / month** via `SUBSCRIPTION_PRICE_AMOUNT_CENTS = 1000` in `settings.py` (env-overridable) - Persists **Invoice** and **Payment** rows; both registered in Django admin (with payment inline on invoices) - Checkout Session API redirects users to Stripe hosted payment; webhook verifies signatures and upserts ledger idempotently ## API - `POST /api/finance/checkout/` — JWT auth → `{ checkout_url, session_id }` - `GET /api/finance/invoices/` / `GET /api/finance/payments/` — own records - `POST /api/finance/webhooks/stripe/` — Stripe signature-verified webhook ## Config Documented in `.env.example` / `.env.prod.example`: `STRIPE_SECRET_KEY`, `STRIPE_PUBLISHABLE_KEY`, `STRIPE_WEBHOOK_SECRET`, optional `STRIPE_PRICE_ID`, `FRONTEND_BASE_URL` ## Test plan - [x] `uv run python manage.py test finance` (17 tests) - [ ] Set Stripe test keys locally; create checkout session; complete payment in Stripe test mode - [ ] Confirm Invoice/Payment appear in `/admin/` - [ ] Point Stripe webhook to `/api/finance/webhooks/stripe/` and verify `checkout.session.completed` / `invoice.paid`Reviewed-on: #23 |
||
|
westfarn
|
d1660792ad |
Dockerize chat_backend + Ollama LAN + DB file storage (#6) (#7)
Unit Tests / test (push) Successful in 13s
## Summary Implements [chat_backend#6](#6) Part A: - **uv** packaging (`pyproject.toml` + `uv.lock`), Docker/compose (dev + prod), entrypoint/validate-env, Gitea unit-test + auto-deploy workflows (mirror `scha`) - Env-driven Django settings (`DJANGO_*`, `DATABASE_URL`, CSRF/CORS) - **`OLLAMA_BASE_URL`** wired through all Ollama/LangChain clients (prod → `http://10.0.0.128:11434`) - **DatabaseStorage** — prompt/document file blobs in Postgres (`StoredFile`), not container FS; RAG materializes temp paths for loaders - ASGI via `gunicorn` + `UvicornWorker` (HTTP + WebSockets) Companion server-infra PR registers `app_catalog` / `host_apps` (port **8003**). ## Test plan - [ ] `uv sync && cd llm_be && SKIP_RAG_INIT=1 uv run python manage.py test` - [ ] `docker compose build && docker compose up` against bundled Postgres - [ ] Confirm Ollama calls use `OLLAMA_BASE_URL` (not hardcoded localhost) - [ ] Upload a document / prompt file → row in `chat_backend_storedfile`, no disk under `media/` - [ ] After server-infra merge + secret/Postgres/NPM: deploy via `deploy.sh --app chat_backend --env prod`Reviewed-on: #7 |