c21f052d5b0a33f890a86baf7f826cf9c33b9f40
13
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  westfarnandCursor
|
c21f052d5b |
Read attached chat images with a vision model.
Image uploads were treated as unknown documents and never shown to the model. PNG, JPEG, WebP, and GIF now go to Ollama vision, and history returns the image bytes. Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
westfarn
|
32f57c2376 |
Enable password self-registration and keep SSO off (#81)
## Summary - Closes #80. - Password self-registration is on by default (`ENABLE_ACCOUNT_REGISTRATION`). `/user/create/` issues tokens and `needs_checkout`, and the web sign-up page already redirects into Stripe Checkout. - Browser SSO stays off (`ENABLE_SSO`, default false). Live Google and Microsoft client ids stay available for Drive linking, but public settings no longer advertise them, and login/signup OAuth start and callback are rejected. Stripe path checked before this change: Checkout builds a subscription session (Founders at $10/month, or `STRIPE_PRICE_ID`), stores a draft invoice, and signed `checkout.session.completed` / `invoice.paid` webhooks grant the plan. Checkout, webhook, portal, registration, and OAuth tests passed. ## Deploy note If `chat_backend_prod.env` or `chat_backend_beta.env` on the control node still sets `ENABLE_ACCOUNT_REGISTRATION=false`, that line overrides this default. Set it to `true` or remove it, and leave `ENABLE_SSO` unset or `false`. ## Test plan - [ ] `GET /api/public/settings/` returns `enable_account_registration: true` and `oauth.google` / `oauth.microsoft` false. - [ ] Sign-up form creates an account and redirects to Stripe Checkout. No Google or Microsoft button. - [ ] Checkout success grants the Founders plan after the webhook. - [ ] `/api/auth/oauth/google/start/?intent=signup` returns 403. Drive connect still starts. - [ ] Setting `ENABLE_ACCOUNT_REGISTRATION=false` closes `/user/create/` again. Reviewed-on: #81 |
||
|
westfarn
|
4fc2b87bb4 |
Grant mobile plans from verified App Store and Play purchases (#78) (#79)
## Summary - Mobile access is granted by `POST /api/finance/store/confirm/` after the app finishes a StoreKit or Play Billing purchase. - App Store signed transactions are checked against Apple Root CA G3. Play purchase tokens are checked with the Play Developer API. - The RevenueCat webhook no longer grants a plan. Stripe Checkout and the Stripe webhook stay for the website. - A purchase account token must match the signed-in user, and a transaction already linked to another account is rejected. Access ends when the verified period end passes unless the app confirms again. Client: ai_ml_operations/chat_web_app#122 ## Config - `APPLE_IAP_BUNDLE_ID` (default `ai.hesychia.chat`) - `GOOGLE_PLAY_PACKAGE_NAME` - `GOOGLE_PLAY_SERVICE_ACCOUNT_JSON` (JSON or a file path) - Plan `revenuecat_product_id` or `REVENUECAT_PRODUCT_PLAN_MAP` maps the store product to a plan ## Test plan - [x] `python manage.py test monetization.tests.test_store_purchases monetization.tests.test_revenuecat monetization.tests.test_plans_quotas` - [ ] Confirm an App Store sandbox transaction and a Play license-test purchase against a configured backend - [ ] Confirm a token already linked to another user is rejected - [ ] Confirm `GET /subscription/` returns `store_account_token` and flips a lapsed store sub to canceled Reviewed-on: #79 |
||
|
westfarn
|
093e5462a5 |
Eval harness (#62 P4), status frames, and agentic runs (#63) (#71)
## Summary - Closes Phase 4 of [#62](#62): `evals/suite.json` (≥40 graded questions), `run_evals` management command, and manually-triggered `.gitea/workflows/run-evals.yml`. - Emits versioned WS `status` frames during grounded chat (evaluating / searching / reading_sources / refining / writing) for [chat_web_app#96](ai_ml_operations/chat_web_app#96). - Implements [#63](#63): Redis/Celery optional infra, `AgentRun`/`AgentStep`, tool registry (SSRF-safe `fetch_url`, tenant-scoped docs), LangGraph orchestrator, progress frames, REST `GET/POST /api/agent_runs/…`, gated by `ALLOW_AGENTIC_TASKS` (default off). ## Test plan - [x] `SKIP_RAG_INIT=1 uv run python manage.py test` for evals, ws frames, agent tools, consumers, grounding - [ ] Manual: with `ALLOW_AGENTIC_TASKS=false`, chat identical to today - [ ] Manual: status frames visible in FE with #96 branch - [ ] Manual (GPU): `python manage.py run_evals --runs 3` - [ ] Manual: `ALLOW_AGENTIC_TASKS=true` multi-step research prompt creates AgentRun + framesReviewed-on: #71 |
||
|
westfarn
|
e1e086a474 |
Monetization app + RevenueCat webhooks (store IAP ledger) (#69)
## Summary - Rename `finance` → **`monetization`** Django app (keep `finance_*` tables via `label = "finance"`) - Add `services/stripe.py` + `services/revenuecat.py`; RevenueCat webhook upserts **subscription + Invoice/Payment** (billing history parity with Stripe) - Mount `/api/monetization/` + keep `/api/finance/` alias - Extend `Source`/`Provider` with `revenuecat`; product→plan mapping via `revenuecat_product_id` / `REVENUECAT_PRODUCT_PLAN_MAP` Closes #68. Companion to [chat_web_app#100](ai_ml_operations/chat_web_app#100). ## Test plan - [x] `manage.py test monetization.tests` (54 OK) - [x] Smoke `chat_backend.tests.test_views_documents` + `test_oauth` - [ ] Deploy: set `REVENUECAT_WEBHOOK_SECRET`; point RC webhook at `/api/finance/webhooks/revenuecat/` - [ ] Map store product IDs on `SubscriptionPlan.revenuecat_product_id` (or env JSON map) - [ ] Sandbox INITIAL_PURCHASE → subscription `source=revenuecat` + invoice in `/finance/invoices/`Reviewed-on: #69 |
||
|
westfarn
|
d8f5b8ebf2 |
Always-on grounded retrieval + role-scoped Ollama models (#62 Phases 1–3) (#65)
## Summary - Closes Phases 1–3 of [#62](#62) (Phase 4 eval harness left for a follow-up). - **Accuracy:** Retrieval is decided every turn (`GroundingDecider`, fails open). `FAST` no longer skips search — it only selects `OLLAMA_MODEL_FAST`. Search failures surface an explicit error instead of hallucinating from parametric memory. - **Search:** Pluggable `services/search/` with **SearxNG primary** + DDGS failover, ranking/dedupe/rumour filtering, numbered dated source blocks, citations persisted on `Prompt.citations` and emitted as `{"v":1,"type":"citations",...}` after stream end. - **Models:** Role-scoped `OLLAMA_MODEL_THINKING` / `_FAST` / `_UTILITY` / `OLLAMA_EMBED_MODEL=nomic-embed-text`, configurable `num_ctx`, real model name on `PromptMetric`, `reindex_embeddings` management command + loud embedding-dimension mismatch. ## SearxNG (ops) See README **SearxNG** section. Short version: run `searxng/searxng` on the GPU host, enable `json` in `settings.yml`, set `SEARXNG_BASE_URL=http://10.0.0.128:8080` in prod/beta secrets, open `:8080` on the LAN firewall like Ollama. ## Test plan - [x] `SKIP_RAG_INIT=1 python manage.py test chat_backend.tests` — 442 OK (6 skipped) - [ ] Deploy beta with updated secrets (`OLLAMA_MODEL_*`, `OLLAMA_EMBED_MODEL=nomic-embed-text`, `SEARXNG_BASE_URL`) - [ ] After embed change: `python manage.py reindex_embeddings` - [ ] Verify `did Taylor Swift get married` in FAST and THINKING returns grounded answer with citations frame - [ ] Kill SearxNG and confirm factual turns return search_unavailable (not Joe Alwyn hallucination); non-factual chat still worksReviewed-on: #65 |
||
|
westfarn
|
d54094f5e0 |
Tier-gated RAG + Drive document sources (#42) (#54)
## Summary Implements epic [#42](#42) (children #43–#53) and advances [#11](#11). - **Entitlement:** `allows_rag` on plans (founders / backer / pro / business; not standard); exposed as `features.rag` - **Gates:** document REST + WS `PromptType.RAG` use `assert_feature_allowed(..., "rag")` - **Lifecycle:** dedupe ingest, delete vectors by `document_id`, honor `active`, fix document detail PATCH/DELETE - **Workspaces:** auto-create default company workspace; fail-closed scoping - **Drive:** personal + company Google/Microsoft connect (`link_drive` / `link_company_drive`), resource selection, sync, webhooks stubs, `sync_drive_connections` management command - **Docs/env:** README + `.env*.example` updated Companion FE: `chat_web_app` branch `feature/rag-epic-42-ui` (#81–#85). ## Test plan - [x] `SKIP_RAG_INIT=1 uv run python manage.py test` (457 OK) - [ ] Migrate finance `0004` + chat_backend `0028` on beta - [ ] Verify Standard user: Documents API 403 + no RAG retrieval - [ ] Verify Founders/Pro: upload + list + active toggle - [ ] Connect Google/Microsoft Drive (incremental scopes) and Sync - [ ] Company manager: `link_company_drive`; non-manager 403 - [ ] Run `manage.py sync_drive_connections`Reviewed-on: #54 |
||
|
westfarn
|
841c0962d9 |
Multi-plan subscriptions, quotas, and token usage APIs (#16 #17 #36) (#37)
## Summary Implements [#16](#16), [#17](#17), and [#36](#36) in one backend PR. - **#36 Multi-plan catalog**: Founders ($10, public), Standard ($15), Pro ($40), Business ($99), Backer ($0). Future tiers seeded but hidden/`is_selectable=false`. Backer email whitelist auto-assigns Founders-level access with no checkout. - **#36 Feature + prompt gating**: plan feature flags (text vs image); rolling **6h** prompt windows (100 / 200 / 300 / 300 / 300). Enforced in both chat consumers when `ENFORCE_SUBSCRIPTION_GATES=true`. - **#17 Token-period quotas**: optional `monthly_token_quota` on plans + per-user override; calendar-month aggregation from `PromptMetric`; warn/block when reported token totals exceed cap. Null provider usage never fabricated as 0; tracked via `turns_missing_token_usage`. - **#16 Token API exposure**: `tokens_in` / `tokens_out` on conversation + prompt serializers (null when unknown). `GET /api/finance/subscription/` returns plan + usage snapshot for the FE. - Checkout defaults to **Founders**; Stripe paid webhooks assign Founders. Registration/OAuth redeem Backer whitelist and return `needs_checkout`. Companion FE PR: `chat_web_app` branch `feature/plans-quotas-token-usage`. ## Test plan - [ ] `manage.py migrate` seeds five plans; admin can add Backer emails - [ ] Public `GET /api/finance/plans/` returns only Founders - [ ] Register with Backer email → active Backer, `needs_checkout=false`, checkout rejected - [ ] Founders checkout + paid webhook → active Founders subscription - [ ] Chat turn blocked without subscription / when prompt window exceeded / when token period exceeded - [ ] Standard plan denies image feature; Pro/Founders/Backer allow - [ ] Conversation/prompt API returns `null` tokens when unreported, sums when present - [ ] `finance.tests.test_plans_quotas` + existing finance/checkout tests passReviewed-on: #37 |
||
|
westfarn
|
67f16565e9 |
Add Stripe Customer Portal session API for account billing (#35)
## Summary - Companion to [chat_web_app#33](ai_ml_operations/chat_web_app#33) (Account billing + Customer Portal) - Follow-on from finance MVP [#21](#21): add authenticated `POST /api/finance/portal/` that creates a Stripe Billing Portal session and returns `portal_url` - Resolve Stripe customer from the user's latest `Invoice.stripe_customer_id`; return `400` when missing (user must complete Checkout first) - Document `STRIPE_PORTAL_RETURN_URL` (default `{FRONTEND_BASE_URL}/account/`) in settings + env examples ## Test plan - [ ] `manage.py test finance.tests.test_portal finance.tests.test_checkout` - [ ] Authenticated portal create with invoice that has `stripe_customer_id` → `201` + `portal_url` - [ ] No customer / unpaid user → `400` with clear detail - [ ] Missing `STRIPE_SECRET_KEY` → `503` - [ ] Unauthenticated → `401` - [ ] Custom `return_url` in body overrides default portal return URLReviewed-on: #35 |
||
|
westfarn
|
acb3a51618 |
Add Google/Microsoft SSO OAuth for register and sign-in (#24) (#29)
Unit Tests / test (push) Successful in 10s
## Summary - Closes #24 (backend half) - Add `OAuthIdentity` model (provider + `sub`, access/refresh tokens) for SSO now and Drive reuse later (#11) - Endpoints: `GET /api/auth/oauth/<google|microsoft>/start/` and `/callback/` - Create or link `CustomUser` by verified email; issue same JWT access/refresh; redirect FE to `/auth/callback/` - Document `GOOGLE_OAUTH_*` / `MICROSOFT_OAUTH_*` / `OAUTH_CALLBACK_BASE_URL` in `.env.example` and `.env.prod.example` - Expose configured providers on `GET /api/public/settings/` as `oauth.google` / `oauth.microsoft` ## Pair with - Frontend PR: `chat_web_app` branch `feature/sso-oauth-24` ## Test plan - [ ] `python manage.py test chat_backend.tests.test_oauth` - [ ] With local Google/Microsoft client IDs set, complete start → IdP → callback → JWT redirect - [ ] Existing password user with same email links identity (no duplicate) - [ ] Unverified / missing email redirects with error code - [ ] Registration disabled: signup start 403; login without account → `account_not_found` - [ ] Secrets not committed; env examples onlyReviewed-on: #29 |
||
|
westfarn
|
30ce3d048d |
Gate self-serve registration behind ENABLE_ACCOUNT_REGISTRATION (#25)
Unit Tests / test (push) Successful in 9s
## Summary - Add `ENABLE_ACCOUNT_REGISTRATION` (default `false`) for self-serve sign-up, set via env / control-node secret (`chat_backend_<env>.env`) - Harden `POST /api/user/create/` to create company + user, return JWTs, and reject when the flag is off - Expose `GET /api/public/settings/` with `enable_account_registration` for the SPA Supports frontend: [chat_web_app#31](ai_ml_operations/chat_web_app#31) ## Test plan - [ ] `cd llm_be && uv run python manage.py test chat_backend.tests.test_views_users` - [ ] Confirm `POST /api/user/create/` returns 403 when flag unset/false - [ ] With `ENABLE_ACCOUNT_REGISTRATION=true`, register returns access/refresh + creates company - [ ] `GET /api/public/settings/` returns the flag - [ ] Add `ENABLE_ACCOUNT_REGISTRATION=false` to control-node `chat_backend_prod.env` / `chat_backend_beta.env` before deploy (see `.env.prod.example`)Reviewed-on: #25 |
||
|
westfarn
|
ad44359804 |
Add finance app with Stripe Checkout subscriptions (#21) (#23)
Unit Tests / test (push) Successful in 10s
## Summary - Closes #21 — new Django `finance` app with Stripe as payment provider - Subscription price defaults to **$10 USD / month** via `SUBSCRIPTION_PRICE_AMOUNT_CENTS = 1000` in `settings.py` (env-overridable) - Persists **Invoice** and **Payment** rows; both registered in Django admin (with payment inline on invoices) - Checkout Session API redirects users to Stripe hosted payment; webhook verifies signatures and upserts ledger idempotently ## API - `POST /api/finance/checkout/` — JWT auth → `{ checkout_url, session_id }` - `GET /api/finance/invoices/` / `GET /api/finance/payments/` — own records - `POST /api/finance/webhooks/stripe/` — Stripe signature-verified webhook ## Config Documented in `.env.example` / `.env.prod.example`: `STRIPE_SECRET_KEY`, `STRIPE_PUBLISHABLE_KEY`, `STRIPE_WEBHOOK_SECRET`, optional `STRIPE_PRICE_ID`, `FRONTEND_BASE_URL` ## Test plan - [x] `uv run python manage.py test finance` (17 tests) - [ ] Set Stripe test keys locally; create checkout session; complete payment in Stripe test mode - [ ] Confirm Invoice/Payment appear in `/admin/` - [ ] Point Stripe webhook to `/api/finance/webhooks/stripe/` and verify `checkout.session.completed` / `invoice.paid`Reviewed-on: #23 |
||
|
westfarn
|
d1660792ad |
Dockerize chat_backend + Ollama LAN + DB file storage (#6) (#7)
Unit Tests / test (push) Successful in 13s
## Summary Implements [chat_backend#6](#6) Part A: - **uv** packaging (`pyproject.toml` + `uv.lock`), Docker/compose (dev + prod), entrypoint/validate-env, Gitea unit-test + auto-deploy workflows (mirror `scha`) - Env-driven Django settings (`DJANGO_*`, `DATABASE_URL`, CSRF/CORS) - **`OLLAMA_BASE_URL`** wired through all Ollama/LangChain clients (prod → `http://10.0.0.128:11434`) - **DatabaseStorage** — prompt/document file blobs in Postgres (`StoredFile`), not container FS; RAG materializes temp paths for loaders - ASGI via `gunicorn` + `UvicornWorker` (HTTP + WebSockets) Companion server-infra PR registers `app_catalog` / `host_apps` (port **8003**). ## Test plan - [ ] `uv sync && cd llm_be && SKIP_RAG_INIT=1 uv run python manage.py test` - [ ] `docker compose build && docker compose up` against bundled Postgres - [ ] Confirm Ollama calls use `OLLAMA_BASE_URL` (not hardcoded localhost) - [ ] Upload a document / prompt file → row in `chat_backend_storedfile`, no disk under `media/` - [ ] After server-infra merge + secret/Postgres/NPM: deploy via `deploy.sh --app chat_backend --env prod`Reviewed-on: #7 |