## Summary
- Follow-up to [#71](#71) / [#63](#63).
- **Bugfix:** `run_agentic_turn` discarded `ws_send`, so `run_started` / `plan_ready` / `step_*` / `run_completed` frames only hit an empty Redis group and never the connected client during inline agent runs.
- Thread `ws_send` through `execute_agent_run` / `_broadcast` (WS + channel-layer group).
- Route agentic turns in `consumers_graph` the same way as `consumers.py`.
- Add offline `test_agent_orchestrator.py` coverage.
## Test plan
- [x] `SKIP_RAG_INIT=1 uv run python manage.py test chat_backend.tests.test_agent_orchestrator chat_backend.tests.test_agent_tools chat_backend.tests.test_consumers`
- [ ] Manual: `ALLOW_AGENTIC_TASKS=true`, multi-step prompt — confirm agent frames arrive on the live WS
- [ ] Manual: graph consumer path (`ws/conditional_chat/`) also routes agentic turnsReviewed-on: #72
## Summary
- Closes [#67](#67) — new `PromptFeedback` model (unique on `(prompt, user)`) with `rating` (`up`|`down`), optional `reason` / `comment`, and timestamps via `TimeInfoBase`.
- `POST /api/prompt_feedback` upserts `{ prompt_id, rating, reason?, comment? }`; `DELETE /api/prompt_feedback?prompt_id=` clears the caller's vote.
- `GET conversation_details` now nests the caller's `feedback: { rating, reason, comment }` (or `null`) on each prompt so [chat_web_app#101](ai_ml_operations/chat_web_app#101) can rehydrate thumbs UI.
- Auth required; users can only rate assistant prompts in their own non-deleted conversations. Distinct from app-wide `POST /feedbacks/`.
- Joinable to `PromptMetric` via `prompt_id` for per-model accuracy slices.
## Test plan
- [ ] `uv run python manage.py test chat_backend.tests.test_views_prompt_feedback`
- [ ] Upsert thumbs up, then down with reason/comment — one row updated
- [ ] DELETE clears vote; second DELETE → 404
- [ ] Rating another user's prompt → 404; rating a user message → 400
- [ ] Reload conversation details → assistant prompts show caller feedback only
- [ ] Companion FE [chat_web_app#101](ai_ml_operations/chat_web_app#101) thumbs + reason popover against this APIReviewed-on: #70
## Summary
`reindex_embeddings` crashed with `ModuleNotFoundError: No module named 'networkx'` when Unstructured hit spreadsheet files via `partition/xlsx.py`. Reindex clears Chroma first, so a mid-run crash leaves an empty/partial vector store.
### True fix
Depend on **`unstructured[xlsx]==0.18.21`** (not bare `unstructured`). That extra pulls the required spreadsheet partition deps: `networkx`, `msoffcrypto-tool`, `xlrd` (plus openpyxl/pandas already present).
### Resilience
Also catch per-document ingest failures so one corrupt/unsupported file cannot abort a full rebuild after Chroma was cleared. Successful files still ingest; failures are logged.
## Test plan
- [x] CI / unit tests on this PR
- [ ] Merge + redeploy image to all hosts
- [ ] On each host: `docker compose -p chat_backend_prod exec web bash -lc 'cd /app/llm_be && SKIP_RAG_INIT=1 uv run python manage.py reindex_embeddings'`
- [ ] Confirm non-zero chunk count and no `networkx` / `msoffcrypto` / `xlrd` import errors
## Ops workaround (running containers only, until deploy)
```bash
uv pip install 'unstructured[xlsx]==0.18.21'
cd /app/llm_be && SKIP_RAG_INIT=1 uv run python manage.py reindex_embeddings
```Reviewed-on: #66
## Summary
- Closes Phases 1–3 of [#62](#62) (Phase 4 eval harness left for a follow-up).
- **Accuracy:** Retrieval is decided every turn (`GroundingDecider`, fails open). `FAST` no longer skips search — it only selects `OLLAMA_MODEL_FAST`. Search failures surface an explicit error instead of hallucinating from parametric memory.
- **Search:** Pluggable `services/search/` with **SearxNG primary** + DDGS failover, ranking/dedupe/rumour filtering, numbered dated source blocks, citations persisted on `Prompt.citations` and emitted as `{"v":1,"type":"citations",...}` after stream end.
- **Models:** Role-scoped `OLLAMA_MODEL_THINKING` / `_FAST` / `_UTILITY` / `OLLAMA_EMBED_MODEL=nomic-embed-text`, configurable `num_ctx`, real model name on `PromptMetric`, `reindex_embeddings` management command + loud embedding-dimension mismatch.
## SearxNG (ops)
See README **SearxNG** section. Short version: run `searxng/searxng` on the GPU host, enable `json` in `settings.yml`, set `SEARXNG_BASE_URL=http://10.0.0.128:8080` in prod/beta secrets, open `:8080` on the LAN firewall like Ollama.
## Test plan
- [x] `SKIP_RAG_INIT=1 python manage.py test chat_backend.tests` — 442 OK (6 skipped)
- [ ] Deploy beta with updated secrets (`OLLAMA_MODEL_*`, `OLLAMA_EMBED_MODEL=nomic-embed-text`, `SEARXNG_BASE_URL`)
- [ ] After embed change: `python manage.py reindex_embeddings`
- [ ] Verify `did Taylor Swift get married` in FAST and THINKING returns grounded answer with citations frame
- [ ] Kill SearxNG and confirm factual turns return search_unavailable (not Joe Alwyn hallucination); non-factual chat still worksReviewed-on: #65
## Summary
- Closes [#57](#57)
- Companion FE: [chat_web_app#90](ai_ml_operations/chat_web_app#90) / [PR #91](ai_ml_operations/chat_web_app#91)
- `POST /api/drive/connections/<id>/sync/` enqueues via Django 6 Tasks (`drive_tasks.enqueue_drive_sync`) and returns **202** with `last_sync_status=pending`
- Webhooks + `manage.py sync_drive_connections` use the same enqueue path (`--sync-now` for inline/cron)
- With default `ImmediateBackend`, sync still runs in-process but on a **daemon thread** so HTTP returns quickly; swap `TASKS` later for a durable worker
- Duplicate syncs while already `pending` are skipped (unless `force=True`)
## Test plan
- [x] `manage.py test chat_backend.tests.test_drive_tasks chat_backend.tests.test_views_drive chat_backend.tests.test_management_drive_sync`
- [ ] Manual: Sync now returns fast; connection flips pending → ok/error
- [ ] Manual: Google Drive API disabled → `last_sync_status=error` + message in `last_sync_error`
- [ ] Manual: second Sync while pending does not stack jobsReviewed-on: #58
## Summary
- Closes [#55](#55) (related [#46](#46))
- **Personal** Google Drive / OneDrive works for users **without** a company (personal RAG)
- **Company** Drive still requires company + manager
- Schema: nullable `DriveConnection.company`, personal `DocumentWorkspace.user`, ownership check constraints + conditional uniques
- Runtime: `ensure_personal_workspace`, personal sync → personal WS, chat/document APIs fall back to personal WS when `company_id` is null
- Supersedes the interim "reject with `no_company`" approach (wrong for personal connect)
## Test plan
- [x] OAuth: personal Drive callback with `user.company=NULL` succeeds (`company_id=NULL` on connection)
- [x] OAuth: company Drive still returns `no_company` / `forbidden` appropriately
- [x] Chat tenant scope creates personal workspace for solo users
- [x] Drive sync + document view suites (`91` related tests)
- [ ] Manual: solo entitled user connects Google Drive → success, sync lands in personal workspace
- [ ] Manual: company manager company Drive still works
- [ ] Migrate prod/staging with `0029_personal_drive_rag_without_company`Reviewed-on: #56
## Summary
- Closes [#40](#40)
- Aligns chat/RAG with the abc_worker stove-pipe pattern ([b13cec8](b13cec88f9)): immutable `ChatCompanyScope` per turn, conversation ownership validation, fail-closed Chroma filters
- Prefer ASGI/JWT identity over client email; never bind identity from `conversation_id` alone
- Close `ConversationDetailView` IDOR (prompts only for `request.user`)
## Changes
- New `services/chat_tenant_scope.py` with frozen `ChatCompanyScope` + ownership checks
- WebSocket consumers (`consumers.py` / `consumers_graph.py`) validate scope before `get_messages` / RAG
- `search_documents` requires a workspace (no more `filter: None` over the shared collection)
- Ingest writes `company_id` metadata (retrieval still keys on `workspace_id` for back-compat)
- Legacy `get_retriever` always applies a workspace filter
## Test plan
- [x] `manage.py test chat_backend.tests.test_chat_tenant_scope chat_backend.tests.test_consumers chat_backend.tests.test_services_rag chat_backend.tests.test_views_conversations`
- [ ] Manual: user A cannot stream RAG context from user B `conversation_id`
- [ ] Manual: RAG still returns own-company docs after deploy (existing vectors with `workspace_id` only)
- [ ] Follow-up: FE can send JWT `token`/`access` on WS payloads for stronger identity bindingReviewed-on: #41
## Summary
- Closes#31
- Ignore WebSocket `type: ping` heartbeats so keepalives no longer create conversations or hit title/LLM pipelines
- Reject empty/whitespace user messages in both chat consumers, `PromptSerializer`, and REST conversation prompt POST
## Test plan
- [x] `UserPromptGuardTestCase`, `PromptSerializerTestCase` blank/whitespace cases
- [x] `WebSocketReceiveGuardTestCase` ping ignore + empty message rejection (both WS routes)
- [ ] Deploy to beta; leave idle tab open and confirm no new rogue conversations
- [ ] Confirm normal chat send still works
Related FE: https://git.aimloperations.com/ai_ml_operations/chat_web_app/issues/51Reviewed-on: #32
## Summary
- Add `ENABLE_ACCOUNT_REGISTRATION` (default `false`) for self-serve sign-up, set via env / control-node secret (`chat_backend_<env>.env`)
- Harden `POST /api/user/create/` to create company + user, return JWTs, and reject when the flag is off
- Expose `GET /api/public/settings/` with `enable_account_registration` for the SPA
Supports frontend: [chat_web_app#31](ai_ml_operations/chat_web_app#31)
## Test plan
- [ ] `cd llm_be && uv run python manage.py test chat_backend.tests.test_views_users`
- [ ] Confirm `POST /api/user/create/` returns 403 when flag unset/false
- [ ] With `ENABLE_ACCOUNT_REGISTRATION=true`, register returns access/refresh + creates company
- [ ] `GET /api/public/settings/` returns the flag
- [ ] Add `ENABLE_ACCOUNT_REGISTRATION=false` to control-node `chat_backend_prod.env` / `chat_backend_beta.env` before deploy (see `.env.prod.example`)Reviewed-on: #25
## Summary
- Closes#20 — name the AI assistant **Hesychia** in system / generation prompts
- Add shared `assistant_identity.py` with `ASSISTANT_NAME` + concise calm/stillness tone
- Prepend identity to chat (`llm_service`), RAG, data analysis, and the views system message
- Document that identity lives in code (not env); add unit coverage
## Test plan
- [x] `uv run python manage.py test chat_backend.tests.test_assistant_identity chat_backend.tests.test_services_llm chat_backend.tests.test_services_data_analysis`
- [ ] Fresh chat: ask "who are you?" → responds as Hesychia
- [ ] Confirm classifiers/moderators/title generator unchanged (not assistant identity)
Related: companion frontend rebrand `chat_web_app#29`Reviewed-on: #22
Closes#15
## Summary
- Add nullable `tokens_in` / `tokens_out` `IntegerField`s to `PromptMetric` to record real prompt/completion token counts per turn.
- New `extract_token_usage()` helper parses provider usage payloads (LangChain `usage_metadata`, OpenAI-style `prompt_tokens`/`completion_tokens`, Ollama `prompt_eval_count`/`eval_count`). When a provider reports no usage, the fields stay **null** — counts are never estimated/fabricated.
- `create_prompt_metric` / `finish_prompt_metric` in both `consumers.py` and `consumers_graph.py` accept and persist optional `tokens_in` / `tokens_out` (added to `update_fields` only when present).
- Admin panel (this ticket's deliverable):
- `PromptMetricAdmin` lists `tokens_in` / `tokens_out` and adds `event` / `model_name` / `has_file` filters.
- `ConversationAdmin` shows summed `tokens_in` / `tokens_out` / `tokens_total` per conversation.
- Migration `0023_promptmetric_tokens_in_promptmetric_tokens_out` (existing rows remain valid — null).
## Note on live capture
The streaming chat path uses LangChain `StrOutputParser`, which yields plain string chunks with no usage metadata, so live turns currently persist `null` tokens (honest, per acceptance criteria — no fabricated counts). The plumbing + helper are in place so wiring real provider usage is a drop-in once the services expose it.
## Follow-ups
- #16 — Show token in/out in chat web app UI (FE + API exposure)
- #17 — Token-based billing, quotas, and enforcement
## Test plan
- [x] `uv run python manage.py test` — full suite green (266 tests, 6 skipped)
- [x] Model: token fields default null + persist when set
- [x] `extract_token_usage`: LangChain / OpenAI / Ollama key variants, attribute sources, bool/float handling, missing usage → (None, None)
- [x] Metric lifecycle: tokens persist when provided, stay null when absent (both consumers)
- [x] Admin: conversation token totals sum across metrics and ignore other conversationsReviewed-on: #18
Closes#5
## Summary
- Replaces the three scattered test modules (`chat_backend/tests.py`, `services/tests.py`, `services/prompt_classifier/tests.py`) with a `chat_backend/tests/` package: **242 deterministic tests plus 6 opt-in live-Ollama checks**, up from 10 tests (3 of which were skipped and 4 of which were never even discovered).
- The suite runs fully offline — no Ollama, Chroma, SMTP or network access. LangChain runnables are replaced by a small `FakeChain`, Chroma/embeddings are mocked, email uses Django's locmem backend, and blobs go through `DatabaseStorage`.
- New `llm_be/test_runner.py` (wired via `TEST_RUNNER`) sets `SKIP_RAG_INIT=1` and an MD5 password hasher, so the suite cannot accidentally reach a model server and finishes in ~5s on SQLite (~17s on Postgres) instead of ~30s.
## Coverage
| Area | File |
|------|------|
| `TimeInfoBase.save`, slugs, `get_duration`, `file_exists`, cascades | `test_models.py` |
| `DatabaseStorage` save/open/exists/size/listdir/delete/times | `test_storage.py` |
| JWT claim, prompt/user/feedback/document serializers | `test_serializers.py` |
| auth + token, invite, feedback, company users, set-password, TOS | `test_views_users.py` |
| conversation list/order/create/detail/soft-delete | `test_views_conversations.py` |
| all four analytics endpoints, including empty-month behaviour | `test_views_analytics.py` |
| workspace + document upload/list/detail, 404 and 400 paths | `test_views_documents.py` |
| prompt classifier rules/parsing, moderation fail-safe, title cleanup | `test_services_classifiers.py` |
| CSV/XLSX/DOCX/PDF analysis, plot generation, error payloads | `test_services_data_analysis.py` |
| loader selection, filename sanitising, ingest, temp-file cleanup, search filters | `test_services_rag.py` |
| history formatting and streaming | `test_services_llm.py` |
| document re-index on create/delete, `SKIP_RAG_INIT` guard | `test_signals.py` |
| consumer DB helpers, LangGraph nodes (moderation, classification, generation, search flags), websocket routes | `test_consumers.py` |
Live checks (non-deterministic, need a model server):
```bash
cd llm_be
RUN_LIVE_OLLAMA_TESTS=1 uv run python manage.py test chat_backend.tests.test_live_ollama
```
## Bugs the tests surfaced (fixed here)
1. **`ConversationDetailView.post` silently dropped every prompt.** `import datetime` shadowed `from datetime import datetime`, so `datetime.now()` raised `AttributeError` inside a bare `except` and the endpoint returned 200 without saving. Now uses `timezone.now()`.
2. **Prompt attachments never reached the LLM.** `get_conversation_file_async` (both consumers) did `sync_to_async(prompt.file.read)` — with `DatabaseStorage` the attribute access itself opens the blob, i.e. a DB query in async context, raising `SynchronousOnlyOperation` that was swallowed and returned `(None, None)`. The read now happens inside the thread.
3. **`DatabaseStorage._save` crashed on a str-backed `ContentFile`** (`TypeError: sequence item 0: expected a bytes-like object`); chunks are encoded when needed.
4. **`services/prompt_classifier/__init__.,py`** (note the comma) meant the directory was only an implicit namespace package, which is why its test module was never collected. Renamed, and its duplicate live-Ollama tests folded into `test_live_ollama.py`.
Known-broken paths deliberately left untested and unchanged: `reset_password` / `ResetUserPassword` reference an unimported `requests` plus undefined locals, and `DocumentDetailView.get` references an undefined `workspaces` on its success path. Worth a follow-up ticket.
## Test plan
- [x] `cd llm_be && uv run python manage.py test` → 248 tests, OK (6 skipped, all opt-in live)
- [x] Same suite against Postgres 16 (`DATABASE_URL=postgres://…`) → OK, matching the containerized run in `deploy.yml`
- [x] `uv run black` clean on all added files
- [ ] Gitea Actions **Unit Tests** + **CI** green on this PRReviewed-on: #12
westfarn