Always-on grounded retrieval + role-scoped Ollama models (#62 Phases 1–3) #65

Merged
westfarn merged 2 commits from feature/accuracy-grounded-retrieval-62 into master 2026-08-02 11:46:02 -07:00
Owner

Summary

  • Closes Phases 1–3 of #62 (Phase 4 eval harness left for a follow-up).
  • Accuracy: Retrieval is decided every turn (GroundingDecider, fails open). FAST no longer skips search — it only selects OLLAMA_MODEL_FAST. Search failures surface an explicit error instead of hallucinating from parametric memory.
  • Search: Pluggable services/search/ with SearxNG primary + DDGS failover, ranking/dedupe/rumour filtering, numbered dated source blocks, citations persisted on Prompt.citations and emitted as {"v":1,"type":"citations",...} after stream end.
  • Models: Role-scoped OLLAMA_MODEL_THINKING / _FAST / _UTILITY / OLLAMA_EMBED_MODEL=nomic-embed-text, configurable num_ctx, real model name on PromptMetric, reindex_embeddings management command + loud embedding-dimension mismatch.

SearxNG (ops)

See README SearxNG section. Short version: run searxng/searxng on the GPU host, enable json in settings.yml, set SEARXNG_BASE_URL=http://10.0.0.128:8080 in prod/beta secrets, open :8080 on the LAN firewall like Ollama.

Test plan

  • SKIP_RAG_INIT=1 python manage.py test chat_backend.tests — 442 OK (6 skipped)
  • Deploy beta with updated secrets (OLLAMA_MODEL_*, OLLAMA_EMBED_MODEL=nomic-embed-text, SEARXNG_BASE_URL)
  • After embed change: python manage.py reindex_embeddings
  • Verify did Taylor Swift get married in FAST and THINKING returns grounded answer with citations frame
  • Kill SearxNG and confirm factual turns return search_unavailable (not Joe Alwyn hallucination); non-factual chat still works
## Summary - Closes Phases 1–3 of [#62](https://git.aimloperations.com/ai_ml_operations/chat_backend/issues/62) (Phase 4 eval harness left for a follow-up). - **Accuracy:** Retrieval is decided every turn (`GroundingDecider`, fails open). `FAST` no longer skips search — it only selects `OLLAMA_MODEL_FAST`. Search failures surface an explicit error instead of hallucinating from parametric memory. - **Search:** Pluggable `services/search/` with **SearxNG primary** + DDGS failover, ranking/dedupe/rumour filtering, numbered dated source blocks, citations persisted on `Prompt.citations` and emitted as `{"v":1,"type":"citations",...}` after stream end. - **Models:** Role-scoped `OLLAMA_MODEL_THINKING` / `_FAST` / `_UTILITY` / `OLLAMA_EMBED_MODEL=nomic-embed-text`, configurable `num_ctx`, real model name on `PromptMetric`, `reindex_embeddings` management command + loud embedding-dimension mismatch. ## SearxNG (ops) See README **SearxNG** section. Short version: run `searxng/searxng` on the GPU host, enable `json` in `settings.yml`, set `SEARXNG_BASE_URL=http://10.0.0.128:8080` in prod/beta secrets, open `:8080` on the LAN firewall like Ollama. ## Test plan - [x] `SKIP_RAG_INIT=1 python manage.py test chat_backend.tests` — 442 OK (6 skipped) - [ ] Deploy beta with updated secrets (`OLLAMA_MODEL_*`, `OLLAMA_EMBED_MODEL=nomic-embed-text`, `SEARXNG_BASE_URL`) - [ ] After embed change: `python manage.py reindex_embeddings` - [ ] Verify `did Taylor Swift get married` in FAST and THINKING returns grounded answer with citations frame - [ ] Kill SearxNG and confirm factual turns return search_unavailable (not Joe Alwyn hallucination); non-factual chat still works
westfarn added 1 commit 2026-08-02 07:06:02 -07:00
Always-on grounded retrieval with SearxNG and role-scoped Ollama models (#62).
CI / test (pull_request) Successful in 11s
Unit Tests / test (pull_request) Successful in 10s
d51033809f
Phases 1–3: split THINKING/FAST/UTILITY/EMBED models, structured search with
SearxNG primary + DDGS failover, fail-open grounding, citations on Prompt + WS
frames, single history window with prompt budgeting, and reindex_embeddings.
westfarn added 1 commit 2026-08-02 11:23:42 -07:00
Point SEARXNG_BASE_URL at :8088 (dta_webapp owns :8080 on GPU host).
CI / test (pull_request) Successful in 11s
Unit Tests / test (pull_request) Successful in 10s
51464943a9
Aligns with server-infra#10 / PR #11 — SearxNG binds 8088 on ai-server-4080.
westfarn merged commit d8f5b8ebf2 into master 2026-08-02 11:46:02 -07:00
westfarn deleted branch feature/accuracy-grounded-retrieval-62 2026-08-02 11:46:02 -07:00
Sign in to join this conversation.
No Reviewers
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: ai_ml_operations/chat_backend#65