Eval harness (#62 P4), status frames, and agentic runs (#63) (#71)
Deploy Beta / unit-tests (push) Successful in 11s
Unit Tests / test (push) Successful in 11s
Deploy Beta / docker (push) Successful in 30s
Deploy Beta / deploy-beta (push) Successful in 7m19s

## Summary
- Closes Phase 4 of [#62](#62): `evals/suite.json` (≥40 graded questions), `run_evals` management command, and manually-triggered `.gitea/workflows/run-evals.yml`.
- Emits versioned WS `status` frames during grounded chat (evaluating / searching / reading_sources / refining / writing) for [chat_web_app#96](ai_ml_operations/chat_web_app#96).
- Implements [#63](#63): Redis/Celery optional infra, `AgentRun`/`AgentStep`, tool registry (SSRF-safe `fetch_url`, tenant-scoped docs), LangGraph orchestrator, progress frames, REST `GET/POST /api/agent_runs/…`, gated by `ALLOW_AGENTIC_TASKS` (default off).

## Test plan
- [x] `SKIP_RAG_INIT=1 uv run python manage.py test` for evals, ws frames, agent tools, consumers, grounding
- [ ] Manual: with `ALLOW_AGENTIC_TASKS=false`, chat identical to today
- [ ] Manual: status frames visible in FE with #96 branch
- [ ] Manual (GPU): `python manage.py run_evals --runs 3`
- [ ] Manual: `ALLOW_AGENTIC_TASKS=true` multi-step research prompt creates AgentRun + framesReviewed-on: #71
This commit was merged in pull request #71.
This commit is contained in:
2026-08-04 04:08:50 -07:00
parent e1e086a474
commit 093e5462a5
46 changed files with 4669 additions and 117 deletions
+25
View File
@@ -94,6 +94,31 @@ REVENUECAT_WEBHOOK_SECRET=
# Enforce plan feature + prompt/token quotas on chat turns (default true).
# ENFORCE_SUBSCRIPTION_GATES=true
FRONTEND_BASE_URL=http://localhost:3000
# Agentic task execution (#63) — long-running, multi-step, tool-using turns.
# Default false: chat behaves exactly like the always-on grounded path (#62),
# no planner/tools/AgentRun rows. Requires a plan with allows_rag (or
# allows_all_future_features) — see monetization SubscriptionPlan.allows_feature.
ALLOW_AGENTIC_TASKS=false
# Redis — optional. Unset = InMemory channel layer (single process, fine for
# dev/tests) and agent work runs on a daemon thread instead of Celery.
# REDIS_URL=redis://127.0.0.1:6379/0
# CELERY_BROKER_URL=redis://127.0.0.1:6379/0
# Orchestrator plans + synthesises; sub-agents run independent plan steps
# concurrently on a smaller/cheaper model.
# OLLAMA_MODEL_ORCHESTRATOR=gpt-oss:20b
# OLLAMA_MODEL_SUBAGENT=llama3.2
# AGENT_MAX_PLAN_STEPS=8
# AGENT_MAX_ITERATIONS=12
# AGENT_WALL_CLOCK_SECONDS=600
# AGENT_SUBAGENT_CONCURRENCY=3
# AGENT_TOOL_TIMEOUT_SECONDS=20
# AGENT_TOOL_OUTPUT_MAX_CHARS=8000
# AGENT_MAX_TOOL_CALLS_PER_RUN=40
# AGENT_FETCH_URL_MAX_BYTES=2097152
# Run a worker once REDIS_URL/CELERY_BROKER_URL are set:
# docker compose --profile agentic up redis worker
# uv run celery -A llm_be worker --loglevel=info
# STRIPE_CHECKOUT_SUCCESS_URL=http://localhost:3000/billing/success?session_id={CHECKOUT_SESSION_ID}
# STRIPE_CHECKOUT_CANCEL_URL=http://localhost:3000/billing/cancel
# Customer Portal return URL (plan change / cancel / payment method).