Read attached chat images with a vision model (#82)
Deploy Beta / unit-tests (push) Successful in 10s
Unit Tests / test (push) Successful in 9s
Deploy Beta / docker (push) Successful in 23s
Deploy Beta / deploy-beta (push) Successful in 1m22s

## Summary
- Image attachments (PNG, JPEG, WebP, GIF) skip the document parser and go to a vision-capable Ollama model (`OLLAMA_MODEL_VISION`, default `gemma4:latest`).
- The image is stored on the prompt. `PromptSerializer` returns `file_type` and base64 `image` so history can redraw it.
- Unsupported image types and files over 8 MB return a websocket error and are not saved.
- A later text turn does not reuse a previous image as a spreadsheet.

Pairs with `chat_web_app` `feat/128-vision-image-prompt` ([#128](ai_ml_operations/chat_web_app#128)).

## Test plan
- [ ] `manage.py test chat_backend.tests.test_vision chat_backend.tests.test_consumers.GraphNodeTestCase`
- [ ] Send a prompt plus PNG over `ws/chat_again/` and confirm the vision model is called.
- [ ] `GET conversation_details` includes `image` and `file_type` for that turn.
- [ ] CSV upload still follows the document path.

Reviewed-on: #82
This commit was merged in pull request #82.
This commit is contained in:
westfarn committed 2026-10-20 06:13:31 -07:00
1 parent 32f57c2376
commit fc06202a1f
11 files changed
+387 -44

No files matched your search

+2
View File
@@ -16,6 +16,8 @@ OLLAMA_BASE_URL=http://127.0.0.1:11434
# OLLAMA_MODEL=gpt-oss:20b
# OLLAMA_MODEL_THINKING=gpt-oss:20b
# OLLAMA_MODEL_FAST=gemma4:latest
# Vision (image + prompt). Must be a vision-capable tag. Default gemma4:latest.
# OLLAMA_MODEL_VISION=gemma4:latest
# OLLAMA_MODEL_UTILITY=llama3.2
# OLLAMA_EMBED_MODEL=nomic-embed-text
# OLLAMA_NUM_CTX_THINKING=16384