Read attached chat images with a vision model (#82)
Deploy Beta / unit-tests (push) Successful in 10s
Unit Tests / test (push) Successful in 9s
Deploy Beta / docker (push) Successful in 23s
Deploy Beta / deploy-beta (push) Successful in 1m22s

## Summary
- Image attachments (PNG, JPEG, WebP, GIF) skip the document parser and go to a vision-capable Ollama model (`OLLAMA_MODEL_VISION`, default `gemma4:latest`).
- The image is stored on the prompt. `PromptSerializer` returns `file_type` and base64 `image` so history can redraw it.
- Unsupported image types and files over 8 MB return a websocket error and are not saved.
- A later text turn does not reuse a previous image as a spreadsheet.

Pairs with `chat_web_app` `feat/128-vision-image-prompt` ([#128](ai_ml_operations/chat_web_app#128)).

## Test plan
- [ ] `manage.py test chat_backend.tests.test_vision chat_backend.tests.test_consumers.GraphNodeTestCase`
- [ ] Send a prompt plus PNG over `ws/chat_again/` and confirm the vision model is called.
- [ ] `GET conversation_details` includes `image` and `file_type` for that turn.
- [ ] CSV upload still follows the document path.

Reviewed-on: #82
This commit was merged in pull request #82.
This commit is contained in:
westfarn committed 2026-10-20 06:13:31 -07:00
1 parent 32f57c2376
commit fc06202a1f
11 files changed
+387 -44

No files matched your search

+9 -1
View File
@@ -257,6 +257,7 @@ class PromptSerializer(serializers.ModelSerializer):
tokens_in = serializers.SerializerMethodField()
tokens_out = serializers.SerializerMethodField()
feedback = serializers.SerializerMethodField()
image = serializers.SerializerMethodField()
class Meta:
model = Prompt
@@ -269,8 +270,10 @@ class PromptSerializer(serializers.ModelSerializer):
"tokens_out",
"citations",
"feedback",
"file_type",
"image",
)
read_only_fields = ("citations", "feedback")
read_only_fields = ("citations", "feedback", "file_type", "image")
def _token_pair(self, obj):
cache = self.context.setdefault("_prompt_token_cache", {})
@@ -314,6 +317,11 @@ class PromptSerializer(serializers.ModelSerializer):
self.context["_prompt_feedback_by_id"] = by_prompt
return by_prompt.get(obj.id)
def get_image(self, obj):
from chat_backend.services.vision import image_base64
return image_base64(obj)
def validate_message(self, value: str) -> str:
if value is None or not str(value).strip():
raise serializers.ValidationError("Message text cannot be empty.")