Read attached chat images with a vision model (#82)
## Summary - Image attachments (PNG, JPEG, WebP, GIF) skip the document parser and go to a vision-capable Ollama model (`OLLAMA_MODEL_VISION`, default `gemma4:latest`). - The image is stored on the prompt. `PromptSerializer` returns `file_type` and base64 `image` so history can redraw it. - Unsupported image types and files over 8 MB return a websocket error and are not saved. - A later text turn does not reuse a previous image as a spreadsheet. Pairs with `chat_web_app` `feat/128-vision-image-prompt` ([#128](ai_ml_operations/chat_web_app#128)). ## Test plan - [ ] `manage.py test chat_backend.tests.test_vision chat_backend.tests.test_consumers.GraphNodeTestCase` - [ ] Send a prompt plus PNG over `ws/chat_again/` and confirm the vision model is called. - [ ] `GET conversation_details` includes `image` and `file_type` for that turn. - [ ] CSV upload still follows the document path. Reviewed-on: #82
This commit was merged in pull request #82.
This commit is contained in:
1 parent
32f57c2376
commit
fc06202a1f
11 files changed
+387
-44
No files matched your search
@@ -257,6 +257,7 @@ class PromptSerializer(serializers.ModelSerializer):
|
||||
tokens_in = serializers.SerializerMethodField()
|
||||
tokens_out = serializers.SerializerMethodField()
|
||||
feedback = serializers.SerializerMethodField()
|
||||
image = serializers.SerializerMethodField()
|
||||
|
||||
class Meta:
|
||||
model = Prompt
|
||||
@@ -269,8 +270,10 @@ class PromptSerializer(serializers.ModelSerializer):
|
||||
"tokens_out",
|
||||
"citations",
|
||||
"feedback",
|
||||
"file_type",
|
||||
"image",
|
||||
)
|
||||
read_only_fields = ("citations", "feedback")
|
||||
read_only_fields = ("citations", "feedback", "file_type", "image")
|
||||
|
||||
def _token_pair(self, obj):
|
||||
cache = self.context.setdefault("_prompt_token_cache", {})
|
||||
@@ -314,6 +317,11 @@ class PromptSerializer(serializers.ModelSerializer):
|
||||
self.context["_prompt_feedback_by_id"] = by_prompt
|
||||
return by_prompt.get(obj.id)
|
||||
|
||||
def get_image(self, obj):
|
||||
from chat_backend.services.vision import image_base64
|
||||
|
||||
return image_base64(obj)
|
||||
|
||||
def validate_message(self, value: str) -> str:
|
||||
if value is None or not str(value).strip():
|
||||
raise serializers.ValidationError("Message text cannot be empty.")
|
||||
|
||||
Reference in new issue
Block a user