ai_ml_operations/chat_backend
2
0
Fork 0
Code Issues 23 Pull Requests Actions Packages Projects Releases Wiki Activity

Add PromptFeedback API for per-message thumbs up/down ratings #67

New Issue
Closed
opened 2026-08-03 12:10:42 -07:00 by westfarn · 0 comments
westfarn commented 2026-08-03 12:10:42 -07:00
Owner
Copy Link
Copy Source

Problem

Frontend work in chat_web_app#97 needs a per-message rating API so thumbs up/down can feed accuracy measurement for chat_backend#62.

App-wide POST /feedbacks/ is unrelated (product bugs) and must not be reused.

Required shape

Model: PromptFeedback

  • prompt — FK to the assistant Prompt
  • user — FK to the rating user
  • rating — up | down
  • reason — optional short code (incorrect, out_of_date, didnt_follow_instructions, unsafe, other)
  • comment — optional free text
  • created_at, updated_at
  • Unique on (prompt, user) so re-votes update rather than duplicate

Endpoints

  • POST /api/prompt_feedback — upsert { prompt_id, rating, reason?, comment? }
  • PATCH /api/prompt_feedback/{id} or same POST with reason/comment — follow-up after thumbs-down reason chips
  • DELETE /api/prompt_feedback?prompt_id= (or by id) — clear a vote

History hydration

  • Include the current user's feedback on each assistant prompt in GET conversation_details (e.g. nested feedback: { rating, reason, comment } or null).

Joinability

  • Feedback must be joinable to PromptMetric so ratings can be sliced by model. Note PromptMetric.model_name is currently hard-coded to "llama3.2"; #62 fixes that — without it per-model slices are meaningless.

Acceptance

  • Upsert creates or updates unique (prompt, user) row
  • DELETE clears the vote
  • conversation_details returns the caller's rating for each assistant prompt
  • Feedback joinable to PromptMetric for per-model analytics
  • Auth required; users can only rate/clear their own feedback; can only rate prompts they can see

Related

## Problem Frontend work in [chat_web_app#97](https://git.aimloperations.com/ai_ml_operations/chat_web_app/issues/97) needs a per-message rating API so thumbs up/down can feed accuracy measurement for [chat_backend#62](https://git.aimloperations.com/ai_ml_operations/chat_backend/issues/62). App-wide `POST /feedbacks/` is unrelated (product bugs) and must not be reused. ## Required shape ### Model: `PromptFeedback` - `prompt` — FK to the assistant `Prompt` - `user` — FK to the rating user - `rating` — `up` | `down` - `reason` — optional short code (`incorrect`, `out_of_date`, `didnt_follow_instructions`, `unsafe`, `other`) - `comment` — optional free text - `created_at`, `updated_at` - **Unique** on `(prompt, user)` so re-votes update rather than duplicate ### Endpoints - `POST /api/prompt_feedback` — upsert `{ prompt_id, rating, reason?, comment? }` - `PATCH /api/prompt_feedback/{id}` or same POST with reason/comment — follow-up after thumbs-down reason chips - `DELETE /api/prompt_feedback?prompt_id=` (or by id) — clear a vote ### History hydration - Include the current user's feedback on each assistant prompt in `GET conversation_details` (e.g. nested `feedback: { rating, reason, comment }` or null). ### Joinability - Feedback must be joinable to `PromptMetric` so ratings can be sliced by model. Note `PromptMetric.model_name` is currently hard-coded to `"llama3.2"`; #62 fixes that — without it per-model slices are meaningless. ## Acceptance - [ ] Upsert creates or updates unique `(prompt, user)` row - [ ] DELETE clears the vote - [ ] `conversation_details` returns the caller's rating for each assistant prompt - [ ] Feedback joinable to `PromptMetric` for per-model analytics - [ ] Auth required; users can only rate/clear their own feedback; can only rate prompts they can see ## Related - Frontend: [chat_web_app#97](https://git.aimloperations.com/ai_ml_operations/chat_web_app/issues/97) - Accuracy / citations: [chat_backend#62](https://git.aimloperations.com/ai_ml_operations/chat_backend/issues/62)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
twestfall westfarn (Ryan Westfall)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: ai_ml_operations/chat_backend#67