Async Drive sync via Django tasks (#57) (#58)
Deploy Beta / unit-tests (push) Successful in 11s
Unit Tests / test (push) Successful in 10s
Deploy Beta / docker (push) Successful in 20s
Deploy Beta / deploy-beta (push) Successful in 49s

## Summary
- Closes [#57](#57)
- Companion FE: [chat_web_app#90](ai_ml_operations/chat_web_app#90) / [PR #91](ai_ml_operations/chat_web_app#91)
- `POST /api/drive/connections/<id>/sync/` enqueues via Django 6 Tasks (`drive_tasks.enqueue_drive_sync`) and returns **202** with `last_sync_status=pending`
- Webhooks + `manage.py sync_drive_connections` use the same enqueue path (`--sync-now` for inline/cron)
- With default `ImmediateBackend`, sync still runs in-process but on a **daemon thread** so HTTP returns quickly; swap `TASKS` later for a durable worker
- Duplicate syncs while already `pending` are skipped (unless `force=True`)

## Test plan
- [x] `manage.py test chat_backend.tests.test_drive_tasks chat_backend.tests.test_views_drive chat_backend.tests.test_management_drive_sync`
- [ ] Manual: Sync now returns fast; connection flips pending → ok/error
- [ ] Manual: Google Drive API disabled → `last_sync_status=error` + message in `last_sync_error`
- [ ] Manual: second Sync while pending does not stack jobsReviewed-on: #58
This commit was merged in pull request #58.
This commit is contained in:
2026-08-02 04:06:54 -07:00
parent 7025dab857
commit fd12bb972d
7 changed files with 289 additions and 55 deletions
+6 -4
View File
@@ -235,7 +235,7 @@ true for Founders/Pro/Business/Backer, false for Standard by default).
| `GET /api/drive/connections/` | Caller's personal connections + their company's company connections |
| `DELETE /api/drive/connections/<id>/` | Disconnect (owner for personal, company manager for company) — deactivates + clears tokens, keeps history |
| `POST /api/drive/connections/<id>/resources/` | `{ "resource_ids": [...], "resource_labels": [...] }` — folder/shared-drive/site ids to sync; empty = provider root |
| `POST /api/drive/connections/<id>/sync/` | Sync now (`chat_backend/services/drive_sync.py::sync_connection`) |
| `POST /api/drive/connections/<id>/sync/` | Enqueue sync now (`chat_backend/drive_tasks.py`) — returns **202** with `connection.last_sync_status=pending`; poll `GET /api/drive/connections/` for `ok` / `error` + `last_sync_error` |
**Provider scope differences:**
- Google: same `drive.readonly` scope for personal and company; company sync
@@ -243,10 +243,12 @@ true for Founders/Pro/Business/Backer, false for Standard by default).
- Microsoft: personal uses `Files.Read`; company uses `Files.Read.All
Sites.Read.All` and syncs SharePoint sites (`selected_resource_ids` = site ids).
**Workers / webhooks (#52):**
- `python manage.py sync_drive_connections [--connection-id N]` — cron/worker entry point.
**Workers / webhooks (#52, #57):**
- `python manage.py sync_drive_connections [--connection-id N]` — enqueue sync tasks (default).
- `python manage.py sync_drive_connections --sync-now` — run sync inline in this process (cron/debug).
- Django 6 `TASKS` (see `settings.py`): default `ImmediateBackend` runs tasks in-process; Sync now still returns 202 by dispatching on a background thread. Swap `TASKS` to a durable queue + worker for production scale.
- `POST /api/drive/webhooks/google/` / `POST /api/drive/webhooks/microsoft/` —
provider push-notification stubs (`AllowAny`); acknowledge `200` and call
provider push-notification stubs (`AllowAny`); acknowledge `200` and enqueue
`sync_connection` when the notification's `connection_id` is resolvable,
else just `200` (no-op). Microsoft's subscription-creation `validationToken`
handshake is echoed back as `text/plain`.