Add tier-gated RAG and Drive document sources (#42)
Ship allows_rag entitlement (founders/backer/pro/business), enforce it on document APIs and RAG chat, harden ingest/delete/active lifecycle, and add Google/Microsoft Drive connect + sync for personal and company knowledge bases. Closes #43 #44 #45 #46 #47 #48 #49 #50 #51 #52 #53 Parent epic: #42 Related: #11
This commit is contained in:
@@ -93,6 +93,8 @@ with `COMPOSE_DATABASE_URL` if needed.
|
||||
| `ENABLE_ACCOUNT_REGISTRATION` | `false` | optional | Self-serve sign-up; keep false until ready |
|
||||
| `STRIPE_SECRET_KEY` / `STRIPE_PUBLISHABLE_KEY` / `STRIPE_WEBHOOK_SECRET` | empty | yes for billing | Stripe API + webhook |
|
||||
| `STRIPE_PRICE_ID` | empty | optional | Pre-created Price; else `$10/mo` from settings |
|
||||
| `GOOGLE_OAUTH_CLIENT_ID` / `..._SECRET` | empty | for SSO/Drive | Also used for Drive linking (#47), incremental scopes |
|
||||
| `MICROSOFT_OAUTH_CLIENT_ID` / `..._SECRET` / `..._TENANT` | empty / `common` | for SSO/Drive | Also used for Drive linking (#47), incremental scopes |
|
||||
| `FRONTEND_BASE_URL` | `http://localhost:3000` | set in prod/beta | Checkout success/cancel, portal return, OAuth return |
|
||||
| `STRIPE_PORTAL_RETURN_URL` | `{FRONTEND}/account/` | optional | Stripe Customer Portal return URL |
|
||||
| `CORS_ALLOWED_ORIGINS` | local + chat FE (+ beta FE default) | set in prod/beta | Frontend origin(s) |
|
||||
@@ -209,6 +211,48 @@ Subscription audit (`UserAuthEvent` on the user admin):
|
||||
- `subscription_started` — first active plan (Checkout, Backer redeem, admin assign)
|
||||
- `subscription_updated` — plan/status/cancel-at-period-end changes (portal + webhooks)
|
||||
|
||||
### Drive / RAG sync ([#47](https://git.aimloperations.com/ai_ml_operations/chat_backend/issues/47)-[#53](https://git.aimloperations.com/ai_ml_operations/chat_backend/issues/53))
|
||||
|
||||
Personal Google Drive / OneDrive and company Google Shared Drive / SharePoint
|
||||
sync into the existing RAG `Document` pipeline. Every endpoint below is gated
|
||||
by `assert_feature_allowed(user, "rag")` (`SubscriptionPlan.allows_rag` —
|
||||
true for Founders/Pro/Business/Backer, false for Standard by default).
|
||||
|
||||
**Connect (OAuth, reuses `#24` SSO app registrations with incremental scopes):**
|
||||
|
||||
| | |
|
||||
|--|--|
|
||||
| Personal | `GET /api/auth/oauth/<google\|microsoft>/start/?intent=link_drive` (authenticated) |
|
||||
| Company | `GET /api/auth/oauth/<google\|microsoft>/start/?intent=link_company_drive` (company manager only) |
|
||||
| Callback | Same `/api/auth/oauth/<provider>/callback/` as SSO; the signed OAuth `state` carries the linking `user_id` since the browser has no session on the IdP redirect. Upserts a `DriveConnection` and redirects to `{FRONTEND_BASE_URL}/account/?drive_connected=1&provider=<provider>&kind=<personal\|company>` (or `?error=<code>`) |
|
||||
|
||||
**Manage:**
|
||||
|
||||
| Method / path | Notes |
|
||||
|--|--|
|
||||
| `GET /api/drive/connections/` | Caller's personal connections + their company's company connections |
|
||||
| `DELETE /api/drive/connections/<id>/` | Disconnect (owner for personal, company manager for company) — deactivates + clears tokens, keeps history |
|
||||
| `POST /api/drive/connections/<id>/resources/` | `{ "resource_ids": [...], "resource_labels": [...] }` — folder/shared-drive/site ids to sync; empty = provider root |
|
||||
| `POST /api/drive/connections/<id>/sync/` | Sync now (`chat_backend/services/drive_sync.py::sync_connection`) |
|
||||
|
||||
**Provider scope differences:**
|
||||
- Google: same `drive.readonly` scope for personal and company; company sync
|
||||
reads Shared Drives via `corpora=drive` + `supportsAllDrives`.
|
||||
- Microsoft: personal uses `Files.Read`; company uses `Files.Read.All
|
||||
Sites.Read.All` and syncs SharePoint sites (`selected_resource_ids` = site ids).
|
||||
|
||||
**Workers / webhooks (#52):**
|
||||
- `python manage.py sync_drive_connections [--connection-id N]` — cron/worker entry point.
|
||||
- `POST /api/drive/webhooks/google/` / `POST /api/drive/webhooks/microsoft/` —
|
||||
provider push-notification stubs (`AllowAny`); acknowledge `200` and call
|
||||
`sync_connection` when the notification's `connection_id` is resolvable,
|
||||
else just `200` (no-op). Microsoft's subscription-creation `validationToken`
|
||||
handshake is echoed back as `text/plain`.
|
||||
|
||||
Google-native Docs/Sheets/Slides are exported to `.docx`/`.xlsx`/`.pdf` before
|
||||
ingest (Chroma/RAG loaders don't read the native formats). Documents whose
|
||||
remote file was deleted upstream are removed on the next sync.
|
||||
|
||||
## Security note
|
||||
|
||||
Secrets previously hardcoded in `settings.py` (email password, captcha, Django
|
||||
|
||||
Reference in New Issue
Block a user