Fix reindex_embeddings: install unstructured[xlsx] extras #66

2026-08-02T11:55:15-07:00

westfarn commented

2026-08-02 11:55:15 -07:00

Summary

reindex_embeddings crashed with ModuleNotFoundError: No module named 'networkx' when Unstructured hit spreadsheet files via partition/xlsx.py. Reindex clears Chroma first, so a mid-run crash leaves an empty/partial vector store.

True fix

Depend on unstructured[xlsx]==0.18.21 (not bare unstructured). That extra pulls the required spreadsheet partition deps: networkx, msoffcrypto-tool, xlrd (plus openpyxl/pandas already present).

Resilience

Also catch per-document ingest failures so one corrupt/unsupported file cannot abort a full rebuild after Chroma was cleared. Successful files still ingest; failures are logged.

Test plan

CI / unit tests on this PR
Merge + redeploy image to all hosts
On each host: docker compose -p chat_backend_prod exec web bash -lc 'cd /app/llm_be && SKIP_RAG_INIT=1 uv run python manage.py reindex_embeddings'
Confirm non-zero chunk count and no networkx / msoffcrypto / xlrd import errors

Ops workaround (running containers only, until deploy)

uv pip install 'unstructured[xlsx]==0.18.21'
cd /app/llm_be && SKIP_RAG_INIT=1 uv run python manage.py reindex_embeddings

## Summary `reindex_embeddings` crashed with `ModuleNotFoundError: No module named 'networkx'` when Unstructured hit spreadsheet files via `partition/xlsx.py`. Reindex clears Chroma first, so a mid-run crash leaves an empty/partial vector store. ### True fix Depend on **`unstructured[xlsx]==0.18.21`** (not bare `unstructured`). That extra pulls the required spreadsheet partition deps: `networkx`, `msoffcrypto-tool`, `xlrd` (plus openpyxl/pandas already present). ### Resilience Also catch per-document ingest failures so one corrupt/unsupported file cannot abort a full rebuild after Chroma was cleared. Successful files still ingest; failures are logged. ## Test plan - [x] CI / unit tests on this PR - [ ] Merge + redeploy image to all hosts - [ ] On each host: `docker compose -p chat_backend_prod exec web bash -lc 'cd /app/llm_be && SKIP_RAG_INIT=1 uv run python manage.py reindex_embeddings'` - [ ] Confirm non-zero chunk count and no `networkx` / `msoffcrypto` / `xlrd` import errors ## Ops workaround (running containers only, until deploy) ```bash uv pip install 'unstructured[xlsx]==0.18.21' cd /app/llm_be && SKIP_RAG_INIT=1 uv run python manage.py reindex_embeddings ```

westfarn added 1 commit 2026-08-02 11:55:15 -07:00

Fix reindex_embeddings crash: add networkx for Unstructured xlsx.

CI / test (pull_request) Successful in 10s

Details

Unit Tests / test (pull_request) Successful in 10s

Details

8ff3ad83d1

unstructured.partition.xlsx imports networkx; without it reindex dies
mid-run after clearing Chroma. Also skip per-document ingest failures
so one bad file cannot abort the whole rebuild.

westfarn added 1 commit 2026-08-02 11:56:22 -07:00

Depend on unstructured[xlsx] so spreadsheet ingest has required deps.

CI / test (pull_request) Successful in 11s

Details

Unit Tests / test (pull_request) Successful in 10s

Details

f0b8973334

Bare unstructured omits networkx/msoffcrypto/xlrd; xlsx partition then
crashes reindex. The [xlsx] extra is the supported install path.

Co-authored-by: Cursor <cursoragent@cursor.com>

westfarn changed title from ~~Fix reindex_embeddings: add networkx for Unstructured xlsx~~ to Fix reindex_embeddings: install unstructured[xlsx] extras

2026-08-02 11:56:33 -07:00

westfarn merged commit bf3fffa343 into master

2026-08-02 11:57:11 -07:00

westfarn deleted branch fix/reindex-networkx-dep

2026-08-02 11:57:11 -07:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: ai_ml_operations/chat_backend#66