Module migrate 

Database schema migrations.

Creates all required tables and ensures idempotent execution. Designed to be run via ctx init.

Schema

┌──────────────┐     ┌──────────────┐     ┌──────────────┐
│  documents   │──┐  │   chunks     │──┐  │  embeddings  │
│              │  │  │              │  │  │              │
│ id (PK)      │  │  │ id (PK)      │  │  │ chunk_id(PK) │
│ source       │  └──│ document_id  │  └──│ model        │
│ source_id    │     │ chunk_index  │     │ dims         │
│ source_url   │     │ text         │     │ created_at   │
│ title        │     │ hash         │     │ hash         │
│ author       │     └──────────────┘     └──────────────┘
│ created_at   │
│ updated_at   │     ┌──────────────┐     ┌──────────────┐
│ content_type │     │  chunks_fts  │     │chunk_vectors │
│ body         │     │  (FTS5)      │     │              │
│ metadata_json│     │ chunk_id     │     │ chunk_id(PK) │
│ raw_json     │     │ document_id  │     │ document_id  │
│ dedup_hash   │     │ text         │     │ embedding    │
└──────────────┘     └──────────────┘     └──────────────┘

┌──────────────┐
│ checkpoints  │
│              │
│ source (PK)  │
│ cursor       │
│ updated_at   │
└──────────────┘

Tables

Table	Purpose
documents	Normalized document metadata and body text
chunks	Text segments with content hashes
checkpoints	Incremental sync cursors per connector
chunks_fts	FTS5 full-text index over chunk text (BM25)
embeddings	Embedding metadata (model, dims, hash)
chunk_vectors	Embedding vectors stored as BLOBs

Indexes

• idx_chunks_document_id — fast chunk lookup by document
• idx_documents_source — fast document filtering by connector
• idx_documents_updated_at — efficient date range queries
• idx_chunk_vectors_document_id — fast vector lookup by document

Idempotency

All operations use CREATE TABLE IF NOT EXISTS or check for existing objects before creation. Running ctx init multiple times is safe.

Functions

run_migrations

Run all database migrations.