InfoGrep indexes the real content of every PDF, Office doc, note, and image caption in a folder, then makes it searchable by keyword, meaning, or knowledge graph — from a terminal, a browser, or directly as tools your coding agent calls over MCP. Everything runs on your machine; your files are only ever read.
$ git clone https://github.com/QingyaoAi/InfoGrep.git && cd InfoGrep $ uv sync --extra dev $ uv run infogrep index ~/Documents [infogrep] added=4213 modified=0 deleted=0 unchanged=0 $ uv run infogrep search "quarterly OKR draft" 1. [12.840] ~/Documents/planning/2026-okrs.docx (sparse)
Exact keyword, semantic, knowledge-graph, and folder-metadata search — run independently or combined with reciprocal rank fusion. Nothing is indexed into the folder you search.
┌───────────────────────────────────────────────┐
│ MCP server / CLI / browser UI │
│ search_sparse · search_dense · search_kb │
│ search_graph · search_hybrid │
│ index_status · reindex │
└───────────────────────┬─────────────────────────┘
│
┌───────────────┬───────────────────┼────────────────┬───────────────┐
│ │ │ │
┌──────▼──────┐ ┌──────▼──────┐ ┌───────▼──────┐ ┌───────▼──────┐
│ Sparse │ │ Dense │ │ Knowledge │ │ Folder │
│ (Pyserini │ │ (embeddings │ │ base │ │ metadata │
│ BM25, │ │ + Zvec ANN, │ │ (Obsidian │ │ graph │
│ bilingual) │ │off by │ │ graph, live │ │ (paths only, │
│ │ │ default) │ │ vault) │ │ no content) │
└──────┬──────┘ └──────┬──────┘ └───────┬──────┘ └───────┬──────┘
└───────────────┴───────────────────┴────────────────┘
│ reciprocal rank fusion
┌──────▼──────┐
│ Fusion │
└──────┬──────┘
│
┌─────────────▼──────────────┐
│ Side-car index store │
│ ~/.infogrep/indexes/<dir>/ │
└─────────────▲──────────────┘
│
┌───────────────────────────┴────────────────────────────┐
│ Ingestion pipeline │
│ walk (include/exclude globs) → extract (per file type) │
│ → chunk into passages → index (sparse/dense) │
│ → build folder/filename metadata graph │
│ → manifest.sqlite tracks hash/mtime for deltas │
└──────────────────────────────────────────────────────────┘
Traverse the target directory using include/exclude glob patterns, skipping VCS/dependency/cache trees by default.
Pull text per file type — PyMuPDF for PDFs, python-docx/pptx/openpyxl for Office, macOS textutil for legacy .doc, UTF-8 for everything else. Files with no extractable content are still indexed by name/path.
Long documents are split into overlapping passages that preserve page numbers, so results can be cited precisely.
Passages go into a SQLite manifest (for change detection) plus a BM25 Lucene index and, optionally, a Zvec vector store for embeddings.
Every indexed file's path only (never content) contributes to a folder tree, materialized as an Obsidian-compatible vault of linked notes plus a compact JSON index for fast lookups.
Query any retriever directly, or fuse all enabled ones with reciprocal rank fusion — no tuning required. The metadata graph pulls in sibling files from a hit's folder, not just files that matched directly.
A manifest diff finds added/modified/deleted files since the last run, so only the delta gets re-processed. A no-op run does nothing.
grep and Spotlight only see file names or plain text. InfoGrep reads
what's actually inside your files and returns citable, structured results.
PDF (incl. scans via OCR), Word/PowerPoint/Excel, legacy .doc, RTF/OpenDocument, text, markdown, and JSON.
BM25 keyword search, embedding-based semantic search, Obsidian vault graph expansion, and a folder/filename metadata graph — separately or fused via RRF.
Claude Code, Codex, and any MCP-aware agent can call search_hybrid and get back citable path/page/snippet/score results.
Everything runs on your machine. The index lives in a side-car location — your files and folder structure are never touched.
Only changed files are re-processed on each run, and a daily auto-reindex can be scheduled via macOS launchd.
English gets Porter stemming; Chinese/Japanese/Korean get CJK bigram analysis — in a single index, out of the box.
A metadata graph over paths and file names (never content) lets a hit pull in sibling files from its own folder — on by default, no model needed.
Set include = ["**/*"] to index every file instead — anything without a
dedicated extractor is still indexed by name/path.
| Documents |
pdfdocdocxpptpptxxlsxlsxrtfodtodsodp
|
|---|---|
| Text & markup |
txtmdmarkdownrsttexcsvtsvjsonjsonl
|
| Images (name/path; content with OCR) |
pngjpgjpeggifbmptiftiffwebpsvgheicheif
|
The same sparse/dense/knowledge-base/folder-graph retrieval, from wherever you're working.
infogrep index <dir> # build / update the index for a directory infogrep search <query> -d <dir> # query (modes: hybrid [default] | sparse | dense | kb | graph) infogrep search <query> --prf # sparse query expansion (RM3) infogrep status <dir> # index status + staleness (pending changes) infogrep mcp --dir <dir> # run the MCP server (stdio) for coding agents infogrep serve --dir <dir> # browser UI to test search (http://127.0.0.1:7421) infogrep schedule install <dir> --at 03:00 # daily auto-reindex via launchd
claude mcp add infogrep -- uv run infogrep mcp --dir /path/to/your/project # Tools exposed: # search_sparse · search_dense · search_kb · search_graph # search_hybrid · index_status · reindex # # Each result carries: path, page, snippet, score, retriever # search_hybrid fuses enabled retrievers with reciprocal rank fusion # and reports which were used / skipped (and why).
infogrep serve --dir <dir> # http://127.0.0.1:7421 by default # Search box · mode selector (hybrid/sparse/dense/kb/graph) # result snippets with path/page/score · add/switch folders # JSON API: /api/search /api/status (bound to localhost)
[sparse]
enabled = true
language = "en+zh" # "en" | "zh" | "ja" | "ko" | "en+zh"
field_boosts = { contents = 1.0, filename = 2.0, pathtext = 1.0 }
[dense]
enabled = false # off by default — needs a model + RAM/GPU
model_name = "Qwen/Qwen3-Embedding-0.6B"
[kb]
enabled = false # Obsidian vault graph search
vault = "My Vault"
hops = 1
[graph]
enabled = true # folder/filename metadata graph — cheap, on by default
hops = 1 # folder hops to expand from a matched folder
max_folders = 5
[ingest]
ocr = false # OCR scanned PDF pages (needs tesseract)
Clone the repo, run the installer for a Spotlight-style menu-bar app, or wire the MCP server straight into your coding agent.