beyhan/graphify
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

simplify README, move technical details to docs/how-it-works.md

Browse Source 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Safi
2026-05-03 13:31:28 +01:00
co-authored by Claude Sonnet 4.6
parent d753413409
commit b11a8a7813
2 changed files with 243 additions and 314 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+153 -314
View File
@@ -21,67 +21,45 @@
</a> </a>
</p> </p>
**An AI coding assistant skill.** Type `/graphify` in Claude Code, Codex, OpenCode, Cursor, Gemini CLI, GitHub Copilot CLI, VS Code Copilot Chat, Aider, OpenClaw, Factory Droid, Trae, Hermes, Kiro, Pi, or Google Antigravity - it reads your files, builds a knowledge graph, and gives you back structure you didn't know was there. Understand a codebase faster. Find the "why" behind architectural decisions. Type `/graphify` in your AI coding assistant and it maps your entire project — code, docs, PDFs, images, videos — into a knowledge graph you can query instead of grepping through files.
Fully multimodal. Drop in code, PDFs, markdown, screenshots, diagrams, whiteboard photos, images in other languages, or video and audio files - graphify extracts concepts and relationships from all of it and connects them into one graph. Videos are transcribed with Whisper using a domain-aware prompt derived from your corpus. YAML/YML files (Kubernetes, Kustomize, Helm, config) are indexed for semantic extraction. SQL files are AST-extracted deterministically — tables, views, functions, foreign keys, and FROM/JOIN relationships map directly into the graph with no LLM needed. 25 languages supported via tree-sitter AST (Python, JS, TS, Go, Rust, Java, C, C++, Ruby, C#, Kotlin, Scala, PHP, Swift, Lua, Zig, PowerShell, Elixir, Objective-C, Julia, Verilog, SystemVerilog, Vue, Svelte, Dart). Works in Claude Code, Codex, OpenCode, Cursor, Gemini CLI, GitHub Copilot CLI, VS Code Copilot Chat, Aider, OpenClaw, Factory Droid, Trae, Hermes, Kiro, Pi, and Google Antigravity.
> Andrej Karpathy keeps a `/raw` folder where he drops papers, tweets, screenshots, and notes. graphify is the answer to that problem - 71.5x fewer tokens per query vs reading the raw files, persistent across sessions, honest about what it found vs guessed.
``` ```
/graphify . # works on any folder - your codebase, notes, papers, anything /graphify .
``` ```
That's it. You get three files:
``` ```
graphify-out/ graphify-out/
├── graph.html interactive graph - open in any browser, click nodes, search, filter by community ├── graph.html open in any browser — click nodes, filter, search
├── GRAPH_REPORT.md god nodes, surprising connections, suggested questions ├── GRAPH_REPORT.md the highlights: key concepts, surprising connections, suggested questions
├── graph.json persistent graph - query weeks later without re-reading └── graph.json the full graph — query it anytime without re-reading your files
└── cache/ SHA256 cache - re-runs only process changed files
``` ```
Add a `.graphifyignore` file to exclude folders you don't want in the graph: ---
```
# .graphifyignore
vendor/
node_modules/
dist/
*.generated.py
```
Same syntax as `.gitignore` — including `!` negation patterns to re-include specific files. You can keep a single `.graphifyignore` at your repo root — patterns work correctly even when graphify is run on a subfolder. Discovery never crosses a VCS boundary (`.git`, `.hg`, etc.), so sibling projects in a shared workspace don't leak rules into each other. Without a VCS root, only the scan folder's own `.graphifyignore` applies. Inline comments are supported: `vendor/ # legacy deps`.
## How it works
graphify runs in three passes. First, a deterministic AST pass extracts structure from code files (classes, functions, imports, call graphs, docstrings, rationale comments) with no LLM needed. Second, video and audio files are transcribed locally with faster-whisper using a domain-aware prompt derived from corpus god nodes — transcripts are cached so re-runs are instant. Third, Claude subagents run in parallel over docs, papers, images, and transcripts to extract concepts, relationships, and design rationale. The results are merged into a NetworkX graph, clustered with Leiden community detection, and exported as interactive HTML, queryable JSON, and a plain-language audit report.
**Clustering is graph-topology-based — no embeddings.** Leiden finds communities by edge density. The semantic similarity edges that Claude extracts (`semantically_similar_to`, marked INFERRED) are already in the graph, so they influence community detection directly. The graph structure is the similarity signal — no separate embedding step or vector database needed.
Every relationship is tagged `EXTRACTED` (found directly in source), `INFERRED` (reasonable inference, with a confidence score), or `AMBIGUOUS` (flagged for review). You always know what was found vs guessed.
## Install ## Install
**Requires:** Python 3.10+ and one of: [Claude Code](https://claude.ai/code), [Codex](https://openai.com/codex), [OpenCode](https://opencode.ai), [Cursor](https://cursor.com), [Gemini CLI](https://github.com/google-gemini/gemini-cli), [GitHub Copilot CLI](https://docs.github.com/en/copilot/how-tos/copilot-cli), [VS Code Copilot Chat](https://code.visualstudio.com/docs/copilot/overview), [Aider](https://aider.chat), [OpenClaw](https://openclaw.ai), [Factory Droid](https://factory.ai), [Trae](https://trae.ai), [Kiro](https://kiro.dev), [Pi](https://pi.dev), Hermes, or [Google Antigravity](https://antigravity.google) **Requires Python 3.10+**
```bash ```bash
# Recommended — works on Mac and Linux with no PATH setup needed
uv tool install graphifyy && graphify install uv tool install graphifyy && graphify install
# or with pipx # or: pipx install graphifyy && graphify install
pipx install graphifyy && graphify install # or: pip install graphifyy && graphify install
# or plain pip
pip install graphifyy && graphify install
``` ```
> **Official package:** The PyPI package is named `graphifyy` (install with `pip install graphifyy`). Other packages named `graphify*` on PyPI are not affiliated with this project. The only official repository is [safishamsi/graphify](https://github.com/safishamsi/graphify). The CLI and skill command are still `graphify`. > **Official package:** The PyPI package is `graphifyy` (double-y). Other `graphify*` packages on PyPI are not affiliated. The CLI command is still `graphify`.
> **`graphify: command not found`?** Use `uv tool install graphifyy` (recommended) or `pipx install graphifyy` — both put the CLI in a managed location that's automatically on PATH. With plain `pip`, you may need to add `~/.local/bin` (Linux) or `~/Library/Python/3.x/bin` (Mac) to your PATH, or run `python -m graphify` instead. On Windows, pip scripts land in `%APPDATA%\Python\PythonXY\Scripts`. > **`graphify: command not found`?** Use `uv tool install graphifyy` or `pipx install graphifyy` — both put the CLI on PATH automatically. With plain `pip`, add `~/.local/bin` (Linux) or `~/Library/Python/3.x/bin` (Mac) to your PATH, or run `python -m graphify`.
### Platform support ### Pick your platform
| Platform | Install command | | Platform | Install command |
|----------|----------------| |----------|----------------|
| Claude Code (Linux/Mac) | `graphify install` | | Claude Code (Linux/Mac) | `graphify install` |
| Claude Code (Windows) | `graphify install` (auto-detected) or `graphify install --platform windows` | | Claude Code (Windows) | `graphify install --platform windows` |
| Codex | `graphify install --platform codex` | | Codex | `graphify install --platform codex` |
| OpenCode | `graphify install --platform opencode` | | OpenCode | `graphify install --platform opencode` |
| GitHub Copilot CLI | `graphify install --platform copilot` | | GitHub Copilot CLI | `graphify install --platform copilot` |
@@ -98,19 +76,14 @@ pip install graphifyy && graphify install
| Cursor | `graphify cursor install` | | Cursor | `graphify cursor install` |
| Google Antigravity | `graphify antigravity install` | | Google Antigravity | `graphify antigravity install` |
Codex users also need `multi_agent = true` under `[features]` in `~/.codex/config.toml` for parallel extraction. Factory Droid uses the `Task` tool for parallel subagent dispatch. OpenClaw and Aider use sequential extraction (parallel agent support is still early on those platforms). Trae uses the Agent tool for parallel subagent dispatch and does **not** support PreToolUse hooks — AGENTS.md is the always-on mechanism. Codex supports PreToolUse hooks — `graphify codex install` installs one in `.codex/hooks.json` in addition to writing AGENTS.md. > Codex users: also add `multi_agent = true` under `[features]` in `~/.codex/config.toml`.
> Codex uses `$graphify` instead of `/graphify`.
Then open your AI coding assistant and type: ---
``` ## Make your assistant always use the graph
/graphify .
```
Note: Codex uses `$` instead of `/` for skill calling, so type `$graphify .` instead. Run this once in your project after building a graph:
### Make your assistant always use the graph (recommended)
After building a graph, run this once in your project:
| Platform | Command | | Platform | Command |
|----------|---------| |----------|---------|
@@ -131,343 +104,209 @@ After building a graph, run this once in your project:
| Pi coding agent | `graphify pi install` | | Pi coding agent | `graphify pi install` |
| Google Antigravity | `graphify antigravity install` | | Google Antigravity | `graphify antigravity install` |
**Claude Code** does two things: writes a `CLAUDE.md` section telling Claude to read `graphify-out/GRAPH_REPORT.md` before answering architecture questions, and installs a **PreToolUse hook** (`settings.json`) that fires before every Glob and Grep call. If a knowledge graph exists, Claude sees: _"graphify: Knowledge graph exists. Read GRAPH_REPORT.md for god nodes and community structure before searching raw files."_ — so Claude navigates via the graph instead of grepping through every file. This writes a small config file that tells your assistant to read `GRAPH_REPORT.md` before answering questions about your codebase. On platforms that support hooks (Claude Code, Codex, Gemini CLI), a hook fires automatically before every file-read call — your assistant navigates by the graph instead of grepping through everything.
**Codex** writes to `AGENTS.md` and also installs a **PreToolUse hook** in `.codex/hooks.json` that fires before every Bash tool call — same always-on mechanism as Claude Code. Uninstall with the matching command (e.g. `graphify claude uninstall`).
**OpenCode** writes to `AGENTS.md` and also installs a **`tool.execute.before` plugin** (`.opencode/plugins/graphify.js` + `opencode.json` registration) that fires before bash tool calls and injects the graph reminder into tool output when the graph exists. ---
**Cursor** writes `.cursor/rules/graphify.mdc` with `alwaysApply: true` — Cursor includes it in every conversation automatically, no hook needed. ## What's in the report
**Gemini CLI** copies the skill to `~/.gemini/skills/graphify/SKILL.md`, writes a `GEMINI.md` section, and installs a `BeforeTool` hook in `.gemini/settings.json` that fires before file-read tool calls — same always-on mechanism as Claude Code. - **God nodes** — the most-connected concepts in your project. Everything flows through these.
- **Surprising connections** — links between things that live in different files or modules. Ranked by how unexpected they are.
- **The "why"** — inline comments (`# NOTE:`, `# WHY:`, `# HACK:`), docstrings, and design rationale from docs are extracted as separate nodes linked to the code they explain.
- **Suggested questions** — 4–5 questions the graph is uniquely positioned to answer.
- **Confidence tags** — every inferred relationship is marked `EXTRACTED`, `INFERRED`, or `AMBIGUOUS`. You always know what was found vs guessed.
**Aider, OpenClaw, Factory Droid, Trae, and Hermes** write the same rules to `AGENTS.md` in your project root and copy the skill to the platform's global skill directory. These platforms don't support tool hooks, so AGENTS.md is the always-on mechanism. ---
**Pi coding agent** copies the skill to `~/.pi/agent/skills/graphify/SKILL.md`. Run `graphify pi install` to set it up. ## What files it handles
**Kiro IDE/CLI** writes the skill to `.kiro/skills/graphify/SKILL.md` (invoked via `/graphify`) and a steering file to `.kiro/steering/graphify.md` with `inclusion: always` — Kiro injects this into every conversation automatically, no hook needed. | Type | Extensions |
|------|-----------|
| Code (25 languages) | `.py .ts .js .jsx .tsx .go .rs .java .c .cpp .rb .cs .kt .scala .php .swift .lua .zig .ps1 .ex .exs .m .jl .vue .svelte .sql` |
| Docs | `.md .mdx .html .txt .rst .yaml .yml` |
| Office | `.docx .xlsx` (requires `pip install graphifyy[office]`) |
| PDFs | `.pdf` |
| Images | `.png .jpg .webp .gif` |
| Video / Audio | `.mp4 .mov .mp3 .wav` and more (requires `pip install graphifyy[video]`) |
| YouTube / URLs | any video URL (requires `pip install graphifyy[video]`) |
**Google Antigravity** writes `.agents/rules/graphify.md` (always-on rules) and `.agents/workflows/graphify.md` (registers `/graphify` as a slash command). No hook equivalent exists in Antigravity — rules are the always-on mechanism. Code is extracted locally with no API calls (AST via tree-sitter). Everything else goes through your AI assistant's model API.
**GitHub Copilot CLI** copies the skill to `~/.copilot/skills/graphify/SKILL.md`. Run `graphify copilot install` to set it up. ---
**VS Code Copilot Chat** installs a Python-only skill (works on Windows PowerShell and macOS/Linux alike) and writes `.github/copilot-instructions.md` in your project root — VS Code reads this automatically every session, making graph context always-on without any hook mechanism. Run `graphify vscode install`. Note: this configures the chat panel in VS Code, not the Copilot CLI terminal tool. ## Common commands
Uninstall with the matching uninstall command (e.g. `graphify claude uninstall`). ```bash
/graphify . # build graph for current folder
/graphify ./docs --update # re-extract only changed files
/graphify . --cluster-only # rerun clustering without re-extracting
/graphify . --no-viz # skip the HTML, just the report + JSON
/graphify . --wiki # build a markdown wiki from the graph
**Always-on vs explicit trigger — what's the difference?** /graphify query "what connects auth to the database?"
/graphify path "UserService" "DatabasePool"
/graphify explain "RateLimiter"
The always-on hook surfaces `GRAPH_REPORT.md` — a one-page summary of god nodes, communities, and surprising connections. Your assistant reads this before searching files, so it navigates by structure instead of keyword matching. That covers most everyday questions. /graphify add https://arxiv.org/abs/1706.03762 # fetch a paper and add it
/graphify add <youtube-url> # transcribe and add a video
`/graphify query`, `/graphify path`, and `/graphify explain` go deeper: they traverse the raw `graph.json` hop by hop, trace exact paths between nodes, and surface edge-level detail (relation type, confidence score, source location). Use them when you want a specific question answered from the graph rather than a general orientation. graphify hook install # auto-rebuild on git commit
graphify merge-graphs a.json b.json # combine two graphs
```
Think of it this way: the always-on hook gives your assistant a map. The `/graphify` commands let it navigate the map precisely. See the [full command reference](#full-command-reference) below.
### Team workflows ---
`graphify-out/` is designed to be committed to git so every teammate starts with a fresh map. ## Ignoring files
Create a `.graphifyignore` in your project root — same syntax as `.gitignore`, including `!` negation:
```
# .graphifyignore
node_modules/
dist/
*.generated.py
# only index src/, ignore everything else
*
!src/
!src/**
```
---
## Team setup
`graphify-out/` is meant to be committed to git so everyone on the team starts with a map.
**Recommended `.gitignore` additions:** **Recommended `.gitignore` additions:**
``` ```
# keep graph outputs, skip heavy/local-only files graphify-out/manifest.json # mtime-based, breaks after git clone
graphify-out/cost.json # local only
# optional: commit for shared extraction speed, skip to keep repo small # graphify-out/cache/ # optional: commit for speed, skip to keep repo small
graphify-out/cache/
# mtime-based, invalid after git clone - always gitignore this
graphify-out/manifest.json
# local token tracking, not useful to share
graphify-out/cost.json
``` ```
**Shared setup:** **Workflow:**
1. One person runs `/graphify .` to build the initial graph and commits `graphify-out/`. 1. One person runs `/graphify .` and commits `graphify-out/`.
2. Everyone else pulls — their assistant reads `GRAPH_REPORT.md` immediately with no extra steps. 2. Everyone pulls — their assistant reads the graph immediately.
3. Install the post-commit hook (`graphify hook install`) so the graph rebuilds automatically after code changes — no LLM calls needed for code-only updates. 3. Run `graphify hook install` to auto-rebuild after each commit (AST only, no API cost).
4. For doc/paper changes, whoever edits the files runs `/graphify --update` to refresh semantic nodes. 4. When docs or papers change, run `/graphify --update` to refresh those nodes.
**Excluding paths** — create `.graphifyignore` in your project root (same syntax as `.gitignore`). Files matching those patterns are skipped during detection and extraction. ---
``` ## Using the graph directly
# .graphifyignore example
AGENTS.md # graphify install files — don't extract your own instructions as knowledge
CLAUDE.md
GEMINI.md
.gemini/
.opencode/
docs/translations/ # generated content you don't want in the graph
```
## Using `graph.json` with an LLM
`graph.json` is not meant to be pasted into a prompt all at once. The useful
workflow is:
1. Start with `graphify-out/GRAPH_REPORT.md` for the high-level overview.
2. Use `graphify query` to pull a smaller subgraph for the specific question
you want to answer.
3. Give that focused output to your assistant instead of dumping the full raw
corpus.
For example, after running graphify on a project:
```bash ```bash
graphify query "show the auth flow" --graph graphify-out/graph.json # query the graph from the terminal
graphify query "show the auth flow"
graphify query "what connects DigestAuth to Response?" --graph graphify-out/graph.json graphify query "what connects DigestAuth to Response?" --graph graphify-out/graph.json
```
The output includes node labels, edge types, confidence tags, source files, and # expose the graph as an MCP server (for repeated tool-call access)
source locations. That makes it a good intermediate context block for an LLM:
```text
Use this graph query output to answer the question. Prefer the graph structure
over guessing, and cite the source files when possible.
```
If your assistant supports tool calling or MCP, use the graph directly instead
of pasting text. graphify can expose `graph.json` as an MCP server:
```bash
python -m graphify.serve graphify-out/graph.json python -m graphify.serve graphify-out/graph.json
``` ```
That gives the assistant structured graph access for repeated queries such as The MCP server gives your assistant structured access: `query_graph`, `get_node`, `get_neighbors`, `shortest_path`.
`query_graph`, `get_node`, `get_neighbors`, and `shortest_path`.
> **WSL / Linux note:** Ubuntu ships `python3`, not `python`. Install into a project venv to avoid PEP 668 conflicts, and use the full venv path in your `.mcp.json`: > **WSL / Linux note:** Ubuntu ships `python3`, not `python`. Use a venv to avoid conflicts:
> ```bash > ```bash
> python3 -m venv .venv && .venv/bin/pip install "graphifyy[mcp]" > python3 -m venv .venv && .venv/bin/pip install "graphifyy[mcp]"
> ``` > ```
> ```json
> { "mcpServers": { "graphify": { "type": "stdio", "command": ".venv/bin/python3", "args": ["-m", "graphify.serve", "graphify-out/graph.json"] } } }
> ```
> Also note: the PyPI package is `graphifyy` (double-y) — `pip install graphify` installs an unrelated package.
<details> ---
<summary>Manual install (curl)</summary>
```bash ## Privacy
mkdir -p ~/.claude/skills/graphify
curl -fsSL https://raw.githubusercontent.com/safishamsi/graphify/v4/graphify/skill.md \
> ~/.claude/skills/graphify/SKILL.md
```
Add to `~/.claude/CLAUDE.md`: - **Code files** — processed locally via tree-sitter. Nothing leaves your machine.
- **Video / audio** — transcribed locally with faster-whisper. Nothing leaves your machine.
- **Docs, PDFs, images** — sent to your AI assistant's model API (Anthropic, OpenAI, etc.) using your own API key.
- No telemetry, no usage tracking, no analytics.
``` ---
- **graphify** (`~/.claude/skills/graphify/SKILL.md`) - any input to knowledge graph. Trigger: `/graphify`
When the user types `/graphify`, invoke the Skill tool with `skill: "graphify"` before doing anything else.
```
</details> ## Full command reference
## Usage
``` ```
/graphify # run on current directory /graphify # run on current directory
/graphify ./raw # run on a specific folder /graphify ./raw # run on a specific folder
/graphify ./raw --mode deep # more aggressive INFERRED edge extraction /graphify ./raw --mode deep # more aggressive relationship extraction
/graphify ./raw --update # re-extract only changed files, merge into existing graph /graphify ./raw --update # re-extract only changed files
/graphify ./raw --directed # build directed graph (preserves edge direction: source→target) /graphify ./raw --directed # preserve edge direction
/graphify ./raw --cluster-only # rerun clustering on existing graph, no re-extraction /graphify ./raw --cluster-only # rerun clustering on existing graph
/graphify ./raw --no-viz # skip HTML, just produce report + JSON /graphify ./raw --no-viz # skip HTML visualization
/graphify ./raw --obsidian # also generate Obsidian vault (opt-in) /graphify ./raw --obsidian # generate Obsidian vault
/graphify ./raw --obsidian --obsidian-dir ~/vaults/myproject # write vault to a specific directory /graphify ./raw --wiki # build agent-crawlable markdown wiki
/graphify ./raw --svg # export graph.svg
/graphify ./raw --graphml # export for Gephi / yEd
/graphify ./raw --neo4j # generate cypher.txt for Neo4j
/graphify ./raw --neo4j-push bolt://localhost:7687
/graphify ./raw --watch # auto-sync as files change
/graphify ./raw --mcp # start MCP stdio server
/graphify add https://arxiv.org/abs/1706.03762 # fetch a paper, save, update graph /graphify add https://arxiv.org/abs/1706.03762
/graphify add https://x.com/karpathy/status/... # fetch a tweet /graphify add <video-url>
/graphify add <video-url> # download audio, transcribe, add to graph /graphify add https://... --author "Name" --contributor "Name"
/graphify add https://... --author "Name" # tag the original author
/graphify add https://... --contributor "Name" # tag who added it to the corpus
/graphify query "what connects attention to the optimizer?" /graphify query "what connects attention to the optimizer?"
/graphify query "what connects attention to the optimizer?" --dfs # trace a specific path /graphify query "..." --dfs --budget 1500
/graphify query "what connects attention to the optimizer?" --budget 1500 # cap at N tokens
/graphify path "DigestAuth" "Response" /graphify path "DigestAuth" "Response"
/graphify explain "SwinTransformer" /graphify explain "SwinTransformer"
/graphify ./raw --watch # auto-sync graph as files change (code: instant, docs: notifies you) graphify hook install # post-commit + post-checkout hooks
/graphify ./raw --wiki # build agent-crawlable wiki (index.md + article per community)
/graphify ./raw --svg # export graph.svg
/graphify ./raw --graphml # export graph.graphml (Gephi, yEd)
/graphify ./raw --neo4j # generate cypher.txt for Neo4j
/graphify ./raw --neo4j-push bolt://localhost:7687 # push directly to a running Neo4j instance
/graphify ./raw --mcp # start MCP stdio server
# git hooks - platform-agnostic, rebuild graph on commit and branch switch
graphify hook install
graphify hook uninstall graphify hook uninstall
graphify hook status graphify hook status
# always-on assistant instructions - platform-specific graphify claude install / uninstall
graphify claude install # CLAUDE.md + PreToolUse hook (Claude Code) graphify codex install / uninstall
graphify claude uninstall graphify opencode install
graphify codex install # AGENTS.md + PreToolUse hook in .codex/hooks.json (Codex) graphify cursor install / uninstall
graphify opencode install # AGENTS.md + tool.execute.before plugin (OpenCode) graphify gemini install / uninstall
graphify cursor install # .cursor/rules/graphify.mdc (Cursor) graphify copilot install / uninstall
graphify cursor uninstall graphify aider install / uninstall
graphify gemini install # GEMINI.md + BeforeTool hook (Gemini CLI) graphify claw install / uninstall
graphify gemini uninstall graphify droid install / uninstall
graphify copilot install # skill file (GitHub Copilot CLI) graphify trae install / uninstall
graphify copilot uninstall graphify trae-cn install / uninstall
graphify aider install # AGENTS.md (Aider) graphify hermes install / uninstall
graphify aider uninstall graphify kiro install / uninstall
graphify claw install # AGENTS.md (OpenClaw) graphify antigravity install / uninstall
graphify droid install # AGENTS.md (Factory Droid)
graphify trae install # AGENTS.md (Trae)
graphify trae uninstall
graphify trae-cn install # AGENTS.md (Trae CN)
graphify trae-cn uninstall
graphify hermes install # AGENTS.md + ~/.hermes/skills/ (Hermes)
graphify hermes uninstall
graphify kiro install # .kiro/skills/ + .kiro/steering/graphify.md (Kiro IDE/CLI)
graphify kiro uninstall
graphify antigravity install # .agents/rules + .agents/workflows (Google Antigravity)
graphify antigravity uninstall
# query and navigate the graph directly from the terminal (no AI assistant needed) graphify clone https://github.com/karpathy/nanoGPT
graphify query "what connects attention to the optimizer?" graphify merge-graphs a.json b.json --out merged.json
graphify query "show the auth flow" --dfs graphify watch ./src
graphify query "what is CfgNode?" --budget 500 graphify check-update ./src
graphify query "..." --graph path/to/graph.json graphify update ./src
graphify path "DigestAuth" "Response" # shortest path between two nodes graphify cluster-only ./my-project
graphify explain "SwinTransformer" # plain-language explanation of a node
# add content and update the graph from the terminal
graphify add https://arxiv.org/abs/1706.03762 # fetch paper, save to ./raw, update graph
graphify add https://... --author "Name" --contributor "Name"
# clone any GitHub repo and run the full pipeline on it
graphify clone https://github.com/karpathy/nanoGPT # clones to ~/.graphify/repos/karpathy/nanoGPT
graphify clone https://github.com/org/repo --branch dev --out ./my-clone
# cross-repo graphs — merge two or more graph.json outputs into one
graphify merge-graphs repo1/graphify-out/graph.json repo2/graphify-out/graph.json
graphify merge-graphs g1.json g2.json g3.json --out cross-repo.json
# incremental update and maintenance
graphify watch ./src # auto-rebuild on code changes
graphify check-update ./src # check if semantic re-extraction is pending (cron-safe)
graphify update ./src # re-extract code files, no LLM needed
graphify cluster-only ./my-project # rerun clustering on existing graph.json
``` ```
Works with any mix of file types: ---
| Type | Extensions | Extraction | ## Learn more
|------|-----------|------------|
| Code | `.py .ts .js .jsx .tsx .mjs .go .rs .java .c .cpp .rb .cs .kt .scala .php .swift .lua .zig .ps1 .ex .exs .m .mm .jl .vue .svelte .sql` | AST via tree-sitter + call-graph (cross-file for all languages) + Java extends/implements + docstring/comment rationale. SQL: tables, views, functions, foreign keys, FROM/JOIN edges (requires `pip install graphifyy[sql]`) |
| Docs | `.md .mdx .html .txt .rst .yaml .yml` | Concepts + relationships + design rationale via Claude |
| Office | `.docx .xlsx` | Converted to markdown then extracted via Claude (requires `pip install graphifyy[office]`) |
| Papers | `.pdf` | Citation mining + concept extraction |
| Images | `.png .jpg .webp .gif` | Claude vision - screenshots, diagrams, any language |
| Video / Audio | `.mp4 .mov .mkv .webm .avi .m4v .mp3 .wav .m4a .ogg` | Transcribed locally with faster-whisper, transcript fed into Claude extraction (requires `pip install graphifyy[video]`) |
| YouTube / URLs | any video URL | Audio downloaded via yt-dlp, then same Whisper pipeline (requires `pip install graphifyy[video]`) |
## Video and audio corpus - [How it works](docs/how-it-works.md) — the extraction pipeline, community detection, confidence scoring, benchmarks
- [ARCHITECTURE.md](ARCHITECTURE.md) — module breakdown, how to add a language
- [Optional integrations](docs/docker-mcp-sqlite.md) — Docker MCP Toolkit + SQLite
Drop video or audio files into your corpus folder alongside your code and docs — graphify picks them up automatically: ---
```bash
pip install 'graphifyy[video]' # one-time setup
/graphify ./my-corpus # transcribes any video/audio files it finds
```
Add a YouTube video (or any public video URL) directly:
```bash
/graphify add <video-url>
```
yt-dlp downloads audio-only (fast, small), Whisper transcribes it locally, and the transcript is fed into the same extraction pipeline as your other docs. Transcripts are cached in `graphify-out/transcripts/` so re-runs skip already-transcribed files.
For better accuracy on technical content, use a larger model:
```bash
/graphify ./my-corpus --whisper-model medium
```
Audio never leaves your machine. All transcription runs locally.
> **Legal notice:** Only use `/graphify add <url>` to download content you have the rights to. graphify uses yt-dlp for audio extraction — the same terms of service and copyright rules apply.
## What you get
**God nodes** - highest-degree concepts (what everything connects through)
**Surprising connections** - ranked by composite score. Code-paper edges rank higher than code-code. Each result includes a plain-English why.
**Suggested questions** - 4-5 questions the graph is uniquely positioned to answer
**The "why"** - docstrings, inline comments (`# NOTE:`, `# IMPORTANT:`, `# HACK:`, `# WHY:`), and design rationale from docs are extracted as `rationale_for` nodes. Not just what the code does - why it was written that way.
**Confidence scores** - every INFERRED edge has a `confidence_score` (0.0-1.0). You know not just what was guessed but how confident the model was. EXTRACTED edges are always 1.0.
**Semantic similarity edges** - cross-file conceptual links with no structural connection. Two functions solving the same problem without calling each other, a class in code and a concept in a paper describing the same algorithm.
**Hyperedges** - group relationships connecting 3+ nodes that pairwise edges can't express. All classes implementing a shared protocol, all functions in an auth flow, all concepts from a paper section forming one idea.
**Token benchmark** - printed automatically after every run. On a mixed corpus (Karpathy repos + papers + images): **71.5x** fewer tokens per query vs reading raw files. The first run extracts and builds the graph (this costs tokens). Every subsequent query reads the compact graph instead of raw files — that's where the savings compound. The SHA256 cache means re-runs only re-process changed files.
**Auto-sync** (`--watch`) - run in a background terminal and the graph updates itself as your codebase changes. Code file saves trigger an instant rebuild (AST only, no LLM). Doc/image changes notify you to run `--update` for the LLM re-pass.
**Git hooks** (`graphify hook install`) - installs post-commit and post-checkout hooks. Graph rebuilds automatically after every commit and every branch switch. Rebuilds run detached in the background so `git commit` returns instantly — log written to `~/.cache/graphify-rebuild.log` (`tail -f` for status).
**Wiki** (`--wiki`) - Wikipedia-style markdown articles per community and god node, with an `index.md` entry point. Point any agent at `index.md` and it can navigate the knowledge base by reading files instead of parsing JSON.
## Worked examples
| Corpus | Files | Reduction | Output |
|--------|-------|-----------|--------|
| Karpathy repos + 5 papers + 4 images | 52 | **71.5x** | [`worked/karpathy-repos/`](worked/karpathy-repos/) |
| graphify source + Transformer paper | 4 | **5.4x** | [`worked/mixed-corpus/`](worked/mixed-corpus/) |
| httpx (synthetic Python library) | 6 | ~1x | [`worked/httpx/`](worked/httpx/) |
Token reduction scales with corpus size. 6 files fits in a context window anyway, so graph value there is structural clarity, not compression. At 52 files (code + papers + images) you get 71x+. Each `worked/` folder has the raw input files and the actual output (`GRAPH_REPORT.md`, `graph.json`) so you can run it yourself and verify the numbers.
## Privacy
graphify sends file contents to your AI coding assistant's underlying model API for semantic extraction of docs, papers, and images — Anthropic (Claude Code), OpenAI (Codex), or whichever provider your platform uses. Code files are processed locally via tree-sitter AST — no file contents leave your machine for code. Video and audio files are transcribed locally with faster-whisper — audio never leaves your machine. No telemetry, usage tracking, or analytics of any kind. The only network calls are to your platform's model API during extraction, using your own API key.
## Optional integrations
Runbooks for setting up extra tooling alongside graphify. None of these are required.
| Integration | Doc |
|---|---|
| Docker MCP Toolkit + SQLite MCP server (lightweight persistent SQL workspace exposed to any MCP client) | [`docs/docker-mcp-sqlite.md`](docs/docker-mcp-sqlite.md) |
## Tech stack
NetworkX + Leiden (graspologic) + tree-sitter + vis.js. Semantic extraction via Claude (Claude Code), GPT-4 (Codex), or whichever model your platform runs. Video transcription via faster-whisper + yt-dlp (optional, `pip install graphifyy[video]`). No Neo4j required, no server, runs entirely locally.
## Built on graphify — Penpax ## Built on graphify — Penpax
[**Penpax**](https://safishamsi.github.io/penpax.ai) is the enterprise layer on top of graphify. Where graphify turns a folder of files into a knowledge graph, Penpax applies the same graph to your entire working life — continuously. [**Penpax**](https://safishamsi.github.io/penpax.ai) is the always-on layer built on top of graphify — it applies the same graph approach to your entire working life: meetings, browser history, emails, files, and code, updating continuously in the background.
| | graphify | Penpax | Built for people whose work lives across hundreds of conversations and documents they can never fully reconstruct. No cloud, fully on-device.
|---|---|---|
| Input | A folder of files | Browser history, meetings, emails, files, code — everything |
| Runs | On demand | Continuously in the background |
| Scope | A project | Your entire working life |
| Query | CLI / MCP / AI skill | Natural language, always on |
| Privacy | Local by default | Fully on-device, no cloud |
Built for lawyers, consultants, executives, doctors, researchers — anyone whose work lives across hundreds of conversations and documents they can never fully reconstruct.
**Free trial launching soon.** [Join the waitlist →](https://safishamsi.github.io/penpax.ai) **Free trial launching soon.** [Join the waitlist →](https://safishamsi.github.io/penpax.ai)
## What we are building next ---
graphify is the graph layer. Penpax is the always-on layer on top of it — an on-device digital twin that connects your meetings, browser history, files, emails, and code into one continuously updating knowledge graph. No cloud, no training on your data. [Join the waitlist.](https://safishamsi.github.io/penpax.ai)
<details> <details>
<summary>Contributing</summary> <summary>Contributing</summary>
**Worked examples** are the most trust-building contribution. Run `/graphify` on a real corpus, save output to `worked/{slug}/`, write an honest `review.md` evaluating what the graph got right and wrong, submit a PR. **Worked examples** are the most useful contribution. Run `/graphify` on a real corpus, save the output to `worked/{slug}/`, write an honest `review.md` covering what the graph got right and wrong, and open a PR.
**Extraction bugs** - open an issue with the input file, the cache entry (`graphify-out/cache/`), and what was missed or invented. **Extraction bugs** — open an issue with the input file, the cache entry (`graphify-out/cache/`), and what was missed or wrong.
See [ARCHITECTURE.md](ARCHITECTURE.md) for module responsibilities and how to add a language. See [ARCHITECTURE.md](ARCHITECTURE.md) for module responsibilities and how to add a language.
docs/how-it-works.md
+90
View File
@@ -0,0 +1,90 @@
# How graphify works
## The three passes
graphify processes your files in three passes:
**Pass 1 — Code structure (free, no API calls)**
Tree-sitter parses your code files and extracts classes, functions, imports, call graphs, and inline comments. This runs locally with no LLM involved. 25 languages supported. SQL files get special treatment: tables, views, foreign keys, and JOIN relationships are extracted deterministically.
**Pass 2 — Video and audio (local, no API calls)**
Video and audio files are transcribed with faster-whisper. To focus the transcript on your domain, the transcription prompt is seeded with your top god nodes (the most-connected concepts in your code graph so far). Transcripts are cached — re-runs skip already-processed files.
**Pass 3 — Docs, papers, images (Claude subagents, costs tokens)**
Claude runs in parallel over markdown, PDFs, images, and transcripts. Each subagent reads a batch of files and outputs a JSON fragment: nodes, edges, and any group relationships. The fragments are merged into a single graph.
---
## How community detection works
Communities are found using the [Leiden algorithm](https://www.nature.com/articles/s41598-019-41695-z) — a graph-clustering method that groups nodes by edge density. Nodes with many connections between them end up in the same community.
**No embeddings needed.** The semantic similarity edges that Claude extracts (`semantically_similar_to`) are already in the graph, so they influence community shape directly. The graph structure is the similarity signal — there's no separate embedding step or vector database.
---
## Confidence tagging
Every relationship is tagged with one of three labels:
| Tag | Meaning |
|-----|---------|
| `EXTRACTED` | Found directly in the source (e.g. a function call, an import) |
| `INFERRED` | A reasonable inference Claude made, with a `confidence_score` (0.0–1.0) |
| `AMBIGUOUS` | Uncertain — flagged in the report for manual review |
EXTRACTED edges always have confidence 1.0. INFERRED edges use a discrete rubric:
- **0.95** — near-certain (explicit cross-file reference, one plausible target)
- **0.85** — strong evidence (naming + context align)
- **0.75** — reasonable (contextual but not explicit)
- **0.65** — weak (naming similarity only)
- **0.55** — speculative
---
## Token benchmark
The first run extracts and builds the graph — this costs tokens. Every subsequent query reads the compact graph instead of raw files. That's where the savings compound.
On a mixed corpus (Karpathy repos + 5 papers + 4 images, 52 files): **71.5x fewer tokens per query** vs reading the raw files directly.
| Corpus | Files | Reduction |
|--------|-------|-----------|
| Karpathy repos + papers + images | 52 | **71.5x** |
| graphify source + Transformer paper | 4 | **5.4x** |
| httpx (synthetic Python library) | 6 | ~1x |
Token reduction scales with corpus size. Six files already fits in a context window — the graph value there is structural clarity, not compression. At 52 files the savings compound quickly.
Each `worked/` folder in the repo has the raw input files and actual output (`GRAPH_REPORT.md`, `graph.json`) so you can run it yourself and verify.
---
## Parallel extraction
Code files are extracted in parallel using `ProcessPoolExecutor` — bypasses Python's GIL for genuine multiprocessing. Doc/paper/image batches are dispatched as parallel Claude subagents. On a corpus of 84 code files, parallel AST extraction runs in about 1.66x less time than sequential.
---
## SHA256 cache
Every extracted file is fingerprinted by content hash. Re-runs skip unchanged files entirely — only new or modified files go through extraction again. The cache lives in `graphify-out/cache/`.
---
## The graph format
The output `graph.json` uses NetworkX's node-link format. Each node has:
- `id` — stable identifier
- `label` — human-readable name
- `file_type` — `code`, `document`, `paper`, `image`, `rationale`
- `source_file` — where it came from
Each edge has:
- `source`, `target` — node IDs
- `relation` — verb phrase (e.g. `calls`, `imports`, `implements`, `semantically_similar_to`)
- `confidence` — `EXTRACTED`, `INFERRED`, or `AMBIGUOUS`
- `confidence_score` — float (INFERRED only)
- `source_file` — where the relationship was found
Hyperedges (group relationships connecting 3+ nodes) live in `G.graph["hyperedges"]`.