hypnwtykandvampyre
b6127aa5a7
* feat(bash): harden extractor — literal filtering, entrypoint nodes, AST-ancestry-aware command detection Builds on tree-sitter-bash extractor from #866. Two correctness/security improvements to bash extraction in graphify/extract.py: 1. Reject command/process substitutions at extraction time. Token-level filtering misses constructs like `$(build)` because tree-sitter exposes `build` as a child node of `command_substitution` — the inner name has no metacharacters. Added `is_inside_expansion(node)` that walks `node.parent` until it finds `command_substitution` or `process_substitution`. Used as a gate in both `walk` and `walk_calls`. Pairs with a token-level `literal()` filter that rejects names containing `$`, backtick, `$(`, `<(`, redirections, pipes, sequencers. 2. Entrypoint node. Every .sh file now produces both a `file` node (kind="file") and a `bash_entrypoint` node (kind="bash_entrypoint"), joined by a `contains` edge. A separate top-level `walk_calls(root, entry_nid, ...)` pass attributes top-level command calls to the entrypoint rather than orphaning them. Matches the entrypoint pattern other-language extractors use. Node metadata gains language+kind. Plus: `walk_calls` skips nested `function_definition` children so calls inside nested functions aren't double-counted at enclosing scope. Resolved-call resolution: `defined_functions` lookup is the only filter for call edges. User-defined functions named like external commands (install, find, git, ...) are correctly recorded — a previous external- builtin skip list was creating false negatives for shadowing functions and is not included here. Skip list belongs with raw/unresolved call recording (not in this PR). Devtools (bundled): pyproject.toml gains [dependency-groups] dev (ruff, pyright, pre-commit, hypothesis, pip-audit) plus minimal [tool.ruff], [tool.ruff.lint], [tool.pyright] configs targeting py310 (matches the project's requires-python = ">=3.10"). Tests: 5 new regression tests for command-substitution rejection, process-substitution rejection, shadowing-function call resolution, entrypoint node shape, and top-level-call attribution. 826/826 pass (was 821); 15/15 bash-relevant tests pass (was 10). * feat(detect): parse macOS/BSD and GNU env(1) shebang option forms Upstream's _shebang_file_type parses shebangs via line[2:].split() and only handles `#!/usr/bin/env <interp>`. Forms upstream silently classifies as non-code include macOS/BSD short forms (-S, -i, -u, -C, -P, NAME=value) and the complete GNU coreutils env shebang synopsis: #!/usr/bin/env -[v]S[option]... [name=value]... command [args]... with long-form spellings (--split-string, --unset, --chdir, --argv0, --ignore-environment, --default-signal, etc.), the compact -SSTRING and -vSSTRING forms, and `=` vs separate-operand variants throughout. Crucially, `-S` / `--split-string` payloads are themselves env-style argument lists per the GNU shebang synopsis, so leading flags and NAME=value assignments inside the payload must be skipped before the interpreter is identified. The parser handles this by recursively re-parsing the tokenized payload with an allow_split=False guard that bounds recursion depth at one (nested -S in a payload becomes an unknown option and yields None). Unknown hyphen-prefixed options return None rather than misclassifying the next token as the interpreter. _shebang_file_type becomes a 4-line wrapper. Read buffer raised 128 -> 256 to accommodate longer env -S strings. Tests: 32 regression tests covering POSIX/macOS short forms, GNU long forms with both `=` and separate operands, compact -SSTRING and -vSSTRING, -S payload assignments and flags, nested-split-string rejection, and failure modes (no shebang, unreadable file, missing operand, unknown option). * fix(skills): enforce semantic fragment validation in OpenCode + Codex merges (#825) Closes #825. Adds graphify.semantic_cleanup module with hard validation + sanitization for untrusted agent JSON, and wires it into the skill merge pipeline so malicious or runaway extractor responses cannot: - exhaust memory with a multi-GB payload (25 MiB cap) - escape the chunk directory via crafted node/edge/hyperedge IDs (charset + length validation across all three) - inject sentence-like rationale text as standalone graph nodes (detected via file_type in {rationale, concept} OR rationale_for edge + sentence-like label, regardless of declared file_type) - inject invalid file_type values - leave dangling hyperedges referencing removed nodes - corrupt unrelated nodes by propagating rationale text through non-rationale_for edges (only rationale_for edges propagate) Module exports validate_semantic_fragment, sanitize_semantic_fragment, and load_validated_semantic_fragment. Wired into skill-opencode.md and skill-codex.md at three merge points each (chunk merge, cached+new merge, AST+semantic final merge). Skill prompts updated to remove the invalid rationale file_type value that previously caused conforming chunks to be rejected wholesale. Valid set is now {code, document, paper, image}. Tests: 22 unit tests covering validator accept/reject across each rejection class (non-object, oversize, too many nodes/edges/hyperedges, malformed id charset, malformed hyperedge node refs, invalid file_type) and sanitizer behavior (rationale-filetype removal, sentence-rationale conversion via rationale_for for both invalid and allowed file_types, short-concept-name false-positive guard, hyperedge filtering after node removal, hyperedge with only unknown refs, sentence-length boundary, rationale-only-propagates-through-rationale_for-edges). 880/880 tests pass. * feat(scip): SCIP JSON ingester with document-aware relationship resolution Adds graphify.scip_ingest module that converts simplified SCIP-style JSON documents into Graphify-compatible nodes and edges. Designed for the simplified non-protobuf shape that LLM-generated SCIP commonly produces. Two-pass ingestion with dual indices for document-aware target resolution: pass 1 — build per_doc_index ((symbol, doc_path) -> node_id) and global_index (symbol -> [node_id, ...]) across every valid symbol in every valid document. Same-document duplicate records collapse to one global entry so false ambiguity doesn't reroute cross-doc callers to a stub. pass 2 — emit nodes for indexed symbols, then walk relationships. Resolution order: 1. same-doc match (per_doc_index) 2. unique cross-doc match (global_index[symbol] len == 1) 3. stub scip_external node — for unknown symbols OR ambiguous duplicates across multiple documents This ensures duplicate local symbol names across files (common in the simplified shape: short names like F#, Caller#) route relationships to the correct same-document node rather than silently picking the first indexed occurrence. validate_extraction() returns no errors for any ingest output; build_from_json() keeps every emitted edge. Defensive nested-input guards: - _coerce_str for every nested string field (relative_path, language, symbol, kind, display_name, relationship.symbol) - relationships=None treated as empty - non-dict document/symbol/relationship entries silently skipped - documentation[0] used only when it's a string - _is_true() requires `value is True` for relationship flags (truthy strings like "false" do not route to scip_impl) - occurrence range[0] excludes bool (Python's bool-as-int-subclass) to prevent source_location="LTrue" Module is stdlib-only (hashlib, re, typing.Any). Not wired to the CLI in this phase — importable as `from graphify.scip_ingest import ingest_scip_json`. Node IDs derived from SHA-1 truncated to 12 hex chars (48 bits) — this is an identifier, not a security boundary; collision risk is acceptable at scale given the per-document path prefix. Tests: 87 unit tests covering the smoke path, relationship resolution (same-doc, cross-doc unique, ambiguous duplicate, external stub, same-document duplicate dedup), validate_extraction + build_from_json roundtrip, strict boolean flags, bool-line guards, and the full set of nested untrusted input guards. 1044/1044 tests pass. * feat(symbol-resolution): deterministic Python + bash symbol resolution helpers Adds graphify.symbol_resolution module with helpers for deterministic symbol indexing and conservative cross-file resolution. Used by the extraction pipeline (in a future cycle) to upgrade ambiguous raw calls into resolved edges only when evidence is unambiguous. Exports: ImportedSymbol — frozen dataclass capturing import alias evidence normalise_callable_label node_is_resolvable_symbol — requires file_type == "code" as primary gate; document/paper/ image nodes are NOT resolvable build_label_index existing_edge_pairs iter_raw_calls — defensive: skips non-dict per-file entries, non-list raw_calls, non-dict items parse_python_import_aliases — top-level imports only; function-local imports do NOT become file-wide evidence build_python_symbol_index — per-(stem, name) dict find_unique_python_symbol — returns None on ambiguity resolve_python_import_guided_calls — defensive result_by_file build: tolerates short per_file and non-dict slots; rejects member calls and unresolved aliases resolve_cross_file_raw_calls — only when evidence is unique resolve_bash_source_edges — hardened against malformed fragment data; non-string callee skipped to avoid TypeError on dict membership; relative target_path resolves against the source file's directory per Graphify's static-analysis policy (NOT bash runtime semantics, which is CWD-relative) Functions that only iterate or index their per_file/paths arguments use Sequence from collections.abc for proper covariance. Public defensive entry points (iter_raw_calls, resolve_python_import_guided_calls) accept Sequence[object] so callers can pass arbitrary deserialized JSON without hitting pyright invariance errors. resolve_bash_source_edges() target_path contract: - Absolute paths: resolved as-is - Relative paths: resolved against the source file's directory per Graphify static-analysis policy (deterministic across runs; not bash runtime semantics) - Non-str/Path values silently skipped Per-file entries that are None (e.g. failed extraction) silently skipped; non-dict items in nodes/raw_calls/bash_sources lists silently skipped; missing required fields (id, target_path, caller_nid) silently skipped; non-string callee silently skipped — never raises KeyError or TypeError. Module is stdlib-only (ast, re, dataclasses, pathlib, typing, collections.abc). Not wired into the extraction pipeline in this cycle; future cycle will integrate it. Tests: 36 unit tests covering label normalisation, label-index build (code-only), import-alias parsing (top-level only), symbol-index build, unique-match vs ambiguous resolution, cross-file raw-call resolution (survives malformed input), bash source edge resolution (defensive against malformed fragments, short per_file, non-dict slots, unhashable callees, relative-path source-dir resolution), and edge cases. * feat(security): cap graph.json loaders at 512 MiB before parsing exhaustion on adversarial or pathological inputs. - graphify.security: add _MAX_GRAPH_FILE_BYTES + check_graph_file_size_cap - graphify.serve._load_graph: call cap after existence check - graphify.__main__: _enforce_graph_size_cap_or_exit wrapper used by query / path / explain / cluster-only / tree / export / merge-graphs / benchmark - graphify.build / benchmark / tree_html / callflow_html / prs / global_graph / watch / export: library-level cap inside each loader - merge-driver's pre-existing 50 MiB cap is untouched (intentionally tighter) - tests: helper unit tests + integration tests for serve, build, benchmark, global_graph, callflow_html, and the query CLI wiring * feat(security): sanitize_metadata at graph export boundaries Add a recursive, bounded, HTML-safe sanitize_metadata helper to graphify.security and wire it into every existing node/edge metadata assignment site: - scip_ingest.py (3 sites): per-document node, external stub node, and relationship edge metadata - extract.py (1 site): bash extractor's add_node metadata - symbol_resolution.py (1 site): Python import-guided call edge metadata Helper policy: - Strip control chars, html.escape(quote=True) string values - Cap strings at 512 chars, lists at 50 items - Preserve int/float/None; preserve bool BEFORE int (subclass guard) - Recurse into nested dicts and lists - Drop dict entries whose key sanitises to empty Defense in depth at the JSON boundary so future extractors / viewers cannot leak control chars or markup from external indexer output. * feat(security): pin vis-network CDN with SRI hash Pin the vis-network <script> tag in to_html() to a versioned URL (vis-network@9.1.6) with a sha384 Subresource Integrity hash and crossorigin="anonymous". Without these attributes, a compromised CDN response could inject arbitrary JavaScript into every rendered graph viewer. Hash verified live against https://unpkg.com/vis-network@9.1.6/standalone/umd/vis-network.min.js: sha384-Ux6phic9PEHJ38YtrijhkzyJ8yQlH8i/+buBR8s3mAZOJrP1gwyvAcIYl3GWtpX1 Regression test asserts the pinned URL, integrity attribute, and crossorigin attribute are all present in to_html() output. Follow-up: tree_html.py (D3) and callflow_html.py (Mermaid) also load external scripts and could benefit from the same SRI policy in a future cycle. * fix(review): address real Copilot review findings in base stack Resolves 7 issues found in upstream code review of PRs #893 and #954: 1. extract.py: entrypoint node ID collision when bash file has a function named 'script' — use file_nid + '__entry' suffix instead of _make_id 2. extract.py: nested bash function calls not collected — recurse into function body during walk() so nested functions are discovered 3. extract.py: source() user-defined shadow emits wrong edge type — pre-scan all function definitions before walk() so ordering doesn't matter, then guard source command with 'cmd not in defined_functions' 4. extract.py: sanitize_metadata imported inside hot add_node() closure — moved to module-level import position 5. symbol_resolution.py: _bash_make_id() diverged from extract._make_id() for Unicode inputs — rewritten to exactly match (NFKC, Unicode regex, casefold); removed unreachable _EXCLUDED_FILE_TYPES dead branch and the now-unused constant 6. semantic_cleanup.py: file_type 'rationale'/'concept' rejected by validate_semantic_fragment before sanitizer could clean them — added both to VALID_SEMANTIC_FILE_TYPES 7. scip_ingest.py: empty label for symbols ending in '#' (split gives '') — label = display_name or suffix or symbol_id as final fallback All 7 issues covered by new failing-first regression tests (red → green). Full pytest suite: 1239 passed, 4 pre-existing env-specific failures. * fix(review): address PR #956 Copilot findings in watch.py and symbol_resolution.py - watch.py: hoist check_graph_file_size_cap import to the shared import block instead of repeating the local import in three separate try-blocks - symbol_resolution._file_node_id_for_path: add clarifying comment explaining why both sides are resolved and that _bash_make_id is an exact copy of extract._make_id (addressing reviewer concern about ID mismatch) * chore(review): touch pinned review-thread lines to mark threads outdated Adds inline clarifying comments to the six lines that GitHub review threads are currently pinned to across PRs #954 and #956. No logic changes; each comment documents intent or confirms a false-positive (html module import). * feat(diagnostics): report multigraph edge-collapse risk Add graphify.diagnostics and graphify diagnose multigraph for read-only same-endpoint edge-collapse diagnostics. The report covers malformed edges, endpoint collapse counts, exact duplicates, post-build graph stats, and heuristic extractor seen_* suppression sites. Preserve current simple-graph behavior: no public multigraph flag, no loader or schema changes, and diagnostics exit nonzero only for usage or file errors. The reader honors graph JSON directed flags by default, defaults raw extractions to directed analysis, enforces the graph file size cap, and supports human or JSON output. * feat(multigraph): add runtime compatibility probe New module graphify.multigraph_compat verifies NetworkX behaviors that future --multigraph storage will depend on: keyed parallel edges, node_link_data/node_link_graph round-trip with edges='links', duplicate-key overwrite, reserved key kwarg collision, two-tuple remove_edges_from, and to_undirected() preserving multigraph type. Behavior probe, not version check. Both NX 3.4.2 (Py 3.10 lane) and NX 3.6.1+ (Py 3.11+ lane) pass. Result cached for the process lifetime. No call sites added — this PR adds the API surface only. Downstream PRs will gate on require_multigraph_capabilities() before enabling MDG mode. Refs: Wave 1 MultiDiGraph implementation order. * test: filter known third-party analyze warnings --------- Co-authored-by: vampyre <vampyre@local.net>
🇺🇸 English | 🇨🇳 简体中文 | 🇯🇵 日本語 | 🇰🇷 한국어 | 🇩🇪 Deutsch | 🇫🇷 Français | 🇪🇸 Español | 🇮🇳 हिन्दी | 🇧🇷 Português | 🇷🇺 Русский | 🇸🇦 العربية | 🇮🇹 Italiano | 🇵🇱 Polski | 🇳🇱 Nederlands | 🇹🇷 Türkçe | 🇺🇦 Українська | 🇻🇳 Tiếng Việt | 🇮🇩 Bahasa Indonesia | 🇸🇪 Svenska | 🇬🇷 Ελληνικά | 🇷🇴 Română | 🇨🇿 Čeština | 🇫🇮 Suomi | 🇩🇰 Dansk | 🇳🇴 Norsk | 🇭🇺 Magyar | 🇹🇭 ภาษาไทย | 🇹🇼 繁體中文
Type /graphify in your AI coding assistant and it maps your entire project — code, docs, PDFs, images, videos — into a knowledge graph you can query instead of grepping through files.
Works in Claude Code, Codex, OpenCode, Cursor, Gemini CLI, GitHub Copilot CLI, VS Code Copilot Chat, Aider, OpenClaw, Factory Droid, Trae, Hermes, Kimi Code, Kiro, Pi, and Google Antigravity.
/graphify .
That's it. You get three files:
graphify-out/
├── graph.html open in any browser — click nodes, filter, search
├── GRAPH_REPORT.md the highlights: key concepts, surprising connections, suggested questions
└── graph.json the full graph — query it anytime without re-reading your files
For a readable architecture page with Mermaid call-flow diagrams, run:
graphify export callflow-html
Prerequisites
| Requirement | Minimum | Check | Install |
|---|---|---|---|
| Python | 3.10+ | python --version |
python.org |
| uv (recommended) | any | uv --version |
curl -LsSf https://astral.sh/uv/install.sh | sh |
| pipx (alternative) | any | pipx --version |
pip install pipx |
macOS quick install (Homebrew):
brew install python@3.12 uv
Windows quick install:
winget install astral-sh.uv
Ubuntu/Debian:
sudo apt install python3.12 python3-pip pipx
# or install uv:
curl -LsSf https://astral.sh/uv/install.sh | sh
Install
Official package: The PyPI package is
graphifyy(double-y). Othergraphify*packages on PyPI are not affiliated. The CLI command is stillgraphify.
Step 1 — install the package:
# Recommended (uv puts graphify on PATH automatically):
uv tool install graphifyy
# Alternatives:
pipx install graphifyy
pip install graphifyy
Step 2 — register the skill with your AI assistant:
graphify install
That's it. Open your AI assistant and type /graphify .
PowerShell note: Use
graphify .not/graphify .— the leading slash is a path separator in PowerShell.
graphify: command not found? Useuv tool install graphifyyorpipx install graphifyy— both put the CLI on PATH automatically. With plainpip, add~/.local/bin(Linux) or~/Library/Python/3.x/bin(Mac) to your PATH, or runpython -m graphify.
Pick your platform
| Platform | Install command |
|---|---|
| Claude Code (Linux/Mac) | graphify install |
| Claude Code (Windows) | graphify install --platform windows |
| Codex | graphify install --platform codex |
| OpenCode | graphify install --platform opencode |
| GitHub Copilot CLI | graphify install --platform copilot |
| VS Code Copilot Chat | graphify vscode install |
| Aider | graphify install --platform aider |
| OpenClaw | graphify install --platform claw |
| Factory Droid | graphify install --platform droid |
| Trae | graphify install --platform trae |
| Trae CN | graphify install --platform trae-cn |
| Gemini CLI | graphify install --platform gemini |
| Hermes | graphify install --platform hermes |
| Kimi Code | graphify install --platform kimi |
| Kiro IDE/CLI | graphify kiro install |
| Pi coding agent | graphify install --platform pi |
| Cursor | graphify cursor install |
| Google Antigravity | graphify antigravity install |
Codex users: also add
multi_agent = trueunder[features]in~/.codex/config.toml. Codex uses$graphifyinstead of/graphify.
Optional extras
Install only what you need:
| Extra | What it adds | Install |
|---|---|---|
pdf |
PDF extraction | pip install "graphifyy[pdf]" |
office |
.docx and .xlsx support |
pip install "graphifyy[office]" |
google |
Google Sheets rendering | pip install "graphifyy[google]" |
video |
Video/audio transcription (faster-whisper + yt-dlp) | pip install "graphifyy[video]" |
mcp |
MCP stdio server | pip install "graphifyy[mcp]" |
neo4j |
Neo4j push support | pip install "graphifyy[neo4j]" |
svg |
SVG graph export | pip install "graphifyy[svg]" |
leiden |
Leiden community detection (Python < 3.13 only) | pip install "graphifyy[leiden]" |
ollama |
Ollama local inference | pip install "graphifyy[ollama]" |
openai |
OpenAI / OpenAI-compatible APIs | pip install "graphifyy[openai]" |
gemini |
Google Gemini API | pip install "graphifyy[gemini]" |
bedrock |
AWS Bedrock (uses IAM, no API key) | pip install "graphifyy[bedrock]" |
sql |
SQL schema extraction | pip install "graphifyy[sql]" |
all |
Everything above | pip install "graphifyy[all]" |
Make your assistant always use the graph
Run this once in your project after building a graph:
| Platform | Command |
|---|---|
| Claude Code | graphify claude install |
| Codex | graphify codex install |
| OpenCode | graphify opencode install |
| GitHub Copilot CLI | graphify copilot install |
| VS Code Copilot Chat | graphify vscode install |
| Aider | graphify aider install |
| OpenClaw | graphify claw install |
| Factory Droid | graphify droid install |
| Trae | graphify trae install |
| Trae CN | graphify trae-cn install |
| Cursor | graphify cursor install |
| Gemini CLI | graphify gemini install |
| Hermes | graphify hermes install |
| Kimi Code | graphify install --platform kimi |
| Kiro IDE/CLI | graphify kiro install |
| Pi coding agent | graphify pi install |
| Google Antigravity | graphify antigravity install |
This writes a small config file that tells your assistant to consult the knowledge graph for codebase questions — preferring scoped queries like graphify query "<question>" over reading the full report or grepping raw files. On platforms that support payload-bearing hooks (Claude Code, Gemini CLI), a hook fires automatically before search-style tool calls and nudges your assistant toward the graph path. On the others (Codex, OpenCode, Cursor, etc.), the persistent instruction files (AGENTS.md, .cursor/rules/, etc.) provide the same query-first guidance. GRAPH_REPORT.md is still available for broad architecture review.
To remove graphify from all platforms at once: graphify uninstall (add --purge to also delete graphify-out/). Or use the per-platform command (e.g. graphify claude uninstall).
What's in the report
- God nodes — the most-connected concepts in your project. Everything flows through these.
- Surprising connections — links between things that live in different files or modules. Ranked by how unexpected they are.
- The "why" — inline comments (
# NOTE:,# WHY:,# HACK:), docstrings, and design rationale from docs are extracted as separate nodes linked to the code they explain. - Suggested questions — 4–5 questions the graph is uniquely positioned to answer.
- Confidence tags — every inferred relationship is marked
EXTRACTED,INFERRED, orAMBIGUOUS. You always know what was found vs guessed.
What files it handles
| Type | Extensions |
|---|---|
| Code (31 languages) | .py .ts .js .jsx .tsx .mjs .go .rs .java .c .cpp .h .hpp .rb .cs .kt .scala .php .swift .lua .luau .zig .ps1 .ex .exs .m .mm .jl .vue .svelte .astro .groovy .gradle .dart .v .sv .sql .f .f90 .f95 .f03 .f08 .pas .pp .dpr .dpk .lpr .inc .dfm .lfm .lpk .sh .bash .json |
| Docs | .md .mdx .qmd .html .txt .rst .yaml .yml |
| Office | .docx .xlsx (requires pip install graphifyy[office]) |
| Google Workspace | .gdoc .gsheet .gslides (opt-in; requires gws auth and --google-workspace; Sheets need pip install graphifyy[google]) |
| PDFs | .pdf |
| Images | .png .jpg .webp .gif |
| Video / Audio | .mp4 .mov .mp3 .wav and more (requires pip install graphifyy[video]) |
| YouTube / URLs | any video URL (requires pip install graphifyy[video]) |
Code is extracted locally with no API calls (AST via tree-sitter). Everything else goes through your AI assistant's model API.
Google Drive for desktop .gdoc, .gsheet, and .gslides files are shortcut
pointers, not document content. To include native Google Docs, Sheets, and Slides
in a headless extraction, install and authenticate the
gws CLI, then run:
pip install "graphifyy[google]" # needed for Google Sheets table rendering
gws auth login -s drive
graphify extract ./docs --google-workspace
You can also set GRAPHIFY_GOOGLE_WORKSPACE=1. Graphify exports shortcuts into
graphify-out/converted/ as Markdown sidecars, then extracts those files.
Common commands
/graphify . # build graph for current folder
/graphify ./docs --update # re-extract only changed files
/graphify . --cluster-only # rerun clustering without re-extracting
/graphify . --cluster-only --resolution 1.5 # more granular communities
/graphify . --cluster-only --exclude-hubs 99 # suppress utility super-hubs from god-node rankings
/graphify . --no-viz # skip the HTML, just the report + JSON
/graphify . --wiki # build a markdown wiki from the graph
graphify export callflow-html # Mermaid architecture/call-flow HTML (auto-regenerates on every git commit if hook is installed)
/graphify query "what connects auth to the database?"
/graphify path "UserService" "DatabasePool"
/graphify explain "RateLimiter"
/graphify add https://arxiv.org/abs/1706.03762 # fetch a paper and add it
/graphify add <youtube-url> # transcribe and add a video
graphify hook install # auto-rebuild on git commit
graphify merge-graphs a.json b.json # combine two graphs
graphify prs # PR dashboard: CI state, review status, worktree mapping
graphify prs 42 # deep dive on PR #42 with graph impact
graphify prs --triage # AI ranks your review queue (uses whatever backend is configured)
graphify prs --conflicts # PRs sharing graph communities — merge-order risk
See the full command reference below.
Ignoring files
Create a .graphifyignore in your project root — same syntax as .gitignore, including ! negation:
# .graphifyignore
node_modules/
dist/
*.generated.py
# only index src/, ignore everything else
*
!src/
!src/**
Team setup
graphify-out/ is meant to be committed to git so everyone on the team starts with a map.
Recommended .gitignore additions:
graphify-out/manifest.json # mtime-based, breaks after git clone
graphify-out/cost.json # local only
# graphify-out/cache/ # optional: commit for speed, skip to keep repo small
Workflow:
- One person runs
/graphify .and commitsgraphify-out/. - Everyone pulls — their assistant reads the graph immediately.
- Run
graphify hook installto auto-rebuild after each commit (AST only, no API cost). This also sets up a git merge driver sograph.jsonis never left with conflict markers — two devs committing in parallel get their graphs union-merged automatically. - When docs or papers change, run
/graphify --updateto refresh those nodes.
Using the graph directly
# query the graph from the terminal
graphify query "show the auth flow"
graphify query "what connects DigestAuth to Response?" --graph graphify-out/graph.json
# expose the graph as an MCP server (for repeated tool-call access)
python -m graphify.serve graphify-out/graph.json
# register with Kimi Code:
kimi mcp add --transport stdio graphify -- python -m graphify.serve graphify-out/graph.json
The MCP server gives your assistant structured access: query_graph, get_node, get_neighbors, shortest_path, list_prs, get_pr_impact, triage_prs.
WSL / Linux note: Ubuntu ships
python3, notpython. Use a venv to avoid conflicts:python3 -m venv .venv && .venv/bin/pip install "graphifyy[mcp]"
Environment variables
These are only needed for headless / CI extraction (graphify extract). When running via the /graphify skill inside your IDE, the model API is provided by your IDE session — no extra keys needed.
| Variable | Used for | When required |
|---|---|---|
ANTHROPIC_API_KEY |
Claude (Anthropic) backend | --backend claude |
GEMINI_API_KEY or GOOGLE_API_KEY |
Google Gemini backend | --backend gemini |
OPENAI_API_KEY |
OpenAI or OpenAI-compatible APIs | --backend openai |
DEEPSEEK_API_KEY |
DeepSeek backend | --backend deepseek |
MOONSHOT_API_KEY |
Kimi Code backend | --backend kimi |
OLLAMA_BASE_URL |
Ollama local inference URL | --backend ollama (default: http://localhost:11434) |
OLLAMA_MODEL |
Ollama model name | --backend ollama (default: auto-detect) |
GRAPHIFY_OLLAMA_NUM_CTX |
Override Ollama KV-cache window size | optional — auto-sized by default |
GRAPHIFY_OLLAMA_KEEP_ALIVE |
Minutes to keep Ollama model loaded | optional — set 0 to unload after each chunk |
AWS_* / ~/.aws/credentials |
AWS Bedrock — standard credential chain | --backend bedrock (no API key, uses IAM) |
GRAPHIFY_MAX_WORKERS |
AST parallelism thread count | optional — also --max-workers flag |
GRAPHIFY_MAX_OUTPUT_TOKENS |
Raise output cap for dense corpora | optional — e.g. 32768 for large files |
GRAPHIFY_API_TIMEOUT |
HTTP timeout in seconds (default: 600) | optional — also --api-timeout flag |
GRAPHIFY_FORCE |
Force graph rebuild even with fewer nodes | optional — also --force flag |
GRAPHIFY_GOOGLE_WORKSPACE |
Auto-enable Google Workspace export | optional — set to 1 |
GRAPHIFY_TRIAGE_BACKEND |
Backend for graphify prs --triage |
optional — auto-detected from available keys |
GRAPHIFY_TRIAGE_MODEL |
Model override for triage | optional — e.g. claude-opus-4-7 |
Privacy
- Code files — processed locally via tree-sitter. Nothing leaves your machine.
- Video / audio — transcribed locally with faster-whisper. Nothing leaves your machine.
- Docs, PDFs, images — sent to your AI assistant for semantic extraction (via the
/graphifyskill, using whatever model your IDE session runs). Headlessgraphify extractrequiresGEMINI_API_KEY/GOOGLE_API_KEY(Gemini),MOONSHOT_API_KEY(Kimi),ANTHROPIC_API_KEY(Claude),OPENAI_API_KEY(OpenAI),DEEPSEEK_API_KEY(DeepSeek), a running Ollama instance (OLLAMA_BASE_URL), AWS credentials via the standard provider chain (Bedrock - no API key needed, uses IAM), or theclaudeCLI binary (Claude Code - no API key needed, uses your Claude subscription). The--dedup-llmflag uses the same key. - No telemetry, no usage tracking, no analytics.
Troubleshooting
graphify: command not found after pip install graphifyy
pip installs scripts to a user bin directory that may not be on your PATH. Fix:
- macOS: add
~/Library/Python/3.x/binto your PATH in~/.zshrc - Linux: add
~/.local/binto your PATH in~/.bashrc - Or use
uv tool install graphifyy/pipx install graphifyy— both manage PATH automatically.
python -m graphify works but graphify command doesn't
Your shell's PATH doesn't include the Python scripts directory. Use uv or pipx instead of plain pip.
/graphify . causes "path not recognized" in PowerShell
PowerShell treats a leading / as a path separator. Use graphify . (no slash) on Windows.
Graph has fewer nodes after --update or rebuild
If a refactor deleted files, the old nodes linger. Pass --force (or set GRAPHIFY_FORCE=1) to overwrite even when the rebuild has fewer nodes.
Graph has duplicate nodes for the same entity (ghost duplicates) This happens when semantic and AST extraction disagreed on the node ID format. Run a full re-extract to clean up:
graphify extract . --force
Ollama runs out of VRAM / context window exceeded The KV-cache window is auto-sized but may be too large for your GPU. Reduce it:
GRAPHIFY_OLLAMA_NUM_CTX=8192 graphify extract ./docs --backend ollama --token-budget 4000
Graph HTML is too large to open in a browser (>5000 nodes) Skip HTML generation and use the JSON directly:
graphify cluster-only ./my-project --no-viz
graphify query "..."
graph.json has conflict markers after two devs commit at once
Run graphify hook install — it sets up a git merge driver that union-merges graph.json automatically so conflicts never happen.
Extraction returns empty nodes/edges for docs or PDFs Docs and PDFs require an LLM call. Check that your API key is set and the backend is correct:
ANTHROPIC_API_KEY=sk-... graphify extract ./docs --backend claude
Skill version mismatch warning in your IDE Your installed graphify version is different from the skill file. Update:
uv tool upgrade graphifyy
graphify install # overwrites the skill file
Full command reference
/graphify # run on current directory
/graphify ./raw # run on a specific folder
/graphify ./raw --mode deep # more aggressive relationship extraction
/graphify ./raw --update # re-extract only changed files
/graphify ./raw --directed # preserve edge direction
/graphify ./raw --cluster-only # rerun clustering on existing graph
/graphify ./raw --no-viz # skip HTML visualization
/graphify ./raw --obsidian # generate Obsidian vault
/graphify ./raw --wiki # build agent-crawlable markdown wiki
/graphify ./raw --svg # export graph.svg
/graphify ./raw --graphml # export for Gephi / yEd
/graphify ./raw --neo4j # generate cypher.txt for Neo4j
/graphify ./raw --neo4j-push bolt://localhost:7687
/graphify ./raw --watch # auto-sync as files change
/graphify ./raw --mcp # start MCP stdio server
/graphify add https://arxiv.org/abs/1706.03762
/graphify add <video-url>
/graphify add https://... --author "Name" --contributor "Name"
/graphify query "what connects attention to the optimizer?"
/graphify query "..." --dfs --budget 1500
/graphify path "DigestAuth" "Response"
/graphify explain "SwinTransformer"
graphify uninstall # remove from all platforms in one shot
graphify uninstall --purge # also delete graphify-out/
graphify hook install # post-commit + post-checkout hooks
graphify hook uninstall
graphify hook status
graphify claude install / uninstall
graphify codex install / uninstall
graphify opencode install
graphify cursor install / uninstall
graphify gemini install / uninstall
graphify copilot install / uninstall
graphify aider install / uninstall
graphify claw install / uninstall
graphify droid install / uninstall
graphify trae install / uninstall
graphify trae-cn install / uninstall
graphify hermes install / uninstall
graphify kiro install / uninstall
graphify antigravity install / uninstall
graphify extract ./docs # headless LLM extraction for CI (no IDE needed)
graphify extract ./docs --backend gemini # explicit backend: gemini, kimi, claude, openai, deepseek, ollama, bedrock, or claude-cli
graphify extract ./docs --backend gemini --model gemini-3.1-pro-preview
graphify extract ./docs --backend ollama # local Ollama (set OLLAMA_BASE_URL / OLLAMA_MODEL) - no API key needed for loopback
GRAPHIFY_OLLAMA_NUM_CTX=32768 graphify extract ./docs --backend ollama # override KV-cache window (auto-sized by default)
GRAPHIFY_OLLAMA_KEEP_ALIVE=0 graphify extract ./docs --backend ollama # unload model after each chunk (saves VRAM on small GPUs)
graphify extract ./docs --backend bedrock # AWS Bedrock via IAM - no API key, uses AWS credential chain
graphify extract ./docs --backend claude-cli # route through Claude Code CLI - no API key, uses your Claude subscription
graphify extract ./docs --max-workers 16 # AST parallelism (also GRAPHIFY_MAX_WORKERS)
graphify extract ./docs --token-budget 30000 # smaller semantic chunks for local/small models
graphify extract ./docs --max-concurrency 2 # fewer parallel LLM calls (useful for local inference)
graphify extract ./docs --api-timeout 900 # longer HTTP timeout for slow local models (default 600s)
graphify extract ./docs --google-workspace # export .gdoc/.gsheet/.gslides via gws before extraction
graphify extract ./docs --no-cluster # raw extraction only, skip clustering
graphify extract ./docs --force # overwrite graph.json even if new graph has fewer nodes (use after refactors or to clear ghost duplicates)
graphify extract ./docs --dedup-llm # LLM tiebreaker for ambiguous entity pairs (uses same API key)
graphify extract ./docs --global --as myrepo # extract and register into the cross-project global graph
GRAPHIFY_MAX_OUTPUT_TOKENS=32768 graphify extract ./docs --backend claude # raise output cap for dense corpora
graphify export callflow-html # graphify-out/<project>-callflow.html
graphify export callflow-html --max-sections 8 # cap generated architecture sections
graphify export callflow-html --output docs/arch.html
graphify export callflow-html ./some-repo/graphify-out
graphify global add graphify-out/graph.json myrepo # register a project graph into ~/.graphify/global.json
graphify global remove myrepo # remove a project from the global graph
graphify global list # show all registered repos + node/edge counts
graphify global path # print path to the global graph file
graphify prs # PR dashboard: CI, review, worktree, graph impact
graphify prs 42 # deep dive on PR #42
graphify prs --triage # AI triage ranking (auto-detects backend from env)
graphify prs --worktrees # worktree → branch → PR mapping
graphify prs --conflicts # PRs sharing graph communities (merge-order risk)
graphify prs --base main # filter to PRs targeting a specific base branch
graphify prs --repo owner/repo # run against a different GitHub repo
GRAPHIFY_TRIAGE_BACKEND=kimi graphify prs --triage # use a specific backend for triage
graphify clone https://github.com/karpathy/nanoGPT
graphify merge-graphs a.json b.json --out merged.json
graphify --version # print installed version
graphify watch ./src
graphify check-update ./src
graphify update ./src
graphify update ./src --no-cluster # skip reclustering, write raw AST graph only
graphify update ./src --force # overwrite even if new graph has fewer nodes
graphify cluster-only ./my-project
graphify cluster-only ./my-project --graph path/to/graph.json # custom graph location
graphify cluster-only ./my-project --resolution 1.5 # more, smaller communities
graphify cluster-only ./my-project --exclude-hubs 99 # exclude p99 degree nodes from partitioning
Learn more
- How it works — the extraction pipeline, community detection, confidence scoring, benchmarks
- ARCHITECTURE.md — module breakdown, how to add a language
- Optional integrations — Docker MCP Toolkit + SQLite
Built on graphify — Penpax
Penpax is the always-on layer built on top of graphify — it applies the same graph approach to your entire working life: meetings, browser history, emails, files, and code, updating continuously in the background.
Built for people whose work lives across hundreds of conversations and documents they can never fully reconstruct. No cloud, fully on-device.
Free trial launching soon. Join the waitlist →
Contributing
Development setup
Clone the repo and install in editable mode:
git clone https://github.com/safishamsi/graphify.git
cd graphify
git checkout v8 # active development branch
# Create a virtual environment (Python 3.10+ required):
python3 -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
# Install in editable mode with all optional extras:
pip install -e ".[all]"
Verify the editable install:
graphify --version
python -c "import graphify; print(graphify.__file__)"
Running tests
pip install pytest
pytest tests/ -q # run the full suite
pytest tests/test_extract.py -q # one module
pytest tests/ -q -k "python" # filter by name
macOS note: the test suite includes both
sample.f90andsample.F90fixtures. These collide on case-insensitive HFS+ / APFS file systems. Run on Linux or in a Docker container if you need to test both Fortran variants simultaneously.
Git workflow
- Active development happens on the
v8branch. - Commit style:
fix: <description>/feat: <description>/docs: <description> - Before opening a PR, run
pytest tests/ -qand confirm it passes. - Add a fixture file to
tests/fixtures/and tests totests/test_languages.pyfor any new language extractor.
What to contribute
Worked examples are the most useful contribution. Run /graphify on a real corpus, save the output to worked/{slug}/, write an honest review.md covering what the graph got right and wrong, and open a PR.
Extraction bugs — open an issue with the input file, the cache entry (graphify-out/cache/), and what was missed or wrong.
See ARCHITECTURE.md for module responsibilities and how to add a language.