Merge #1366: prune deleted-only and root the full build so --update stops duplicating
After #1361 added root= to the --update build_merge call, build_merge's prune (which runs after the merge) began matching freshly re-extracted changed-file nodes and deleting them — a regression in 0.8.41 where --update on a changed file wiped its nodes. This drops `changed` from prune_sources (replace-on- re-extract from #1344 already reconciles changed files), passes root= to the full build's build_from_json so its node-key base matches the update side, and pins the extraction-spec source_file to the verbatim path so the two runs never drift. Re-blessed skillgen expected/ snapshots. Co-Authored-By: RelywOo <RelywOo@users.noreply.github.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Safi
RelywOo
Claude Opus 4.8
@@ -4,6 +4,7 @@ Full release notes with details on each version: [GitHub Releases](https://githu
|
||||
|
||||
## Unreleased
|
||||
|
||||
- Fix: `/graphify --update` no longer deletes a changed file's freshly re-extracted nodes. The `0.8.41` `root=` fix (#1361) made `build_merge`'s prune actually match relative `source_file` values — which then matched the just-re-extracted nodes of *changed* files (still listed in `prune_sources`) and removed them, so an `--update` on a changed file could wipe its nodes. The update runbook now prunes only genuinely **deleted** files; changed files are reconciled by `build_merge`'s replace-on-re-extract (#1344). The full build also now passes `root=` to `build_from_json`, and the extraction-spec `source_file` is pinned to the verbatim path, so the full build and incremental updates never drift on node-key base. (#1366; thanks @RelywOo)
|
||||
- Fix: Java `record` declarations are now modeled as first-class type nodes (they share `class_declaration`'s name/body/interfaces fields), and `new Foo(...)` constructor calls now produce a `calls` edge to the constructed type. Previously a record appeared only as its file node (degree 0) with no incoming edges, and body-level `new` usages were dropped because `object_creation_expression` wasn't a recognized call type and its callee lives in the `type` field rather than `name`. (#1373)
|
||||
- Security fix: `.graphifyignore` and `.gitignore` are now **merged** per directory instead of `.graphifyignore` silently replacing that directory's `.gitignore`. Previously, adding a `.graphifyignore` (e.g. to exclude media) disabled the dir's `.gitignore` entirely, so a file excluded only by `.gitignore` — including neutrally-named secrets like `prod-dump.sql` or `customer-data.json` that the sensitive-file heuristic doesn't catch — got indexed into the graph, whose artifacts embed file contents and are routinely committed. `.gitignore` is read first and `.graphifyignore` last, so `.graphifyignore` patterns (including `!` negations) still win on conflict; adding one can only ever exclude more, never re-include a `.gitignore`-excluded file. (#1363)
|
||||
|
||||
|
||||
@@ -382,7 +382,9 @@ from pathlib import Path
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= mirrors the --update runbook (#1361): relativize source_file to the same
|
||||
# base so the full build and incremental --update never drift apart on re-extract.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = cluster(G)
|
||||
cohesion = score_all(G, communities)
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
@@ -435,7 +437,8 @@ extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(en
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= as in Step 4 / the --update runbook (#1361) — same base for node-key parity.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
|
||||
@@ -385,7 +385,9 @@ from pathlib import Path
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= mirrors the --update runbook (#1361): relativize source_file to the same
|
||||
# base so the full build and incremental --update never drift apart on re-extract.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = cluster(G)
|
||||
cohesion = score_all(G, communities)
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
@@ -438,7 +440,8 @@ extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(en
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= as in Step 4 / the --update runbook (#1361) — same base for node-key parity.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
|
||||
@@ -382,7 +382,9 @@ from pathlib import Path
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= mirrors the --update runbook (#1361): relativize source_file to the same
|
||||
# base so the full build and incremental --update never drift apart on re-extract.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = cluster(G)
|
||||
cohesion = score_all(G, communities)
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
@@ -435,7 +437,8 @@ extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(en
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= as in Step 4 / the --update runbook (#1361) — same base for node-key parity.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
|
||||
@@ -385,7 +385,9 @@ from pathlib import Path
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= mirrors the --update runbook (#1361): relativize source_file to the same
|
||||
# base so the full build and incremental --update never drift apart on re-extract.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = cluster(G)
|
||||
cohesion = score_all(G, communities)
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
@@ -438,7 +440,8 @@ extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(en
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= as in Step 4 / the --update runbook (#1361) — same base for node-key parity.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
|
||||
@@ -382,7 +382,9 @@ from pathlib import Path
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= mirrors the --update runbook (#1361): relativize source_file to the same
|
||||
# base so the full build and incremental --update never drift apart on re-extract.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = cluster(G)
|
||||
cohesion = score_all(G, communities)
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
@@ -435,7 +437,8 @@ extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(en
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= as in Step 4 / the --update runbook (#1361) — same base for node-key parity.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
|
||||
@@ -385,7 +385,9 @@ from pathlib import Path
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= mirrors the --update runbook (#1361): relativize source_file to the same
|
||||
# base so the full build and incremental --update never drift apart on re-extract.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = cluster(G)
|
||||
cohesion = score_all(G, communities)
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
@@ -438,7 +440,8 @@ extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(en
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= as in Step 4 / the --update runbook (#1361) — same base for node-key parity.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
|
||||
@@ -385,7 +385,9 @@ from pathlib import Path
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= mirrors the --update runbook (#1361): relativize source_file to the same
|
||||
# base so the full build and incremental --update never drift apart on re-extract.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = cluster(G)
|
||||
cohesion = score_all(G, communities)
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
@@ -438,7 +440,8 @@ extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(en
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= as in Step 4 / the --update runbook (#1361) — same base for node-key parity.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
|
||||
@@ -377,7 +377,9 @@ from pathlib import Path
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= mirrors the --update runbook (#1361): relativize source_file to the same
|
||||
# base so the full build and incremental --update never drift apart on re-extract.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = cluster(G)
|
||||
cohesion = score_all(G, communities)
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
@@ -430,7 +432,8 @@ extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(en
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= as in Step 4 / the --update runbook (#1361) — same base for node-key parity.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
|
||||
@@ -385,7 +385,9 @@ from pathlib import Path
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= mirrors the --update runbook (#1361): relativize source_file to the same
|
||||
# base so the full build and incremental --update never drift apart on re-extract.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = cluster(G)
|
||||
cohesion = score_all(G, communities)
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
@@ -438,7 +440,8 @@ extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(en
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= as in Step 4 / the --update runbook (#1361) — same base for node-key parity.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
|
||||
@@ -383,7 +383,9 @@ from pathlib import Path
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= mirrors the --update runbook (#1361): relativize source_file to the same
|
||||
# base so the full build and incremental --update never drift apart on re-extract.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = cluster(G)
|
||||
cohesion = score_all(G, communities)
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
@@ -436,7 +438,8 @@ extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(en
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= as in Step 4 / the --update runbook (#1361) — same base for node-key parity.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
|
||||
@@ -381,7 +381,9 @@ from pathlib import Path
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= mirrors the --update runbook (#1361): relativize source_file to the same
|
||||
# base so the full build and incremental --update never drift apart on re-extract.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = cluster(G)
|
||||
cohesion = score_all(G, communities)
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
@@ -434,7 +436,8 @@ extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(en
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= as in Step 4 / the --update runbook (#1361) — same base for node-key parity.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
|
||||
@@ -407,7 +407,9 @@ from pathlib import Path
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= mirrors the --update runbook (#1361): relativize source_file to the same
|
||||
# base so the full build and incremental --update never drift apart on re-extract.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = cluster(G)
|
||||
cohesion = score_all(G, communities)
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
@@ -460,7 +462,8 @@ extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(en
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= as in Step 4 / the --update runbook (#1361) — same base for node-key parity.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
|
||||
+5
-2
@@ -385,7 +385,9 @@ from pathlib import Path
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= mirrors the --update runbook (#1361): relativize source_file to the same
|
||||
# base so the full build and incremental --update never drift apart on re-extract.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = cluster(G)
|
||||
cohesion = score_all(G, communities)
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
@@ -438,7 +440,8 @@ extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(en
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= as in Step 4 / the --update runbook (#1361) — same base for node-key parity.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
|
||||
@@ -61,7 +61,9 @@ confidence_score is REQUIRED on every edge - never omit it, never use 0.5 as a d
|
||||
Node ID format: lowercase, only `[a-z0-9_]`, no dots or slashes. Format: `{stem}_{entity}` where stem is `{parent_dir}_{filename_without_ext}` (the **immediate** parent directory name + the filename stem, both lowercased with non-alphanumeric chars replaced by `_`) and entity is the symbol name similarly normalized. Only one level of parent is used — not the full path. Examples: `src/auth/session.py` + `ValidateToken` → `auth_session_validatetoken`; `lib/utils/helpers.py` + `parse_url` → `utils_helpers_parse_url`; `tests/test_foo.py` + `_helper` → `tests_test_foo_helper`. Top-level files (no parent dir, e.g. `setup.py`) use just the filename stem: `setup_my_func`. This must match the ID the AST extractor generates — using just the filename (e.g., `session_validatetoken`) or the full path (e.g., `src_auth_session_validatetoken`) will create orphan ghost-duplicate nodes. If you are re-extracting a project that had ghost duplicates under the old format, the user should run `graphify extract --force` to rebuild cleanly. CRITICAL: never append chunk numbers, sequence numbers, or any suffix to an ID (no `_c1`, `_c2`, `_chunk2`, etc.). IDs must be deterministic from the label alone — the same entity must always produce the same ID regardless of which chunk processes it.
|
||||
|
||||
Generate the extraction JSON matching this schema exactly:
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0}
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"<FILE_LIST path verbatim>","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"<FILE_LIST path verbatim>","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"<FILE_LIST path verbatim>"}],"input_tokens":0,"output_tokens":0}
|
||||
|
||||
source_file RULE (every node, edge, and hyperedge): set source_file to the path of the originating file EXACTLY as it appears in FILE_LIST — verbatim and absolute. Do NOT shorten to a basename, do NOT re-relativize, do NOT strip any directory prefix, and do NOT change separators (the engine canonicalizes separators and relativizes against the build root downstream). Copy the FILE_LIST entry character-for-character. This keeps the full build and incremental --update on the same base, so build_merge's replace-on-re-extract matches the existing node instead of accumulating a duplicate.
|
||||
|
||||
Then write the JSON to disk using the Write tool at this exact absolute path (no relative paths — Write resolves relative paths against an undefined cwd and the file will be silently lost):
|
||||
CHUNK_PATH
|
||||
|
||||
@@ -93,11 +93,13 @@ from graphify.detect import save_manifest
|
||||
new_extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
incremental = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\"))
|
||||
deleted = list(incremental.get('deleted_files', []))
|
||||
# Also prune old nodes for re-extracted (changed) files before inserting fresh AST.
|
||||
# Without this, build_merge's dedup pass tries to reconcile old and new versions of
|
||||
# the same file's nodes and can collapse same-named symbols across files (#1178).
|
||||
changed = [f for files in incremental.get('new_files', {}).values() for f in files]
|
||||
prune = list(dict.fromkeys(deleted + changed)) or None
|
||||
# prune_sources is ONLY for genuinely DELETED files. Changed/re-extracted files are
|
||||
# handled by build_merge's replace-on-re-extract (#1344): every source_file in
|
||||
# new_chunks is dropped from the base before merge, so old/stale nodes don't survive.
|
||||
# Do NOT add `changed` here: with root= passed, prune_set relativizes to the same base
|
||||
# as the freshly merged nodes and would DELETE the re-extracted content (#1178 is moot
|
||||
# now that replace — not the dedup pass — reconciles changed files).
|
||||
prune = list(deleted) or None
|
||||
|
||||
# Use build_merge() — reads graph.json directly without NetworkX round-trip
|
||||
# so edge direction (calls, implements, imports) is always preserved (#801).
|
||||
|
||||
@@ -61,7 +61,9 @@ confidence_score is REQUIRED on every edge - never omit it, never use 0.5 as a d
|
||||
Node ID format: lowercase, only `[a-z0-9_]`, no dots or slashes. Format: `{stem}_{entity}` where stem is `{parent_dir}_{filename_without_ext}` (the **immediate** parent directory name + the filename stem, both lowercased with non-alphanumeric chars replaced by `_`) and entity is the symbol name similarly normalized. Only one level of parent is used — not the full path. Examples: `src/auth/session.py` + `ValidateToken` → `auth_session_validatetoken`; `lib/utils/helpers.py` + `parse_url` → `utils_helpers_parse_url`; `tests/test_foo.py` + `_helper` → `tests_test_foo_helper`. Top-level files (no parent dir, e.g. `setup.py`) use just the filename stem: `setup_my_func`. This must match the ID the AST extractor generates — using just the filename (e.g., `session_validatetoken`) or the full path (e.g., `src_auth_session_validatetoken`) will create orphan ghost-duplicate nodes. If you are re-extracting a project that had ghost duplicates under the old format, the user should run `graphify extract --force` to rebuild cleanly. CRITICAL: never append chunk numbers, sequence numbers, or any suffix to an ID (no `_c1`, `_c2`, `_chunk2`, etc.). IDs must be deterministic from the label alone — the same entity must always produce the same ID regardless of which chunk processes it.
|
||||
|
||||
Generate the extraction JSON matching this schema exactly:
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0}
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"<FILE_LIST path verbatim>","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"<FILE_LIST path verbatim>","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"<FILE_LIST path verbatim>"}],"input_tokens":0,"output_tokens":0}
|
||||
|
||||
source_file RULE (every node, edge, and hyperedge): set source_file to the path of the originating file EXACTLY as it appears in FILE_LIST — verbatim and absolute. Do NOT shorten to a basename, do NOT re-relativize, do NOT strip any directory prefix, and do NOT change separators (the engine canonicalizes separators and relativizes against the build root downstream). Copy the FILE_LIST entry character-for-character. This keeps the full build and incremental --update on the same base, so build_merge's replace-on-re-extract matches the existing node instead of accumulating a duplicate.
|
||||
|
||||
Then write the JSON to disk using the Write tool at this exact absolute path (no relative paths — Write resolves relative paths against an undefined cwd and the file will be silently lost):
|
||||
CHUNK_PATH
|
||||
|
||||
@@ -93,11 +93,13 @@ from graphify.detect import save_manifest
|
||||
new_extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
incremental = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\"))
|
||||
deleted = list(incremental.get('deleted_files', []))
|
||||
# Also prune old nodes for re-extracted (changed) files before inserting fresh AST.
|
||||
# Without this, build_merge's dedup pass tries to reconcile old and new versions of
|
||||
# the same file's nodes and can collapse same-named symbols across files (#1178).
|
||||
changed = [f for files in incremental.get('new_files', {}).values() for f in files]
|
||||
prune = list(dict.fromkeys(deleted + changed)) or None
|
||||
# prune_sources is ONLY for genuinely DELETED files. Changed/re-extracted files are
|
||||
# handled by build_merge's replace-on-re-extract (#1344): every source_file in
|
||||
# new_chunks is dropped from the base before merge, so old/stale nodes don't survive.
|
||||
# Do NOT add `changed` here: with root= passed, prune_set relativizes to the same base
|
||||
# as the freshly merged nodes and would DELETE the re-extracted content (#1178 is moot
|
||||
# now that replace — not the dedup pass — reconciles changed files).
|
||||
prune = list(deleted) or None
|
||||
|
||||
# Use build_merge() — reads graph.json directly without NetworkX round-trip
|
||||
# so edge direction (calls, implements, imports) is always preserved (#801).
|
||||
|
||||
@@ -25,5 +25,7 @@ Rules:
|
||||
Node ID format: lowercase, only `[a-z0-9_]`, no dots or slashes. Format `{stem}_{entity}` where stem is `{parent_dir}_{filename_without_ext}` (the immediate parent directory + the filename stem, both lowercased with non-alphanumeric chars replaced by `_`) and entity is the symbol name similarly normalized. Only one level of parent. `src/auth/session.py` + `ValidateToken` → `auth_session_validatetoken`. Top-level files use just the filename stem. This must match the AST extractor's ID. Never append chunk or sequence suffixes — IDs must be deterministic from the label alone.
|
||||
|
||||
Output exactly this JSON (no other text):
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0}
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"<FILE_LIST path verbatim>","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"<FILE_LIST path verbatim>","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"<FILE_LIST path verbatim>"}],"input_tokens":0,"output_tokens":0}
|
||||
|
||||
source_file RULE: set source_file to the FILE_LIST path for that file VERBATIM (absolute, no shortening to basename, no re-relativizing, no separator change). Keeps full build and --update on one base so build_merge's replace matches instead of duplicating.
|
||||
```
|
||||
|
||||
@@ -93,11 +93,13 @@ from graphify.detect import save_manifest
|
||||
new_extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
incremental = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\"))
|
||||
deleted = list(incremental.get('deleted_files', []))
|
||||
# Also prune old nodes for re-extracted (changed) files before inserting fresh AST.
|
||||
# Without this, build_merge's dedup pass tries to reconcile old and new versions of
|
||||
# the same file's nodes and can collapse same-named symbols across files (#1178).
|
||||
changed = [f for files in incremental.get('new_files', {}).values() for f in files]
|
||||
prune = list(dict.fromkeys(deleted + changed)) or None
|
||||
# prune_sources is ONLY for genuinely DELETED files. Changed/re-extracted files are
|
||||
# handled by build_merge's replace-on-re-extract (#1344): every source_file in
|
||||
# new_chunks is dropped from the base before merge, so old/stale nodes don't survive.
|
||||
# Do NOT add `changed` here: with root= passed, prune_set relativizes to the same base
|
||||
# as the freshly merged nodes and would DELETE the re-extracted content (#1178 is moot
|
||||
# now that replace — not the dedup pass — reconciles changed files).
|
||||
prune = list(deleted) or None
|
||||
|
||||
# Use build_merge() — reads graph.json directly without NetworkX round-trip
|
||||
# so edge direction (calls, implements, imports) is always preserved (#801).
|
||||
|
||||
@@ -25,5 +25,7 @@ Rules:
|
||||
Node ID format: lowercase, only `[a-z0-9_]`, no dots or slashes. Format `{stem}_{entity}` where stem is `{parent_dir}_{filename_without_ext}` (the immediate parent directory + the filename stem, both lowercased with non-alphanumeric chars replaced by `_`) and entity is the symbol name similarly normalized. Only one level of parent. `src/auth/session.py` + `ValidateToken` → `auth_session_validatetoken`. Top-level files use just the filename stem. This must match the AST extractor's ID. Never append chunk or sequence suffixes — IDs must be deterministic from the label alone.
|
||||
|
||||
Output exactly this JSON (no other text):
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0}
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"<FILE_LIST path verbatim>","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"<FILE_LIST path verbatim>","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"<FILE_LIST path verbatim>"}],"input_tokens":0,"output_tokens":0}
|
||||
|
||||
source_file RULE: set source_file to the FILE_LIST path for that file VERBATIM (absolute, no shortening to basename, no re-relativizing, no separator change). Keeps full build and --update on one base so build_merge's replace matches instead of duplicating.
|
||||
```
|
||||
|
||||
@@ -93,11 +93,13 @@ from graphify.detect import save_manifest
|
||||
new_extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
incremental = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\"))
|
||||
deleted = list(incremental.get('deleted_files', []))
|
||||
# Also prune old nodes for re-extracted (changed) files before inserting fresh AST.
|
||||
# Without this, build_merge's dedup pass tries to reconcile old and new versions of
|
||||
# the same file's nodes and can collapse same-named symbols across files (#1178).
|
||||
changed = [f for files in incremental.get('new_files', {}).values() for f in files]
|
||||
prune = list(dict.fromkeys(deleted + changed)) or None
|
||||
# prune_sources is ONLY for genuinely DELETED files. Changed/re-extracted files are
|
||||
# handled by build_merge's replace-on-re-extract (#1344): every source_file in
|
||||
# new_chunks is dropped from the base before merge, so old/stale nodes don't survive.
|
||||
# Do NOT add `changed` here: with root= passed, prune_set relativizes to the same base
|
||||
# as the freshly merged nodes and would DELETE the re-extracted content (#1178 is moot
|
||||
# now that replace — not the dedup pass — reconciles changed files).
|
||||
prune = list(deleted) or None
|
||||
|
||||
# Use build_merge() — reads graph.json directly without NetworkX round-trip
|
||||
# so edge direction (calls, implements, imports) is always preserved (#801).
|
||||
|
||||
@@ -61,7 +61,9 @@ confidence_score is REQUIRED on every edge - never omit it, never use 0.5 as a d
|
||||
Node ID format: lowercase, only `[a-z0-9_]`, no dots or slashes. Format: `{stem}_{entity}` where stem is `{parent_dir}_{filename_without_ext}` (the **immediate** parent directory name + the filename stem, both lowercased with non-alphanumeric chars replaced by `_`) and entity is the symbol name similarly normalized. Only one level of parent is used — not the full path. Examples: `src/auth/session.py` + `ValidateToken` → `auth_session_validatetoken`; `lib/utils/helpers.py` + `parse_url` → `utils_helpers_parse_url`; `tests/test_foo.py` + `_helper` → `tests_test_foo_helper`. Top-level files (no parent dir, e.g. `setup.py`) use just the filename stem: `setup_my_func`. This must match the ID the AST extractor generates — using just the filename (e.g., `session_validatetoken`) or the full path (e.g., `src_auth_session_validatetoken`) will create orphan ghost-duplicate nodes. If you are re-extracting a project that had ghost duplicates under the old format, the user should run `graphify extract --force` to rebuild cleanly. CRITICAL: never append chunk numbers, sequence numbers, or any suffix to an ID (no `_c1`, `_c2`, `_chunk2`, etc.). IDs must be deterministic from the label alone — the same entity must always produce the same ID regardless of which chunk processes it.
|
||||
|
||||
Generate the extraction JSON matching this schema exactly:
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0}
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"<FILE_LIST path verbatim>","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"<FILE_LIST path verbatim>","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"<FILE_LIST path verbatim>"}],"input_tokens":0,"output_tokens":0}
|
||||
|
||||
source_file RULE (every node, edge, and hyperedge): set source_file to the path of the originating file EXACTLY as it appears in FILE_LIST — verbatim and absolute. Do NOT shorten to a basename, do NOT re-relativize, do NOT strip any directory prefix, and do NOT change separators (the engine canonicalizes separators and relativizes against the build root downstream). Copy the FILE_LIST entry character-for-character. This keeps the full build and incremental --update on the same base, so build_merge's replace-on-re-extract matches the existing node instead of accumulating a duplicate.
|
||||
|
||||
Then write the JSON to disk using the Write tool at this exact absolute path (no relative paths — Write resolves relative paths against an undefined cwd and the file will be silently lost):
|
||||
CHUNK_PATH
|
||||
|
||||
@@ -93,11 +93,13 @@ from graphify.detect import save_manifest
|
||||
new_extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
incremental = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\"))
|
||||
deleted = list(incremental.get('deleted_files', []))
|
||||
# Also prune old nodes for re-extracted (changed) files before inserting fresh AST.
|
||||
# Without this, build_merge's dedup pass tries to reconcile old and new versions of
|
||||
# the same file's nodes and can collapse same-named symbols across files (#1178).
|
||||
changed = [f for files in incremental.get('new_files', {}).values() for f in files]
|
||||
prune = list(dict.fromkeys(deleted + changed)) or None
|
||||
# prune_sources is ONLY for genuinely DELETED files. Changed/re-extracted files are
|
||||
# handled by build_merge's replace-on-re-extract (#1344): every source_file in
|
||||
# new_chunks is dropped from the base before merge, so old/stale nodes don't survive.
|
||||
# Do NOT add `changed` here: with root= passed, prune_set relativizes to the same base
|
||||
# as the freshly merged nodes and would DELETE the re-extracted content (#1178 is moot
|
||||
# now that replace — not the dedup pass — reconciles changed files).
|
||||
prune = list(deleted) or None
|
||||
|
||||
# Use build_merge() — reads graph.json directly without NetworkX round-trip
|
||||
# so edge direction (calls, implements, imports) is always preserved (#801).
|
||||
|
||||
@@ -61,7 +61,9 @@ confidence_score is REQUIRED on every edge - never omit it, never use 0.5 as a d
|
||||
Node ID format: lowercase, only `[a-z0-9_]`, no dots or slashes. Format: `{stem}_{entity}` where stem is `{parent_dir}_{filename_without_ext}` (the **immediate** parent directory name + the filename stem, both lowercased with non-alphanumeric chars replaced by `_`) and entity is the symbol name similarly normalized. Only one level of parent is used — not the full path. Examples: `src/auth/session.py` + `ValidateToken` → `auth_session_validatetoken`; `lib/utils/helpers.py` + `parse_url` → `utils_helpers_parse_url`; `tests/test_foo.py` + `_helper` → `tests_test_foo_helper`. Top-level files (no parent dir, e.g. `setup.py`) use just the filename stem: `setup_my_func`. This must match the ID the AST extractor generates — using just the filename (e.g., `session_validatetoken`) or the full path (e.g., `src_auth_session_validatetoken`) will create orphan ghost-duplicate nodes. If you are re-extracting a project that had ghost duplicates under the old format, the user should run `graphify extract --force` to rebuild cleanly. CRITICAL: never append chunk numbers, sequence numbers, or any suffix to an ID (no `_c1`, `_c2`, `_chunk2`, etc.). IDs must be deterministic from the label alone — the same entity must always produce the same ID regardless of which chunk processes it.
|
||||
|
||||
Generate the extraction JSON matching this schema exactly:
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0}
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"<FILE_LIST path verbatim>","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"<FILE_LIST path verbatim>","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"<FILE_LIST path verbatim>"}],"input_tokens":0,"output_tokens":0}
|
||||
|
||||
source_file RULE (every node, edge, and hyperedge): set source_file to the path of the originating file EXACTLY as it appears in FILE_LIST — verbatim and absolute. Do NOT shorten to a basename, do NOT re-relativize, do NOT strip any directory prefix, and do NOT change separators (the engine canonicalizes separators and relativizes against the build root downstream). Copy the FILE_LIST entry character-for-character. This keeps the full build and incremental --update on the same base, so build_merge's replace-on-re-extract matches the existing node instead of accumulating a duplicate.
|
||||
|
||||
Then write the JSON to disk using the Write tool at this exact absolute path (no relative paths — Write resolves relative paths against an undefined cwd and the file will be silently lost):
|
||||
CHUNK_PATH
|
||||
|
||||
@@ -93,11 +93,13 @@ from graphify.detect import save_manifest
|
||||
new_extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
incremental = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\"))
|
||||
deleted = list(incremental.get('deleted_files', []))
|
||||
# Also prune old nodes for re-extracted (changed) files before inserting fresh AST.
|
||||
# Without this, build_merge's dedup pass tries to reconcile old and new versions of
|
||||
# the same file's nodes and can collapse same-named symbols across files (#1178).
|
||||
changed = [f for files in incremental.get('new_files', {}).values() for f in files]
|
||||
prune = list(dict.fromkeys(deleted + changed)) or None
|
||||
# prune_sources is ONLY for genuinely DELETED files. Changed/re-extracted files are
|
||||
# handled by build_merge's replace-on-re-extract (#1344): every source_file in
|
||||
# new_chunks is dropped from the base before merge, so old/stale nodes don't survive.
|
||||
# Do NOT add `changed` here: with root= passed, prune_set relativizes to the same base
|
||||
# as the freshly merged nodes and would DELETE the re-extracted content (#1178 is moot
|
||||
# now that replace — not the dedup pass — reconciles changed files).
|
||||
prune = list(deleted) or None
|
||||
|
||||
# Use build_merge() — reads graph.json directly without NetworkX round-trip
|
||||
# so edge direction (calls, implements, imports) is always preserved (#801).
|
||||
|
||||
@@ -61,7 +61,9 @@ confidence_score is REQUIRED on every edge - never omit it, never use 0.5 as a d
|
||||
Node ID format: lowercase, only `[a-z0-9_]`, no dots or slashes. Format: `{stem}_{entity}` where stem is `{parent_dir}_{filename_without_ext}` (the **immediate** parent directory name + the filename stem, both lowercased with non-alphanumeric chars replaced by `_`) and entity is the symbol name similarly normalized. Only one level of parent is used — not the full path. Examples: `src/auth/session.py` + `ValidateToken` → `auth_session_validatetoken`; `lib/utils/helpers.py` + `parse_url` → `utils_helpers_parse_url`; `tests/test_foo.py` + `_helper` → `tests_test_foo_helper`. Top-level files (no parent dir, e.g. `setup.py`) use just the filename stem: `setup_my_func`. This must match the ID the AST extractor generates — using just the filename (e.g., `session_validatetoken`) or the full path (e.g., `src_auth_session_validatetoken`) will create orphan ghost-duplicate nodes. If you are re-extracting a project that had ghost duplicates under the old format, the user should run `graphify extract --force` to rebuild cleanly. CRITICAL: never append chunk numbers, sequence numbers, or any suffix to an ID (no `_c1`, `_c2`, `_chunk2`, etc.). IDs must be deterministic from the label alone — the same entity must always produce the same ID regardless of which chunk processes it.
|
||||
|
||||
Generate the extraction JSON matching this schema exactly:
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0}
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"<FILE_LIST path verbatim>","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"<FILE_LIST path verbatim>","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"<FILE_LIST path verbatim>"}],"input_tokens":0,"output_tokens":0}
|
||||
|
||||
source_file RULE (every node, edge, and hyperedge): set source_file to the path of the originating file EXACTLY as it appears in FILE_LIST — verbatim and absolute. Do NOT shorten to a basename, do NOT re-relativize, do NOT strip any directory prefix, and do NOT change separators (the engine canonicalizes separators and relativizes against the build root downstream). Copy the FILE_LIST entry character-for-character. This keeps the full build and incremental --update on the same base, so build_merge's replace-on-re-extract matches the existing node instead of accumulating a duplicate.
|
||||
|
||||
Then write the JSON to disk using the Write tool at this exact absolute path (no relative paths — Write resolves relative paths against an undefined cwd and the file will be silently lost):
|
||||
CHUNK_PATH
|
||||
|
||||
@@ -93,11 +93,13 @@ from graphify.detect import save_manifest
|
||||
new_extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
incremental = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\"))
|
||||
deleted = list(incremental.get('deleted_files', []))
|
||||
# Also prune old nodes for re-extracted (changed) files before inserting fresh AST.
|
||||
# Without this, build_merge's dedup pass tries to reconcile old and new versions of
|
||||
# the same file's nodes and can collapse same-named symbols across files (#1178).
|
||||
changed = [f for files in incremental.get('new_files', {}).values() for f in files]
|
||||
prune = list(dict.fromkeys(deleted + changed)) or None
|
||||
# prune_sources is ONLY for genuinely DELETED files. Changed/re-extracted files are
|
||||
# handled by build_merge's replace-on-re-extract (#1344): every source_file in
|
||||
# new_chunks is dropped from the base before merge, so old/stale nodes don't survive.
|
||||
# Do NOT add `changed` here: with root= passed, prune_set relativizes to the same base
|
||||
# as the freshly merged nodes and would DELETE the re-extracted content (#1178 is moot
|
||||
# now that replace — not the dedup pass — reconciles changed files).
|
||||
prune = list(deleted) or None
|
||||
|
||||
# Use build_merge() — reads graph.json directly without NetworkX round-trip
|
||||
# so edge direction (calls, implements, imports) is always preserved (#801).
|
||||
|
||||
@@ -25,5 +25,7 @@ Rules:
|
||||
Node ID format: lowercase, only `[a-z0-9_]`, no dots or slashes. Format `{stem}_{entity}` where stem is `{parent_dir}_{filename_without_ext}` (the immediate parent directory + the filename stem, both lowercased with non-alphanumeric chars replaced by `_`) and entity is the symbol name similarly normalized. Only one level of parent. `src/auth/session.py` + `ValidateToken` → `auth_session_validatetoken`. Top-level files use just the filename stem. This must match the AST extractor's ID. Never append chunk or sequence suffixes — IDs must be deterministic from the label alone.
|
||||
|
||||
Output exactly this JSON (no other text):
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0}
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"<FILE_LIST path verbatim>","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"<FILE_LIST path verbatim>","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"<FILE_LIST path verbatim>"}],"input_tokens":0,"output_tokens":0}
|
||||
|
||||
source_file RULE: set source_file to the FILE_LIST path for that file VERBATIM (absolute, no shortening to basename, no re-relativizing, no separator change). Keeps full build and --update on one base so build_merge's replace matches instead of duplicating.
|
||||
```
|
||||
|
||||
@@ -93,11 +93,13 @@ from graphify.detect import save_manifest
|
||||
new_extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
incremental = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\"))
|
||||
deleted = list(incremental.get('deleted_files', []))
|
||||
# Also prune old nodes for re-extracted (changed) files before inserting fresh AST.
|
||||
# Without this, build_merge's dedup pass tries to reconcile old and new versions of
|
||||
# the same file's nodes and can collapse same-named symbols across files (#1178).
|
||||
changed = [f for files in incremental.get('new_files', {}).values() for f in files]
|
||||
prune = list(dict.fromkeys(deleted + changed)) or None
|
||||
# prune_sources is ONLY for genuinely DELETED files. Changed/re-extracted files are
|
||||
# handled by build_merge's replace-on-re-extract (#1344): every source_file in
|
||||
# new_chunks is dropped from the base before merge, so old/stale nodes don't survive.
|
||||
# Do NOT add `changed` here: with root= passed, prune_set relativizes to the same base
|
||||
# as the freshly merged nodes and would DELETE the re-extracted content (#1178 is moot
|
||||
# now that replace — not the dedup pass — reconciles changed files).
|
||||
prune = list(deleted) or None
|
||||
|
||||
# Use build_merge() — reads graph.json directly without NetworkX round-trip
|
||||
# so edge direction (calls, implements, imports) is always preserved (#801).
|
||||
|
||||
@@ -61,7 +61,9 @@ confidence_score is REQUIRED on every edge - never omit it, never use 0.5 as a d
|
||||
Node ID format: lowercase, only `[a-z0-9_]`, no dots or slashes. Format: `{stem}_{entity}` where stem is `{parent_dir}_{filename_without_ext}` (the **immediate** parent directory name + the filename stem, both lowercased with non-alphanumeric chars replaced by `_`) and entity is the symbol name similarly normalized. Only one level of parent is used — not the full path. Examples: `src/auth/session.py` + `ValidateToken` → `auth_session_validatetoken`; `lib/utils/helpers.py` + `parse_url` → `utils_helpers_parse_url`; `tests/test_foo.py` + `_helper` → `tests_test_foo_helper`. Top-level files (no parent dir, e.g. `setup.py`) use just the filename stem: `setup_my_func`. This must match the ID the AST extractor generates — using just the filename (e.g., `session_validatetoken`) or the full path (e.g., `src_auth_session_validatetoken`) will create orphan ghost-duplicate nodes. If you are re-extracting a project that had ghost duplicates under the old format, the user should run `graphify extract --force` to rebuild cleanly. CRITICAL: never append chunk numbers, sequence numbers, or any suffix to an ID (no `_c1`, `_c2`, `_chunk2`, etc.). IDs must be deterministic from the label alone — the same entity must always produce the same ID regardless of which chunk processes it.
|
||||
|
||||
Generate the extraction JSON matching this schema exactly:
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0}
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"<FILE_LIST path verbatim>","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"<FILE_LIST path verbatim>","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"<FILE_LIST path verbatim>"}],"input_tokens":0,"output_tokens":0}
|
||||
|
||||
source_file RULE (every node, edge, and hyperedge): set source_file to the path of the originating file EXACTLY as it appears in FILE_LIST — verbatim and absolute. Do NOT shorten to a basename, do NOT re-relativize, do NOT strip any directory prefix, and do NOT change separators (the engine canonicalizes separators and relativizes against the build root downstream). Copy the FILE_LIST entry character-for-character. This keeps the full build and incremental --update on the same base, so build_merge's replace-on-re-extract matches the existing node instead of accumulating a duplicate.
|
||||
|
||||
Then write the JSON to disk using the Write tool at this exact absolute path (no relative paths — Write resolves relative paths against an undefined cwd and the file will be silently lost):
|
||||
CHUNK_PATH
|
||||
|
||||
@@ -93,11 +93,13 @@ from graphify.detect import save_manifest
|
||||
new_extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
incremental = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\"))
|
||||
deleted = list(incremental.get('deleted_files', []))
|
||||
# Also prune old nodes for re-extracted (changed) files before inserting fresh AST.
|
||||
# Without this, build_merge's dedup pass tries to reconcile old and new versions of
|
||||
# the same file's nodes and can collapse same-named symbols across files (#1178).
|
||||
changed = [f for files in incremental.get('new_files', {}).values() for f in files]
|
||||
prune = list(dict.fromkeys(deleted + changed)) or None
|
||||
# prune_sources is ONLY for genuinely DELETED files. Changed/re-extracted files are
|
||||
# handled by build_merge's replace-on-re-extract (#1344): every source_file in
|
||||
# new_chunks is dropped from the base before merge, so old/stale nodes don't survive.
|
||||
# Do NOT add `changed` here: with root= passed, prune_set relativizes to the same base
|
||||
# as the freshly merged nodes and would DELETE the re-extracted content (#1178 is moot
|
||||
# now that replace — not the dedup pass — reconciles changed files).
|
||||
prune = list(deleted) or None
|
||||
|
||||
# Use build_merge() — reads graph.json directly without NetworkX round-trip
|
||||
# so edge direction (calls, implements, imports) is always preserved (#801).
|
||||
|
||||
@@ -25,5 +25,7 @@ Rules:
|
||||
Node ID format: lowercase, only `[a-z0-9_]`, no dots or slashes. Format `{stem}_{entity}` where stem is `{parent_dir}_{filename_without_ext}` (the immediate parent directory + the filename stem, both lowercased with non-alphanumeric chars replaced by `_`) and entity is the symbol name similarly normalized. Only one level of parent. `src/auth/session.py` + `ValidateToken` → `auth_session_validatetoken`. Top-level files use just the filename stem. This must match the AST extractor's ID. Never append chunk or sequence suffixes — IDs must be deterministic from the label alone.
|
||||
|
||||
Output exactly this JSON (no other text):
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0}
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"<FILE_LIST path verbatim>","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"<FILE_LIST path verbatim>","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"<FILE_LIST path verbatim>"}],"input_tokens":0,"output_tokens":0}
|
||||
|
||||
source_file RULE: set source_file to the FILE_LIST path for that file VERBATIM (absolute, no shortening to basename, no re-relativizing, no separator change). Keeps full build and --update on one base so build_merge's replace matches instead of duplicating.
|
||||
```
|
||||
|
||||
@@ -93,11 +93,13 @@ from graphify.detect import save_manifest
|
||||
new_extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
incremental = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\"))
|
||||
deleted = list(incremental.get('deleted_files', []))
|
||||
# Also prune old nodes for re-extracted (changed) files before inserting fresh AST.
|
||||
# Without this, build_merge's dedup pass tries to reconcile old and new versions of
|
||||
# the same file's nodes and can collapse same-named symbols across files (#1178).
|
||||
changed = [f for files in incremental.get('new_files', {}).values() for f in files]
|
||||
prune = list(dict.fromkeys(deleted + changed)) or None
|
||||
# prune_sources is ONLY for genuinely DELETED files. Changed/re-extracted files are
|
||||
# handled by build_merge's replace-on-re-extract (#1344): every source_file in
|
||||
# new_chunks is dropped from the base before merge, so old/stale nodes don't survive.
|
||||
# Do NOT add `changed` here: with root= passed, prune_set relativizes to the same base
|
||||
# as the freshly merged nodes and would DELETE the re-extracted content (#1178 is moot
|
||||
# now that replace — not the dedup pass — reconciles changed files).
|
||||
prune = list(deleted) or None
|
||||
|
||||
# Use build_merge() — reads graph.json directly without NetworkX round-trip
|
||||
# so edge direction (calls, implements, imports) is always preserved (#801).
|
||||
|
||||
@@ -61,7 +61,9 @@ confidence_score is REQUIRED on every edge - never omit it, never use 0.5 as a d
|
||||
Node ID format: lowercase, only `[a-z0-9_]`, no dots or slashes. Format: `{stem}_{entity}` where stem is `{parent_dir}_{filename_without_ext}` (the **immediate** parent directory name + the filename stem, both lowercased with non-alphanumeric chars replaced by `_`) and entity is the symbol name similarly normalized. Only one level of parent is used — not the full path. Examples: `src/auth/session.py` + `ValidateToken` → `auth_session_validatetoken`; `lib/utils/helpers.py` + `parse_url` → `utils_helpers_parse_url`; `tests/test_foo.py` + `_helper` → `tests_test_foo_helper`. Top-level files (no parent dir, e.g. `setup.py`) use just the filename stem: `setup_my_func`. This must match the ID the AST extractor generates — using just the filename (e.g., `session_validatetoken`) or the full path (e.g., `src_auth_session_validatetoken`) will create orphan ghost-duplicate nodes. If you are re-extracting a project that had ghost duplicates under the old format, the user should run `graphify extract --force` to rebuild cleanly. CRITICAL: never append chunk numbers, sequence numbers, or any suffix to an ID (no `_c1`, `_c2`, `_chunk2`, etc.). IDs must be deterministic from the label alone — the same entity must always produce the same ID regardless of which chunk processes it.
|
||||
|
||||
Generate the extraction JSON matching this schema exactly:
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0}
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"<FILE_LIST path verbatim>","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"<FILE_LIST path verbatim>","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"<FILE_LIST path verbatim>"}],"input_tokens":0,"output_tokens":0}
|
||||
|
||||
source_file RULE (every node, edge, and hyperedge): set source_file to the path of the originating file EXACTLY as it appears in FILE_LIST — verbatim and absolute. Do NOT shorten to a basename, do NOT re-relativize, do NOT strip any directory prefix, and do NOT change separators (the engine canonicalizes separators and relativizes against the build root downstream). Copy the FILE_LIST entry character-for-character. This keeps the full build and incremental --update on the same base, so build_merge's replace-on-re-extract matches the existing node instead of accumulating a duplicate.
|
||||
|
||||
Then write the JSON to disk using the Write tool at this exact absolute path (no relative paths — Write resolves relative paths against an undefined cwd and the file will be silently lost):
|
||||
CHUNK_PATH
|
||||
|
||||
@@ -93,11 +93,13 @@ from graphify.detect import save_manifest
|
||||
new_extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
incremental = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\"))
|
||||
deleted = list(incremental.get('deleted_files', []))
|
||||
# Also prune old nodes for re-extracted (changed) files before inserting fresh AST.
|
||||
# Without this, build_merge's dedup pass tries to reconcile old and new versions of
|
||||
# the same file's nodes and can collapse same-named symbols across files (#1178).
|
||||
changed = [f for files in incremental.get('new_files', {}).values() for f in files]
|
||||
prune = list(dict.fromkeys(deleted + changed)) or None
|
||||
# prune_sources is ONLY for genuinely DELETED files. Changed/re-extracted files are
|
||||
# handled by build_merge's replace-on-re-extract (#1344): every source_file in
|
||||
# new_chunks is dropped from the base before merge, so old/stale nodes don't survive.
|
||||
# Do NOT add `changed` here: with root= passed, prune_set relativizes to the same base
|
||||
# as the freshly merged nodes and would DELETE the re-extracted content (#1178 is moot
|
||||
# now that replace — not the dedup pass — reconciles changed files).
|
||||
prune = list(deleted) or None
|
||||
|
||||
# Use build_merge() — reads graph.json directly without NetworkX round-trip
|
||||
# so edge direction (calls, implements, imports) is always preserved (#801).
|
||||
|
||||
@@ -61,7 +61,9 @@ confidence_score is REQUIRED on every edge - never omit it, never use 0.5 as a d
|
||||
Node ID format: lowercase, only `[a-z0-9_]`, no dots or slashes. Format: `{stem}_{entity}` where stem is `{parent_dir}_{filename_without_ext}` (the **immediate** parent directory name + the filename stem, both lowercased with non-alphanumeric chars replaced by `_`) and entity is the symbol name similarly normalized. Only one level of parent is used — not the full path. Examples: `src/auth/session.py` + `ValidateToken` → `auth_session_validatetoken`; `lib/utils/helpers.py` + `parse_url` → `utils_helpers_parse_url`; `tests/test_foo.py` + `_helper` → `tests_test_foo_helper`. Top-level files (no parent dir, e.g. `setup.py`) use just the filename stem: `setup_my_func`. This must match the ID the AST extractor generates — using just the filename (e.g., `session_validatetoken`) or the full path (e.g., `src_auth_session_validatetoken`) will create orphan ghost-duplicate nodes. If you are re-extracting a project that had ghost duplicates under the old format, the user should run `graphify extract --force` to rebuild cleanly. CRITICAL: never append chunk numbers, sequence numbers, or any suffix to an ID (no `_c1`, `_c2`, `_chunk2`, etc.). IDs must be deterministic from the label alone — the same entity must always produce the same ID regardless of which chunk processes it.
|
||||
|
||||
Generate the extraction JSON matching this schema exactly:
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0}
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"<FILE_LIST path verbatim>","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"<FILE_LIST path verbatim>","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"<FILE_LIST path verbatim>"}],"input_tokens":0,"output_tokens":0}
|
||||
|
||||
source_file RULE (every node, edge, and hyperedge): set source_file to the path of the originating file EXACTLY as it appears in FILE_LIST — verbatim and absolute. Do NOT shorten to a basename, do NOT re-relativize, do NOT strip any directory prefix, and do NOT change separators (the engine canonicalizes separators and relativizes against the build root downstream). Copy the FILE_LIST entry character-for-character. This keeps the full build and incremental --update on the same base, so build_merge's replace-on-re-extract matches the existing node instead of accumulating a duplicate.
|
||||
|
||||
Then write the JSON to disk using the Write tool at this exact absolute path (no relative paths — Write resolves relative paths against an undefined cwd and the file will be silently lost):
|
||||
CHUNK_PATH
|
||||
|
||||
@@ -93,11 +93,13 @@ from graphify.detect import save_manifest
|
||||
new_extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
incremental = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\"))
|
||||
deleted = list(incremental.get('deleted_files', []))
|
||||
# Also prune old nodes for re-extracted (changed) files before inserting fresh AST.
|
||||
# Without this, build_merge's dedup pass tries to reconcile old and new versions of
|
||||
# the same file's nodes and can collapse same-named symbols across files (#1178).
|
||||
changed = [f for files in incremental.get('new_files', {}).values() for f in files]
|
||||
prune = list(dict.fromkeys(deleted + changed)) or None
|
||||
# prune_sources is ONLY for genuinely DELETED files. Changed/re-extracted files are
|
||||
# handled by build_merge's replace-on-re-extract (#1344): every source_file in
|
||||
# new_chunks is dropped from the base before merge, so old/stale nodes don't survive.
|
||||
# Do NOT add `changed` here: with root= passed, prune_set relativizes to the same base
|
||||
# as the freshly merged nodes and would DELETE the re-extracted content (#1178 is moot
|
||||
# now that replace — not the dedup pass — reconciles changed files).
|
||||
prune = list(deleted) or None
|
||||
|
||||
# Use build_merge() — reads graph.json directly without NetworkX round-trip
|
||||
# so edge direction (calls, implements, imports) is always preserved (#801).
|
||||
|
||||
@@ -61,7 +61,9 @@ confidence_score is REQUIRED on every edge - never omit it, never use 0.5 as a d
|
||||
Node ID format: lowercase, only `[a-z0-9_]`, no dots or slashes. Format: `{stem}_{entity}` where stem is `{parent_dir}_{filename_without_ext}` (the **immediate** parent directory name + the filename stem, both lowercased with non-alphanumeric chars replaced by `_`) and entity is the symbol name similarly normalized. Only one level of parent is used — not the full path. Examples: `src/auth/session.py` + `ValidateToken` → `auth_session_validatetoken`; `lib/utils/helpers.py` + `parse_url` → `utils_helpers_parse_url`; `tests/test_foo.py` + `_helper` → `tests_test_foo_helper`. Top-level files (no parent dir, e.g. `setup.py`) use just the filename stem: `setup_my_func`. This must match the ID the AST extractor generates — using just the filename (e.g., `session_validatetoken`) or the full path (e.g., `src_auth_session_validatetoken`) will create orphan ghost-duplicate nodes. If you are re-extracting a project that had ghost duplicates under the old format, the user should run `graphify extract --force` to rebuild cleanly. CRITICAL: never append chunk numbers, sequence numbers, or any suffix to an ID (no `_c1`, `_c2`, `_chunk2`, etc.). IDs must be deterministic from the label alone — the same entity must always produce the same ID regardless of which chunk processes it.
|
||||
|
||||
Generate the extraction JSON matching this schema exactly:
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0}
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"<FILE_LIST path verbatim>","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"<FILE_LIST path verbatim>","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"<FILE_LIST path verbatim>"}],"input_tokens":0,"output_tokens":0}
|
||||
|
||||
source_file RULE (every node, edge, and hyperedge): set source_file to the path of the originating file EXACTLY as it appears in FILE_LIST — verbatim and absolute. Do NOT shorten to a basename, do NOT re-relativize, do NOT strip any directory prefix, and do NOT change separators (the engine canonicalizes separators and relativizes against the build root downstream). Copy the FILE_LIST entry character-for-character. This keeps the full build and incremental --update on the same base, so build_merge's replace-on-re-extract matches the existing node instead of accumulating a duplicate.
|
||||
|
||||
Then write the JSON to disk using the Write tool at this exact absolute path (no relative paths — Write resolves relative paths against an undefined cwd and the file will be silently lost):
|
||||
CHUNK_PATH
|
||||
|
||||
@@ -93,11 +93,13 @@ from graphify.detect import save_manifest
|
||||
new_extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
incremental = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\"))
|
||||
deleted = list(incremental.get('deleted_files', []))
|
||||
# Also prune old nodes for re-extracted (changed) files before inserting fresh AST.
|
||||
# Without this, build_merge's dedup pass tries to reconcile old and new versions of
|
||||
# the same file's nodes and can collapse same-named symbols across files (#1178).
|
||||
changed = [f for files in incremental.get('new_files', {}).values() for f in files]
|
||||
prune = list(dict.fromkeys(deleted + changed)) or None
|
||||
# prune_sources is ONLY for genuinely DELETED files. Changed/re-extracted files are
|
||||
# handled by build_merge's replace-on-re-extract (#1344): every source_file in
|
||||
# new_chunks is dropped from the base before merge, so old/stale nodes don't survive.
|
||||
# Do NOT add `changed` here: with root= passed, prune_set relativizes to the same base
|
||||
# as the freshly merged nodes and would DELETE the re-extracted content (#1178 is moot
|
||||
# now that replace — not the dedup pass — reconciles changed files).
|
||||
prune = list(deleted) or None
|
||||
|
||||
# Use build_merge() — reads graph.json directly without NetworkX round-trip
|
||||
# so edge direction (calls, implements, imports) is always preserved (#801).
|
||||
|
||||
@@ -618,6 +618,55 @@ def test_build_merge_replaces_changed_file_stale_edges(tmp_path):
|
||||
assert ("K", "A") in edges, "unchanged file's edge must survive"
|
||||
|
||||
|
||||
def test_build_merge_root_collapses_convention_drift(tmp_path):
|
||||
"""Skill contract: the extraction subagent must emit source_file as the
|
||||
verbatim path from FILE_LIST AND the caller must pass root= (the build root).
|
||||
Then build_merge canonicalizes the new chunk to the same relative base as the
|
||||
stored graph, so re-extraction REPLACES the prior node (incl. stale nodes for
|
||||
that file) instead of accumulating a duplicate. Without root, a drifted
|
||||
relative base (e.g. a bare basename from a different run) mismatches and the
|
||||
graph duplicates. Engine is unchanged — this pins the prompt/root contract."""
|
||||
import networkx as nx
|
||||
|
||||
root = tmp_path
|
||||
graph_path = tmp_path / "graphify-out" / "graph.json"
|
||||
graph_path.parent.mkdir(parents=True)
|
||||
|
||||
# Stored graph: nested project-relative convention + a STALE node for the same
|
||||
# file that the re-extraction no longer emits.
|
||||
stored = {"nodes": [
|
||||
{"id": "wiki_overview_overview", "label": "Overview", "file_type": "document",
|
||||
"source_file": "docs/wiki/overview.md"},
|
||||
{"id": "wiki_overview_stale", "label": "Stale", "file_type": "document",
|
||||
"source_file": "docs/wiki/overview.md"},
|
||||
], "edges": []}
|
||||
G0 = build([stored], dedup=False)
|
||||
saved = json.dumps(nx.node_link_data(G0, edges="edges"))
|
||||
graph_path.write_text(saved, encoding="utf-8")
|
||||
|
||||
# BUG: --update drifted to a bare basename and no root was passed. Different
|
||||
# base -> source_file replace misses -> stale + duplicate both survive.
|
||||
drift = {"nodes": [
|
||||
{"id": "overview_overview", "label": "Overview", "file_type": "document",
|
||||
"source_file": "overview.md"},
|
||||
], "edges": []}
|
||||
G_bug = build_merge([drift], graph_path, dedup=False)
|
||||
assert G_bug.number_of_nodes() == 3, "mismatched base must NOT replace -> stale+dup remain"
|
||||
|
||||
# FIX: subagent emits the verbatim path; caller passes root (the build root).
|
||||
graph_path.write_text(saved, encoding="utf-8")
|
||||
abs_overview = str(root / "docs" / "wiki" / "overview.md")
|
||||
fixed = {"nodes": [
|
||||
{"id": "wiki_overview_overview", "label": "Overview", "file_type": "document",
|
||||
"source_file": abs_overview},
|
||||
], "edges": []}
|
||||
G_ok = build_merge([fixed], graph_path, prune_sources=None, dedup=False, root=root)
|
||||
assert G_ok.number_of_nodes() == 1, "verbatim path + root must collapse to one node"
|
||||
assert "wiki_overview_stale" not in G_ok, "stale node for the re-extracted file must be dropped"
|
||||
assert G_ok.nodes["wiki_overview_overview"]["source_file"] == "docs/wiki/overview.md", \
|
||||
"new chunk must be canonicalized to the stored relative base"
|
||||
|
||||
|
||||
def test_build_merge_rejects_oversized_existing_graph(monkeypatch, tmp_path):
|
||||
"""#F4: build_merge must refuse to read an existing graph.json that
|
||||
exceeds the size cap, rather than json.loads-ing it into memory."""
|
||||
|
||||
@@ -382,7 +382,9 @@ from pathlib import Path
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= mirrors the --update runbook (#1361): relativize source_file to the same
|
||||
# base so the full build and incremental --update never drift apart on re-extract.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = cluster(G)
|
||||
cohesion = score_all(G, communities)
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
@@ -435,7 +437,8 @@ extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(en
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= as in Step 4 / the --update runbook (#1361) — same base for node-key parity.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
|
||||
@@ -385,7 +385,9 @@ from pathlib import Path
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= mirrors the --update runbook (#1361): relativize source_file to the same
|
||||
# base so the full build and incremental --update never drift apart on re-extract.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = cluster(G)
|
||||
cohesion = score_all(G, communities)
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
@@ -438,7 +440,8 @@ extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(en
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= as in Step 4 / the --update runbook (#1361) — same base for node-key parity.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
|
||||
@@ -382,7 +382,9 @@ from pathlib import Path
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= mirrors the --update runbook (#1361): relativize source_file to the same
|
||||
# base so the full build and incremental --update never drift apart on re-extract.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = cluster(G)
|
||||
cohesion = score_all(G, communities)
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
@@ -435,7 +437,8 @@ extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(en
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= as in Step 4 / the --update runbook (#1361) — same base for node-key parity.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
|
||||
@@ -385,7 +385,9 @@ from pathlib import Path
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= mirrors the --update runbook (#1361): relativize source_file to the same
|
||||
# base so the full build and incremental --update never drift apart on re-extract.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = cluster(G)
|
||||
cohesion = score_all(G, communities)
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
@@ -438,7 +440,8 @@ extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(en
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= as in Step 4 / the --update runbook (#1361) — same base for node-key parity.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
|
||||
@@ -382,7 +382,9 @@ from pathlib import Path
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= mirrors the --update runbook (#1361): relativize source_file to the same
|
||||
# base so the full build and incremental --update never drift apart on re-extract.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = cluster(G)
|
||||
cohesion = score_all(G, communities)
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
@@ -435,7 +437,8 @@ extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(en
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= as in Step 4 / the --update runbook (#1361) — same base for node-key parity.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
|
||||
@@ -385,7 +385,9 @@ from pathlib import Path
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= mirrors the --update runbook (#1361): relativize source_file to the same
|
||||
# base so the full build and incremental --update never drift apart on re-extract.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = cluster(G)
|
||||
cohesion = score_all(G, communities)
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
@@ -438,7 +440,8 @@ extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(en
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= as in Step 4 / the --update runbook (#1361) — same base for node-key parity.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
|
||||
@@ -385,7 +385,9 @@ from pathlib import Path
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= mirrors the --update runbook (#1361): relativize source_file to the same
|
||||
# base so the full build and incremental --update never drift apart on re-extract.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = cluster(G)
|
||||
cohesion = score_all(G, communities)
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
@@ -438,7 +440,8 @@ extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(en
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= as in Step 4 / the --update runbook (#1361) — same base for node-key parity.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
|
||||
@@ -377,7 +377,9 @@ from pathlib import Path
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= mirrors the --update runbook (#1361): relativize source_file to the same
|
||||
# base so the full build and incremental --update never drift apart on re-extract.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = cluster(G)
|
||||
cohesion = score_all(G, communities)
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
@@ -430,7 +432,8 @@ extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(en
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= as in Step 4 / the --update runbook (#1361) — same base for node-key parity.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
|
||||
@@ -385,7 +385,9 @@ from pathlib import Path
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= mirrors the --update runbook (#1361): relativize source_file to the same
|
||||
# base so the full build and incremental --update never drift apart on re-extract.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = cluster(G)
|
||||
cohesion = score_all(G, communities)
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
@@ -438,7 +440,8 @@ extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(en
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= as in Step 4 / the --update runbook (#1361) — same base for node-key parity.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
|
||||
@@ -383,7 +383,9 @@ from pathlib import Path
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= mirrors the --update runbook (#1361): relativize source_file to the same
|
||||
# base so the full build and incremental --update never drift apart on re-extract.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = cluster(G)
|
||||
cohesion = score_all(G, communities)
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
@@ -436,7 +438,8 @@ extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(en
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= as in Step 4 / the --update runbook (#1361) — same base for node-key parity.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
|
||||
@@ -381,7 +381,9 @@ from pathlib import Path
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= mirrors the --update runbook (#1361): relativize source_file to the same
|
||||
# base so the full build and incremental --update never drift apart on re-extract.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = cluster(G)
|
||||
cohesion = score_all(G, communities)
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
@@ -434,7 +436,8 @@ extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(en
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= as in Step 4 / the --update runbook (#1361) — same base for node-key parity.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
|
||||
@@ -407,7 +407,9 @@ from pathlib import Path
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= mirrors the --update runbook (#1361): relativize source_file to the same
|
||||
# base so the full build and incremental --update never drift apart on re-extract.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = cluster(G)
|
||||
cohesion = score_all(G, communities)
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
@@ -460,7 +462,8 @@ extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(en
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= as in Step 4 / the --update runbook (#1361) — same base for node-key parity.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
|
||||
@@ -385,7 +385,9 @@ from pathlib import Path
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= mirrors the --update runbook (#1361): relativize source_file to the same
|
||||
# base so the full build and incremental --update never drift apart on re-extract.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = cluster(G)
|
||||
cohesion = score_all(G, communities)
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
@@ -438,7 +440,8 @@ extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(en
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= as in Step 4 / the --update runbook (#1361) — same base for node-key parity.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
|
||||
@@ -61,7 +61,9 @@ confidence_score is REQUIRED on every edge - never omit it, never use 0.5 as a d
|
||||
Node ID format: lowercase, only `[a-z0-9_]`, no dots or slashes. Format: `{stem}_{entity}` where stem is `{parent_dir}_{filename_without_ext}` (the **immediate** parent directory name + the filename stem, both lowercased with non-alphanumeric chars replaced by `_`) and entity is the symbol name similarly normalized. Only one level of parent is used — not the full path. Examples: `src/auth/session.py` + `ValidateToken` → `auth_session_validatetoken`; `lib/utils/helpers.py` + `parse_url` → `utils_helpers_parse_url`; `tests/test_foo.py` + `_helper` → `tests_test_foo_helper`. Top-level files (no parent dir, e.g. `setup.py`) use just the filename stem: `setup_my_func`. This must match the ID the AST extractor generates — using just the filename (e.g., `session_validatetoken`) or the full path (e.g., `src_auth_session_validatetoken`) will create orphan ghost-duplicate nodes. If you are re-extracting a project that had ghost duplicates under the old format, the user should run `graphify extract --force` to rebuild cleanly. CRITICAL: never append chunk numbers, sequence numbers, or any suffix to an ID (no `_c1`, `_c2`, `_chunk2`, etc.). IDs must be deterministic from the label alone — the same entity must always produce the same ID regardless of which chunk processes it.
|
||||
|
||||
Generate the extraction JSON matching this schema exactly:
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0}
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"<FILE_LIST path verbatim>","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"<FILE_LIST path verbatim>","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"<FILE_LIST path verbatim>"}],"input_tokens":0,"output_tokens":0}
|
||||
|
||||
source_file RULE (every node, edge, and hyperedge): set source_file to the path of the originating file EXACTLY as it appears in FILE_LIST — verbatim and absolute. Do NOT shorten to a basename, do NOT re-relativize, do NOT strip any directory prefix, and do NOT change separators (the engine canonicalizes separators and relativizes against the build root downstream). Copy the FILE_LIST entry character-for-character. This keeps the full build and incremental --update on the same base, so build_merge's replace-on-re-extract matches the existing node instead of accumulating a duplicate.
|
||||
|
||||
Then write the JSON to disk using the Write tool at this exact absolute path (no relative paths — Write resolves relative paths against an undefined cwd and the file will be silently lost):
|
||||
CHUNK_PATH
|
||||
|
||||
@@ -93,11 +93,13 @@ from graphify.detect import save_manifest
|
||||
new_extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
incremental = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\"))
|
||||
deleted = list(incremental.get('deleted_files', []))
|
||||
# Also prune old nodes for re-extracted (changed) files before inserting fresh AST.
|
||||
# Without this, build_merge's dedup pass tries to reconcile old and new versions of
|
||||
# the same file's nodes and can collapse same-named symbols across files (#1178).
|
||||
changed = [f for files in incremental.get('new_files', {}).values() for f in files]
|
||||
prune = list(dict.fromkeys(deleted + changed)) or None
|
||||
# prune_sources is ONLY for genuinely DELETED files. Changed/re-extracted files are
|
||||
# handled by build_merge's replace-on-re-extract (#1344): every source_file in
|
||||
# new_chunks is dropped from the base before merge, so old/stale nodes don't survive.
|
||||
# Do NOT add `changed` here: with root= passed, prune_set relativizes to the same base
|
||||
# as the freshly merged nodes and would DELETE the re-extracted content (#1178 is moot
|
||||
# now that replace — not the dedup pass — reconciles changed files).
|
||||
prune = list(deleted) or None
|
||||
|
||||
# Use build_merge() — reads graph.json directly without NetworkX round-trip
|
||||
# so edge direction (calls, implements, imports) is always preserved (#801).
|
||||
|
||||
@@ -61,7 +61,9 @@ confidence_score is REQUIRED on every edge - never omit it, never use 0.5 as a d
|
||||
Node ID format: lowercase, only `[a-z0-9_]`, no dots or slashes. Format: `{stem}_{entity}` where stem is `{parent_dir}_{filename_without_ext}` (the **immediate** parent directory name + the filename stem, both lowercased with non-alphanumeric chars replaced by `_`) and entity is the symbol name similarly normalized. Only one level of parent is used — not the full path. Examples: `src/auth/session.py` + `ValidateToken` → `auth_session_validatetoken`; `lib/utils/helpers.py` + `parse_url` → `utils_helpers_parse_url`; `tests/test_foo.py` + `_helper` → `tests_test_foo_helper`. Top-level files (no parent dir, e.g. `setup.py`) use just the filename stem: `setup_my_func`. This must match the ID the AST extractor generates — using just the filename (e.g., `session_validatetoken`) or the full path (e.g., `src_auth_session_validatetoken`) will create orphan ghost-duplicate nodes. If you are re-extracting a project that had ghost duplicates under the old format, the user should run `graphify extract --force` to rebuild cleanly. CRITICAL: never append chunk numbers, sequence numbers, or any suffix to an ID (no `_c1`, `_c2`, `_chunk2`, etc.). IDs must be deterministic from the label alone — the same entity must always produce the same ID regardless of which chunk processes it.
|
||||
|
||||
Generate the extraction JSON matching this schema exactly:
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0}
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"<FILE_LIST path verbatim>","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"<FILE_LIST path verbatim>","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"<FILE_LIST path verbatim>"}],"input_tokens":0,"output_tokens":0}
|
||||
|
||||
source_file RULE (every node, edge, and hyperedge): set source_file to the path of the originating file EXACTLY as it appears in FILE_LIST — verbatim and absolute. Do NOT shorten to a basename, do NOT re-relativize, do NOT strip any directory prefix, and do NOT change separators (the engine canonicalizes separators and relativizes against the build root downstream). Copy the FILE_LIST entry character-for-character. This keeps the full build and incremental --update on the same base, so build_merge's replace-on-re-extract matches the existing node instead of accumulating a duplicate.
|
||||
|
||||
Then write the JSON to disk using the Write tool at this exact absolute path (no relative paths — Write resolves relative paths against an undefined cwd and the file will be silently lost):
|
||||
CHUNK_PATH
|
||||
|
||||
@@ -93,11 +93,13 @@ from graphify.detect import save_manifest
|
||||
new_extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
incremental = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\"))
|
||||
deleted = list(incremental.get('deleted_files', []))
|
||||
# Also prune old nodes for re-extracted (changed) files before inserting fresh AST.
|
||||
# Without this, build_merge's dedup pass tries to reconcile old and new versions of
|
||||
# the same file's nodes and can collapse same-named symbols across files (#1178).
|
||||
changed = [f for files in incremental.get('new_files', {}).values() for f in files]
|
||||
prune = list(dict.fromkeys(deleted + changed)) or None
|
||||
# prune_sources is ONLY for genuinely DELETED files. Changed/re-extracted files are
|
||||
# handled by build_merge's replace-on-re-extract (#1344): every source_file in
|
||||
# new_chunks is dropped from the base before merge, so old/stale nodes don't survive.
|
||||
# Do NOT add `changed` here: with root= passed, prune_set relativizes to the same base
|
||||
# as the freshly merged nodes and would DELETE the re-extracted content (#1178 is moot
|
||||
# now that replace — not the dedup pass — reconciles changed files).
|
||||
prune = list(deleted) or None
|
||||
|
||||
# Use build_merge() — reads graph.json directly without NetworkX round-trip
|
||||
# so edge direction (calls, implements, imports) is always preserved (#801).
|
||||
|
||||
@@ -25,5 +25,7 @@ Rules:
|
||||
Node ID format: lowercase, only `[a-z0-9_]`, no dots or slashes. Format `{stem}_{entity}` where stem is `{parent_dir}_{filename_without_ext}` (the immediate parent directory + the filename stem, both lowercased with non-alphanumeric chars replaced by `_`) and entity is the symbol name similarly normalized. Only one level of parent. `src/auth/session.py` + `ValidateToken` → `auth_session_validatetoken`. Top-level files use just the filename stem. This must match the AST extractor's ID. Never append chunk or sequence suffixes — IDs must be deterministic from the label alone.
|
||||
|
||||
Output exactly this JSON (no other text):
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0}
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"<FILE_LIST path verbatim>","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"<FILE_LIST path verbatim>","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"<FILE_LIST path verbatim>"}],"input_tokens":0,"output_tokens":0}
|
||||
|
||||
source_file RULE: set source_file to the FILE_LIST path for that file VERBATIM (absolute, no shortening to basename, no re-relativizing, no separator change). Keeps full build and --update on one base so build_merge's replace matches instead of duplicating.
|
||||
```
|
||||
|
||||
@@ -93,11 +93,13 @@ from graphify.detect import save_manifest
|
||||
new_extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
incremental = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\"))
|
||||
deleted = list(incremental.get('deleted_files', []))
|
||||
# Also prune old nodes for re-extracted (changed) files before inserting fresh AST.
|
||||
# Without this, build_merge's dedup pass tries to reconcile old and new versions of
|
||||
# the same file's nodes and can collapse same-named symbols across files (#1178).
|
||||
changed = [f for files in incremental.get('new_files', {}).values() for f in files]
|
||||
prune = list(dict.fromkeys(deleted + changed)) or None
|
||||
# prune_sources is ONLY for genuinely DELETED files. Changed/re-extracted files are
|
||||
# handled by build_merge's replace-on-re-extract (#1344): every source_file in
|
||||
# new_chunks is dropped from the base before merge, so old/stale nodes don't survive.
|
||||
# Do NOT add `changed` here: with root= passed, prune_set relativizes to the same base
|
||||
# as the freshly merged nodes and would DELETE the re-extracted content (#1178 is moot
|
||||
# now that replace — not the dedup pass — reconciles changed files).
|
||||
prune = list(deleted) or None
|
||||
|
||||
# Use build_merge() — reads graph.json directly without NetworkX round-trip
|
||||
# so edge direction (calls, implements, imports) is always preserved (#801).
|
||||
|
||||
@@ -25,5 +25,7 @@ Rules:
|
||||
Node ID format: lowercase, only `[a-z0-9_]`, no dots or slashes. Format `{stem}_{entity}` where stem is `{parent_dir}_{filename_without_ext}` (the immediate parent directory + the filename stem, both lowercased with non-alphanumeric chars replaced by `_`) and entity is the symbol name similarly normalized. Only one level of parent. `src/auth/session.py` + `ValidateToken` → `auth_session_validatetoken`. Top-level files use just the filename stem. This must match the AST extractor's ID. Never append chunk or sequence suffixes — IDs must be deterministic from the label alone.
|
||||
|
||||
Output exactly this JSON (no other text):
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0}
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"<FILE_LIST path verbatim>","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"<FILE_LIST path verbatim>","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"<FILE_LIST path verbatim>"}],"input_tokens":0,"output_tokens":0}
|
||||
|
||||
source_file RULE: set source_file to the FILE_LIST path for that file VERBATIM (absolute, no shortening to basename, no re-relativizing, no separator change). Keeps full build and --update on one base so build_merge's replace matches instead of duplicating.
|
||||
```
|
||||
|
||||
@@ -93,11 +93,13 @@ from graphify.detect import save_manifest
|
||||
new_extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
incremental = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\"))
|
||||
deleted = list(incremental.get('deleted_files', []))
|
||||
# Also prune old nodes for re-extracted (changed) files before inserting fresh AST.
|
||||
# Without this, build_merge's dedup pass tries to reconcile old and new versions of
|
||||
# the same file's nodes and can collapse same-named symbols across files (#1178).
|
||||
changed = [f for files in incremental.get('new_files', {}).values() for f in files]
|
||||
prune = list(dict.fromkeys(deleted + changed)) or None
|
||||
# prune_sources is ONLY for genuinely DELETED files. Changed/re-extracted files are
|
||||
# handled by build_merge's replace-on-re-extract (#1344): every source_file in
|
||||
# new_chunks is dropped from the base before merge, so old/stale nodes don't survive.
|
||||
# Do NOT add `changed` here: with root= passed, prune_set relativizes to the same base
|
||||
# as the freshly merged nodes and would DELETE the re-extracted content (#1178 is moot
|
||||
# now that replace — not the dedup pass — reconciles changed files).
|
||||
prune = list(deleted) or None
|
||||
|
||||
# Use build_merge() — reads graph.json directly without NetworkX round-trip
|
||||
# so edge direction (calls, implements, imports) is always preserved (#801).
|
||||
|
||||
@@ -61,7 +61,9 @@ confidence_score is REQUIRED on every edge - never omit it, never use 0.5 as a d
|
||||
Node ID format: lowercase, only `[a-z0-9_]`, no dots or slashes. Format: `{stem}_{entity}` where stem is `{parent_dir}_{filename_without_ext}` (the **immediate** parent directory name + the filename stem, both lowercased with non-alphanumeric chars replaced by `_`) and entity is the symbol name similarly normalized. Only one level of parent is used — not the full path. Examples: `src/auth/session.py` + `ValidateToken` → `auth_session_validatetoken`; `lib/utils/helpers.py` + `parse_url` → `utils_helpers_parse_url`; `tests/test_foo.py` + `_helper` → `tests_test_foo_helper`. Top-level files (no parent dir, e.g. `setup.py`) use just the filename stem: `setup_my_func`. This must match the ID the AST extractor generates — using just the filename (e.g., `session_validatetoken`) or the full path (e.g., `src_auth_session_validatetoken`) will create orphan ghost-duplicate nodes. If you are re-extracting a project that had ghost duplicates under the old format, the user should run `graphify extract --force` to rebuild cleanly. CRITICAL: never append chunk numbers, sequence numbers, or any suffix to an ID (no `_c1`, `_c2`, `_chunk2`, etc.). IDs must be deterministic from the label alone — the same entity must always produce the same ID regardless of which chunk processes it.
|
||||
|
||||
Generate the extraction JSON matching this schema exactly:
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0}
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"<FILE_LIST path verbatim>","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"<FILE_LIST path verbatim>","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"<FILE_LIST path verbatim>"}],"input_tokens":0,"output_tokens":0}
|
||||
|
||||
source_file RULE (every node, edge, and hyperedge): set source_file to the path of the originating file EXACTLY as it appears in FILE_LIST — verbatim and absolute. Do NOT shorten to a basename, do NOT re-relativize, do NOT strip any directory prefix, and do NOT change separators (the engine canonicalizes separators and relativizes against the build root downstream). Copy the FILE_LIST entry character-for-character. This keeps the full build and incremental --update on the same base, so build_merge's replace-on-re-extract matches the existing node instead of accumulating a duplicate.
|
||||
|
||||
Then write the JSON to disk using the Write tool at this exact absolute path (no relative paths — Write resolves relative paths against an undefined cwd and the file will be silently lost):
|
||||
CHUNK_PATH
|
||||
|
||||
@@ -93,11 +93,13 @@ from graphify.detect import save_manifest
|
||||
new_extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
incremental = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\"))
|
||||
deleted = list(incremental.get('deleted_files', []))
|
||||
# Also prune old nodes for re-extracted (changed) files before inserting fresh AST.
|
||||
# Without this, build_merge's dedup pass tries to reconcile old and new versions of
|
||||
# the same file's nodes and can collapse same-named symbols across files (#1178).
|
||||
changed = [f for files in incremental.get('new_files', {}).values() for f in files]
|
||||
prune = list(dict.fromkeys(deleted + changed)) or None
|
||||
# prune_sources is ONLY for genuinely DELETED files. Changed/re-extracted files are
|
||||
# handled by build_merge's replace-on-re-extract (#1344): every source_file in
|
||||
# new_chunks is dropped from the base before merge, so old/stale nodes don't survive.
|
||||
# Do NOT add `changed` here: with root= passed, prune_set relativizes to the same base
|
||||
# as the freshly merged nodes and would DELETE the re-extracted content (#1178 is moot
|
||||
# now that replace — not the dedup pass — reconciles changed files).
|
||||
prune = list(deleted) or None
|
||||
|
||||
# Use build_merge() — reads graph.json directly without NetworkX round-trip
|
||||
# so edge direction (calls, implements, imports) is always preserved (#801).
|
||||
|
||||
@@ -61,7 +61,9 @@ confidence_score is REQUIRED on every edge - never omit it, never use 0.5 as a d
|
||||
Node ID format: lowercase, only `[a-z0-9_]`, no dots or slashes. Format: `{stem}_{entity}` where stem is `{parent_dir}_{filename_without_ext}` (the **immediate** parent directory name + the filename stem, both lowercased with non-alphanumeric chars replaced by `_`) and entity is the symbol name similarly normalized. Only one level of parent is used — not the full path. Examples: `src/auth/session.py` + `ValidateToken` → `auth_session_validatetoken`; `lib/utils/helpers.py` + `parse_url` → `utils_helpers_parse_url`; `tests/test_foo.py` + `_helper` → `tests_test_foo_helper`. Top-level files (no parent dir, e.g. `setup.py`) use just the filename stem: `setup_my_func`. This must match the ID the AST extractor generates — using just the filename (e.g., `session_validatetoken`) or the full path (e.g., `src_auth_session_validatetoken`) will create orphan ghost-duplicate nodes. If you are re-extracting a project that had ghost duplicates under the old format, the user should run `graphify extract --force` to rebuild cleanly. CRITICAL: never append chunk numbers, sequence numbers, or any suffix to an ID (no `_c1`, `_c2`, `_chunk2`, etc.). IDs must be deterministic from the label alone — the same entity must always produce the same ID regardless of which chunk processes it.
|
||||
|
||||
Generate the extraction JSON matching this schema exactly:
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0}
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"<FILE_LIST path verbatim>","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"<FILE_LIST path verbatim>","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"<FILE_LIST path verbatim>"}],"input_tokens":0,"output_tokens":0}
|
||||
|
||||
source_file RULE (every node, edge, and hyperedge): set source_file to the path of the originating file EXACTLY as it appears in FILE_LIST — verbatim and absolute. Do NOT shorten to a basename, do NOT re-relativize, do NOT strip any directory prefix, and do NOT change separators (the engine canonicalizes separators and relativizes against the build root downstream). Copy the FILE_LIST entry character-for-character. This keeps the full build and incremental --update on the same base, so build_merge's replace-on-re-extract matches the existing node instead of accumulating a duplicate.
|
||||
|
||||
Then write the JSON to disk using the Write tool at this exact absolute path (no relative paths — Write resolves relative paths against an undefined cwd and the file will be silently lost):
|
||||
CHUNK_PATH
|
||||
|
||||
@@ -93,11 +93,13 @@ from graphify.detect import save_manifest
|
||||
new_extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
incremental = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\"))
|
||||
deleted = list(incremental.get('deleted_files', []))
|
||||
# Also prune old nodes for re-extracted (changed) files before inserting fresh AST.
|
||||
# Without this, build_merge's dedup pass tries to reconcile old and new versions of
|
||||
# the same file's nodes and can collapse same-named symbols across files (#1178).
|
||||
changed = [f for files in incremental.get('new_files', {}).values() for f in files]
|
||||
prune = list(dict.fromkeys(deleted + changed)) or None
|
||||
# prune_sources is ONLY for genuinely DELETED files. Changed/re-extracted files are
|
||||
# handled by build_merge's replace-on-re-extract (#1344): every source_file in
|
||||
# new_chunks is dropped from the base before merge, so old/stale nodes don't survive.
|
||||
# Do NOT add `changed` here: with root= passed, prune_set relativizes to the same base
|
||||
# as the freshly merged nodes and would DELETE the re-extracted content (#1178 is moot
|
||||
# now that replace — not the dedup pass — reconciles changed files).
|
||||
prune = list(deleted) or None
|
||||
|
||||
# Use build_merge() — reads graph.json directly without NetworkX round-trip
|
||||
# so edge direction (calls, implements, imports) is always preserved (#801).
|
||||
|
||||
@@ -61,7 +61,9 @@ confidence_score is REQUIRED on every edge - never omit it, never use 0.5 as a d
|
||||
Node ID format: lowercase, only `[a-z0-9_]`, no dots or slashes. Format: `{stem}_{entity}` where stem is `{parent_dir}_{filename_without_ext}` (the **immediate** parent directory name + the filename stem, both lowercased with non-alphanumeric chars replaced by `_`) and entity is the symbol name similarly normalized. Only one level of parent is used — not the full path. Examples: `src/auth/session.py` + `ValidateToken` → `auth_session_validatetoken`; `lib/utils/helpers.py` + `parse_url` → `utils_helpers_parse_url`; `tests/test_foo.py` + `_helper` → `tests_test_foo_helper`. Top-level files (no parent dir, e.g. `setup.py`) use just the filename stem: `setup_my_func`. This must match the ID the AST extractor generates — using just the filename (e.g., `session_validatetoken`) or the full path (e.g., `src_auth_session_validatetoken`) will create orphan ghost-duplicate nodes. If you are re-extracting a project that had ghost duplicates under the old format, the user should run `graphify extract --force` to rebuild cleanly. CRITICAL: never append chunk numbers, sequence numbers, or any suffix to an ID (no `_c1`, `_c2`, `_chunk2`, etc.). IDs must be deterministic from the label alone — the same entity must always produce the same ID regardless of which chunk processes it.
|
||||
|
||||
Generate the extraction JSON matching this schema exactly:
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0}
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"<FILE_LIST path verbatim>","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"<FILE_LIST path verbatim>","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"<FILE_LIST path verbatim>"}],"input_tokens":0,"output_tokens":0}
|
||||
|
||||
source_file RULE (every node, edge, and hyperedge): set source_file to the path of the originating file EXACTLY as it appears in FILE_LIST — verbatim and absolute. Do NOT shorten to a basename, do NOT re-relativize, do NOT strip any directory prefix, and do NOT change separators (the engine canonicalizes separators and relativizes against the build root downstream). Copy the FILE_LIST entry character-for-character. This keeps the full build and incremental --update on the same base, so build_merge's replace-on-re-extract matches the existing node instead of accumulating a duplicate.
|
||||
|
||||
Then write the JSON to disk using the Write tool at this exact absolute path (no relative paths — Write resolves relative paths against an undefined cwd and the file will be silently lost):
|
||||
CHUNK_PATH
|
||||
|
||||
@@ -93,11 +93,13 @@ from graphify.detect import save_manifest
|
||||
new_extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
incremental = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\"))
|
||||
deleted = list(incremental.get('deleted_files', []))
|
||||
# Also prune old nodes for re-extracted (changed) files before inserting fresh AST.
|
||||
# Without this, build_merge's dedup pass tries to reconcile old and new versions of
|
||||
# the same file's nodes and can collapse same-named symbols across files (#1178).
|
||||
changed = [f for files in incremental.get('new_files', {}).values() for f in files]
|
||||
prune = list(dict.fromkeys(deleted + changed)) or None
|
||||
# prune_sources is ONLY for genuinely DELETED files. Changed/re-extracted files are
|
||||
# handled by build_merge's replace-on-re-extract (#1344): every source_file in
|
||||
# new_chunks is dropped from the base before merge, so old/stale nodes don't survive.
|
||||
# Do NOT add `changed` here: with root= passed, prune_set relativizes to the same base
|
||||
# as the freshly merged nodes and would DELETE the re-extracted content (#1178 is moot
|
||||
# now that replace — not the dedup pass — reconciles changed files).
|
||||
prune = list(deleted) or None
|
||||
|
||||
# Use build_merge() — reads graph.json directly without NetworkX round-trip
|
||||
# so edge direction (calls, implements, imports) is always preserved (#801).
|
||||
|
||||
@@ -25,5 +25,7 @@ Rules:
|
||||
Node ID format: lowercase, only `[a-z0-9_]`, no dots or slashes. Format `{stem}_{entity}` where stem is `{parent_dir}_{filename_without_ext}` (the immediate parent directory + the filename stem, both lowercased with non-alphanumeric chars replaced by `_`) and entity is the symbol name similarly normalized. Only one level of parent. `src/auth/session.py` + `ValidateToken` → `auth_session_validatetoken`. Top-level files use just the filename stem. This must match the AST extractor's ID. Never append chunk or sequence suffixes — IDs must be deterministic from the label alone.
|
||||
|
||||
Output exactly this JSON (no other text):
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0}
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"<FILE_LIST path verbatim>","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"<FILE_LIST path verbatim>","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"<FILE_LIST path verbatim>"}],"input_tokens":0,"output_tokens":0}
|
||||
|
||||
source_file RULE: set source_file to the FILE_LIST path for that file VERBATIM (absolute, no shortening to basename, no re-relativizing, no separator change). Keeps full build and --update on one base so build_merge's replace matches instead of duplicating.
|
||||
```
|
||||
|
||||
@@ -93,11 +93,13 @@ from graphify.detect import save_manifest
|
||||
new_extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
incremental = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\"))
|
||||
deleted = list(incremental.get('deleted_files', []))
|
||||
# Also prune old nodes for re-extracted (changed) files before inserting fresh AST.
|
||||
# Without this, build_merge's dedup pass tries to reconcile old and new versions of
|
||||
# the same file's nodes and can collapse same-named symbols across files (#1178).
|
||||
changed = [f for files in incremental.get('new_files', {}).values() for f in files]
|
||||
prune = list(dict.fromkeys(deleted + changed)) or None
|
||||
# prune_sources is ONLY for genuinely DELETED files. Changed/re-extracted files are
|
||||
# handled by build_merge's replace-on-re-extract (#1344): every source_file in
|
||||
# new_chunks is dropped from the base before merge, so old/stale nodes don't survive.
|
||||
# Do NOT add `changed` here: with root= passed, prune_set relativizes to the same base
|
||||
# as the freshly merged nodes and would DELETE the re-extracted content (#1178 is moot
|
||||
# now that replace — not the dedup pass — reconciles changed files).
|
||||
prune = list(deleted) or None
|
||||
|
||||
# Use build_merge() — reads graph.json directly without NetworkX round-trip
|
||||
# so edge direction (calls, implements, imports) is always preserved (#801).
|
||||
|
||||
@@ -61,7 +61,9 @@ confidence_score is REQUIRED on every edge - never omit it, never use 0.5 as a d
|
||||
Node ID format: lowercase, only `[a-z0-9_]`, no dots or slashes. Format: `{stem}_{entity}` where stem is `{parent_dir}_{filename_without_ext}` (the **immediate** parent directory name + the filename stem, both lowercased with non-alphanumeric chars replaced by `_`) and entity is the symbol name similarly normalized. Only one level of parent is used — not the full path. Examples: `src/auth/session.py` + `ValidateToken` → `auth_session_validatetoken`; `lib/utils/helpers.py` + `parse_url` → `utils_helpers_parse_url`; `tests/test_foo.py` + `_helper` → `tests_test_foo_helper`. Top-level files (no parent dir, e.g. `setup.py`) use just the filename stem: `setup_my_func`. This must match the ID the AST extractor generates — using just the filename (e.g., `session_validatetoken`) or the full path (e.g., `src_auth_session_validatetoken`) will create orphan ghost-duplicate nodes. If you are re-extracting a project that had ghost duplicates under the old format, the user should run `graphify extract --force` to rebuild cleanly. CRITICAL: never append chunk numbers, sequence numbers, or any suffix to an ID (no `_c1`, `_c2`, `_chunk2`, etc.). IDs must be deterministic from the label alone — the same entity must always produce the same ID regardless of which chunk processes it.
|
||||
|
||||
Generate the extraction JSON matching this schema exactly:
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0}
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"<FILE_LIST path verbatim>","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"<FILE_LIST path verbatim>","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"<FILE_LIST path verbatim>"}],"input_tokens":0,"output_tokens":0}
|
||||
|
||||
source_file RULE (every node, edge, and hyperedge): set source_file to the path of the originating file EXACTLY as it appears in FILE_LIST — verbatim and absolute. Do NOT shorten to a basename, do NOT re-relativize, do NOT strip any directory prefix, and do NOT change separators (the engine canonicalizes separators and relativizes against the build root downstream). Copy the FILE_LIST entry character-for-character. This keeps the full build and incremental --update on the same base, so build_merge's replace-on-re-extract matches the existing node instead of accumulating a duplicate.
|
||||
|
||||
Then write the JSON to disk using the Write tool at this exact absolute path (no relative paths — Write resolves relative paths against an undefined cwd and the file will be silently lost):
|
||||
CHUNK_PATH
|
||||
|
||||
@@ -93,11 +93,13 @@ from graphify.detect import save_manifest
|
||||
new_extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
incremental = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\"))
|
||||
deleted = list(incremental.get('deleted_files', []))
|
||||
# Also prune old nodes for re-extracted (changed) files before inserting fresh AST.
|
||||
# Without this, build_merge's dedup pass tries to reconcile old and new versions of
|
||||
# the same file's nodes and can collapse same-named symbols across files (#1178).
|
||||
changed = [f for files in incremental.get('new_files', {}).values() for f in files]
|
||||
prune = list(dict.fromkeys(deleted + changed)) or None
|
||||
# prune_sources is ONLY for genuinely DELETED files. Changed/re-extracted files are
|
||||
# handled by build_merge's replace-on-re-extract (#1344): every source_file in
|
||||
# new_chunks is dropped from the base before merge, so old/stale nodes don't survive.
|
||||
# Do NOT add `changed` here: with root= passed, prune_set relativizes to the same base
|
||||
# as the freshly merged nodes and would DELETE the re-extracted content (#1178 is moot
|
||||
# now that replace — not the dedup pass — reconciles changed files).
|
||||
prune = list(deleted) or None
|
||||
|
||||
# Use build_merge() — reads graph.json directly without NetworkX round-trip
|
||||
# so edge direction (calls, implements, imports) is always preserved (#801).
|
||||
|
||||
@@ -25,5 +25,7 @@ Rules:
|
||||
Node ID format: lowercase, only `[a-z0-9_]`, no dots or slashes. Format `{stem}_{entity}` where stem is `{parent_dir}_{filename_without_ext}` (the immediate parent directory + the filename stem, both lowercased with non-alphanumeric chars replaced by `_`) and entity is the symbol name similarly normalized. Only one level of parent. `src/auth/session.py` + `ValidateToken` → `auth_session_validatetoken`. Top-level files use just the filename stem. This must match the AST extractor's ID. Never append chunk or sequence suffixes — IDs must be deterministic from the label alone.
|
||||
|
||||
Output exactly this JSON (no other text):
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0}
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"<FILE_LIST path verbatim>","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"<FILE_LIST path verbatim>","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"<FILE_LIST path verbatim>"}],"input_tokens":0,"output_tokens":0}
|
||||
|
||||
source_file RULE: set source_file to the FILE_LIST path for that file VERBATIM (absolute, no shortening to basename, no re-relativizing, no separator change). Keeps full build and --update on one base so build_merge's replace matches instead of duplicating.
|
||||
```
|
||||
|
||||
@@ -93,11 +93,13 @@ from graphify.detect import save_manifest
|
||||
new_extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
incremental = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\"))
|
||||
deleted = list(incremental.get('deleted_files', []))
|
||||
# Also prune old nodes for re-extracted (changed) files before inserting fresh AST.
|
||||
# Without this, build_merge's dedup pass tries to reconcile old and new versions of
|
||||
# the same file's nodes and can collapse same-named symbols across files (#1178).
|
||||
changed = [f for files in incremental.get('new_files', {}).values() for f in files]
|
||||
prune = list(dict.fromkeys(deleted + changed)) or None
|
||||
# prune_sources is ONLY for genuinely DELETED files. Changed/re-extracted files are
|
||||
# handled by build_merge's replace-on-re-extract (#1344): every source_file in
|
||||
# new_chunks is dropped from the base before merge, so old/stale nodes don't survive.
|
||||
# Do NOT add `changed` here: with root= passed, prune_set relativizes to the same base
|
||||
# as the freshly merged nodes and would DELETE the re-extracted content (#1178 is moot
|
||||
# now that replace — not the dedup pass — reconciles changed files).
|
||||
prune = list(deleted) or None
|
||||
|
||||
# Use build_merge() — reads graph.json directly without NetworkX round-trip
|
||||
# so edge direction (calls, implements, imports) is always preserved (#801).
|
||||
|
||||
@@ -61,7 +61,9 @@ confidence_score is REQUIRED on every edge - never omit it, never use 0.5 as a d
|
||||
Node ID format: lowercase, only `[a-z0-9_]`, no dots or slashes. Format: `{stem}_{entity}` where stem is `{parent_dir}_{filename_without_ext}` (the **immediate** parent directory name + the filename stem, both lowercased with non-alphanumeric chars replaced by `_`) and entity is the symbol name similarly normalized. Only one level of parent is used — not the full path. Examples: `src/auth/session.py` + `ValidateToken` → `auth_session_validatetoken`; `lib/utils/helpers.py` + `parse_url` → `utils_helpers_parse_url`; `tests/test_foo.py` + `_helper` → `tests_test_foo_helper`. Top-level files (no parent dir, e.g. `setup.py`) use just the filename stem: `setup_my_func`. This must match the ID the AST extractor generates — using just the filename (e.g., `session_validatetoken`) or the full path (e.g., `src_auth_session_validatetoken`) will create orphan ghost-duplicate nodes. If you are re-extracting a project that had ghost duplicates under the old format, the user should run `graphify extract --force` to rebuild cleanly. CRITICAL: never append chunk numbers, sequence numbers, or any suffix to an ID (no `_c1`, `_c2`, `_chunk2`, etc.). IDs must be deterministic from the label alone — the same entity must always produce the same ID regardless of which chunk processes it.
|
||||
|
||||
Generate the extraction JSON matching this schema exactly:
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0}
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"<FILE_LIST path verbatim>","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"<FILE_LIST path verbatim>","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"<FILE_LIST path verbatim>"}],"input_tokens":0,"output_tokens":0}
|
||||
|
||||
source_file RULE (every node, edge, and hyperedge): set source_file to the path of the originating file EXACTLY as it appears in FILE_LIST — verbatim and absolute. Do NOT shorten to a basename, do NOT re-relativize, do NOT strip any directory prefix, and do NOT change separators (the engine canonicalizes separators and relativizes against the build root downstream). Copy the FILE_LIST entry character-for-character. This keeps the full build and incremental --update on the same base, so build_merge's replace-on-re-extract matches the existing node instead of accumulating a duplicate.
|
||||
|
||||
Then write the JSON to disk using the Write tool at this exact absolute path (no relative paths — Write resolves relative paths against an undefined cwd and the file will be silently lost):
|
||||
CHUNK_PATH
|
||||
|
||||
@@ -93,11 +93,13 @@ from graphify.detect import save_manifest
|
||||
new_extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
incremental = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\"))
|
||||
deleted = list(incremental.get('deleted_files', []))
|
||||
# Also prune old nodes for re-extracted (changed) files before inserting fresh AST.
|
||||
# Without this, build_merge's dedup pass tries to reconcile old and new versions of
|
||||
# the same file's nodes and can collapse same-named symbols across files (#1178).
|
||||
changed = [f for files in incremental.get('new_files', {}).values() for f in files]
|
||||
prune = list(dict.fromkeys(deleted + changed)) or None
|
||||
# prune_sources is ONLY for genuinely DELETED files. Changed/re-extracted files are
|
||||
# handled by build_merge's replace-on-re-extract (#1344): every source_file in
|
||||
# new_chunks is dropped from the base before merge, so old/stale nodes don't survive.
|
||||
# Do NOT add `changed` here: with root= passed, prune_set relativizes to the same base
|
||||
# as the freshly merged nodes and would DELETE the re-extracted content (#1178 is moot
|
||||
# now that replace — not the dedup pass — reconciles changed files).
|
||||
prune = list(deleted) or None
|
||||
|
||||
# Use build_merge() — reads graph.json directly without NetworkX round-trip
|
||||
# so edge direction (calls, implements, imports) is always preserved (#801).
|
||||
|
||||
@@ -61,7 +61,9 @@ confidence_score is REQUIRED on every edge - never omit it, never use 0.5 as a d
|
||||
Node ID format: lowercase, only `[a-z0-9_]`, no dots or slashes. Format: `{stem}_{entity}` where stem is `{parent_dir}_{filename_without_ext}` (the **immediate** parent directory name + the filename stem, both lowercased with non-alphanumeric chars replaced by `_`) and entity is the symbol name similarly normalized. Only one level of parent is used — not the full path. Examples: `src/auth/session.py` + `ValidateToken` → `auth_session_validatetoken`; `lib/utils/helpers.py` + `parse_url` → `utils_helpers_parse_url`; `tests/test_foo.py` + `_helper` → `tests_test_foo_helper`. Top-level files (no parent dir, e.g. `setup.py`) use just the filename stem: `setup_my_func`. This must match the ID the AST extractor generates — using just the filename (e.g., `session_validatetoken`) or the full path (e.g., `src_auth_session_validatetoken`) will create orphan ghost-duplicate nodes. If you are re-extracting a project that had ghost duplicates under the old format, the user should run `graphify extract --force` to rebuild cleanly. CRITICAL: never append chunk numbers, sequence numbers, or any suffix to an ID (no `_c1`, `_c2`, `_chunk2`, etc.). IDs must be deterministic from the label alone — the same entity must always produce the same ID regardless of which chunk processes it.
|
||||
|
||||
Generate the extraction JSON matching this schema exactly:
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0}
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"<FILE_LIST path verbatim>","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"<FILE_LIST path verbatim>","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"<FILE_LIST path verbatim>"}],"input_tokens":0,"output_tokens":0}
|
||||
|
||||
source_file RULE (every node, edge, and hyperedge): set source_file to the path of the originating file EXACTLY as it appears in FILE_LIST — verbatim and absolute. Do NOT shorten to a basename, do NOT re-relativize, do NOT strip any directory prefix, and do NOT change separators (the engine canonicalizes separators and relativizes against the build root downstream). Copy the FILE_LIST entry character-for-character. This keeps the full build and incremental --update on the same base, so build_merge's replace-on-re-extract matches the existing node instead of accumulating a duplicate.
|
||||
|
||||
Then write the JSON to disk using the Write tool at this exact absolute path (no relative paths — Write resolves relative paths against an undefined cwd and the file will be silently lost):
|
||||
CHUNK_PATH
|
||||
|
||||
@@ -93,11 +93,13 @@ from graphify.detect import save_manifest
|
||||
new_extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
incremental = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\"))
|
||||
deleted = list(incremental.get('deleted_files', []))
|
||||
# Also prune old nodes for re-extracted (changed) files before inserting fresh AST.
|
||||
# Without this, build_merge's dedup pass tries to reconcile old and new versions of
|
||||
# the same file's nodes and can collapse same-named symbols across files (#1178).
|
||||
changed = [f for files in incremental.get('new_files', {}).values() for f in files]
|
||||
prune = list(dict.fromkeys(deleted + changed)) or None
|
||||
# prune_sources is ONLY for genuinely DELETED files. Changed/re-extracted files are
|
||||
# handled by build_merge's replace-on-re-extract (#1344): every source_file in
|
||||
# new_chunks is dropped from the base before merge, so old/stale nodes don't survive.
|
||||
# Do NOT add `changed` here: with root= passed, prune_set relativizes to the same base
|
||||
# as the freshly merged nodes and would DELETE the re-extracted content (#1178 is moot
|
||||
# now that replace — not the dedup pass — reconciles changed files).
|
||||
prune = list(deleted) or None
|
||||
|
||||
# Use build_merge() — reads graph.json directly without NetworkX round-trip
|
||||
# so edge direction (calls, implements, imports) is always preserved (#801).
|
||||
|
||||
@@ -61,7 +61,9 @@ confidence_score is REQUIRED on every edge - never omit it, never use 0.5 as a d
|
||||
Node ID format: lowercase, only `[a-z0-9_]`, no dots or slashes. Format: `{stem}_{entity}` where stem is `{parent_dir}_{filename_without_ext}` (the **immediate** parent directory name + the filename stem, both lowercased with non-alphanumeric chars replaced by `_`) and entity is the symbol name similarly normalized. Only one level of parent is used — not the full path. Examples: `src/auth/session.py` + `ValidateToken` → `auth_session_validatetoken`; `lib/utils/helpers.py` + `parse_url` → `utils_helpers_parse_url`; `tests/test_foo.py` + `_helper` → `tests_test_foo_helper`. Top-level files (no parent dir, e.g. `setup.py`) use just the filename stem: `setup_my_func`. This must match the ID the AST extractor generates — using just the filename (e.g., `session_validatetoken`) or the full path (e.g., `src_auth_session_validatetoken`) will create orphan ghost-duplicate nodes. If you are re-extracting a project that had ghost duplicates under the old format, the user should run `graphify extract --force` to rebuild cleanly. CRITICAL: never append chunk numbers, sequence numbers, or any suffix to an ID (no `_c1`, `_c2`, `_chunk2`, etc.). IDs must be deterministic from the label alone — the same entity must always produce the same ID regardless of which chunk processes it.
|
||||
|
||||
Generate the extraction JSON matching this schema exactly:
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0}
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"<FILE_LIST path verbatim>","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"<FILE_LIST path verbatim>","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"<FILE_LIST path verbatim>"}],"input_tokens":0,"output_tokens":0}
|
||||
|
||||
source_file RULE (every node, edge, and hyperedge): set source_file to the path of the originating file EXACTLY as it appears in FILE_LIST — verbatim and absolute. Do NOT shorten to a basename, do NOT re-relativize, do NOT strip any directory prefix, and do NOT change separators (the engine canonicalizes separators and relativizes against the build root downstream). Copy the FILE_LIST entry character-for-character. This keeps the full build and incremental --update on the same base, so build_merge's replace-on-re-extract matches the existing node instead of accumulating a duplicate.
|
||||
|
||||
Then write the JSON to disk using the Write tool at this exact absolute path (no relative paths — Write resolves relative paths against an undefined cwd and the file will be silently lost):
|
||||
CHUNK_PATH
|
||||
|
||||
@@ -93,11 +93,13 @@ from graphify.detect import save_manifest
|
||||
new_extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
incremental = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\"))
|
||||
deleted = list(incremental.get('deleted_files', []))
|
||||
# Also prune old nodes for re-extracted (changed) files before inserting fresh AST.
|
||||
# Without this, build_merge's dedup pass tries to reconcile old and new versions of
|
||||
# the same file's nodes and can collapse same-named symbols across files (#1178).
|
||||
changed = [f for files in incremental.get('new_files', {}).values() for f in files]
|
||||
prune = list(dict.fromkeys(deleted + changed)) or None
|
||||
# prune_sources is ONLY for genuinely DELETED files. Changed/re-extracted files are
|
||||
# handled by build_merge's replace-on-re-extract (#1344): every source_file in
|
||||
# new_chunks is dropped from the base before merge, so old/stale nodes don't survive.
|
||||
# Do NOT add `changed` here: with root= passed, prune_set relativizes to the same base
|
||||
# as the freshly merged nodes and would DELETE the re-extracted content (#1178 is moot
|
||||
# now that replace — not the dedup pass — reconciles changed files).
|
||||
prune = list(deleted) or None
|
||||
|
||||
# Use build_merge() — reads graph.json directly without NetworkX round-trip
|
||||
# so edge direction (calls, implements, imports) is always preserved (#801).
|
||||
|
||||
@@ -320,7 +320,9 @@ from pathlib import Path
|
||||
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= mirrors the --update runbook (#1361): relativize source_file to the same
|
||||
# base so the full build and incremental --update never drift apart on re-extract.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = cluster(G)
|
||||
cohesion = score_all(G, communities)
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
@@ -373,7 +375,8 @@ extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(en
|
||||
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\"))
|
||||
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding=\"utf-8\"))
|
||||
|
||||
G = build_from_json(extraction)
|
||||
# root= as in Step 4 / the --update runbook (#1361) — same base for node-key parity.
|
||||
G = build_from_json(extraction, root='INPUT_PATH')
|
||||
communities = {int(k): v for k, v in analysis['communities'].items()}
|
||||
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
|
||||
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
|
||||
|
||||
@@ -25,5 +25,7 @@ Rules:
|
||||
Node ID format: lowercase, only `[a-z0-9_]`, no dots or slashes. Format `{stem}_{entity}` where stem is `{parent_dir}_{filename_without_ext}` (the immediate parent directory + the filename stem, both lowercased with non-alphanumeric chars replaced by `_`) and entity is the symbol name similarly normalized. Only one level of parent. `src/auth/session.py` + `ValidateToken` → `auth_session_validatetoken`. Top-level files use just the filename stem. This must match the AST extractor's ID. Never append chunk or sequence suffixes — IDs must be deterministic from the label alone.
|
||||
|
||||
Output exactly this JSON (no other text):
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0}
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"<FILE_LIST path verbatim>","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"<FILE_LIST path verbatim>","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"<FILE_LIST path verbatim>"}],"input_tokens":0,"output_tokens":0}
|
||||
|
||||
source_file RULE: set source_file to the FILE_LIST path for that file VERBATIM (absolute, no shortening to basename, no re-relativizing, no separator change). Keeps full build and --update on one base so build_merge's replace matches instead of duplicating.
|
||||
```
|
||||
|
||||
@@ -61,7 +61,9 @@ confidence_score is REQUIRED on every edge - never omit it, never use 0.5 as a d
|
||||
Node ID format: lowercase, only `[a-z0-9_]`, no dots or slashes. Format: `{stem}_{entity}` where stem is `{parent_dir}_{filename_without_ext}` (the **immediate** parent directory name + the filename stem, both lowercased with non-alphanumeric chars replaced by `_`) and entity is the symbol name similarly normalized. Only one level of parent is used — not the full path. Examples: `src/auth/session.py` + `ValidateToken` → `auth_session_validatetoken`; `lib/utils/helpers.py` + `parse_url` → `utils_helpers_parse_url`; `tests/test_foo.py` + `_helper` → `tests_test_foo_helper`. Top-level files (no parent dir, e.g. `setup.py`) use just the filename stem: `setup_my_func`. This must match the ID the AST extractor generates — using just the filename (e.g., `session_validatetoken`) or the full path (e.g., `src_auth_session_validatetoken`) will create orphan ghost-duplicate nodes. If you are re-extracting a project that had ghost duplicates under the old format, the user should run `graphify extract --force` to rebuild cleanly. CRITICAL: never append chunk numbers, sequence numbers, or any suffix to an ID (no `_c1`, `_c2`, `_chunk2`, etc.). IDs must be deterministic from the label alone — the same entity must always produce the same ID regardless of which chunk processes it.
|
||||
|
||||
Generate the extraction JSON matching this schema exactly:
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0}
|
||||
{"nodes":[{"id":"session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"<FILE_LIST path verbatim>","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"<FILE_LIST path verbatim>","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"<FILE_LIST path verbatim>"}],"input_tokens":0,"output_tokens":0}
|
||||
|
||||
source_file RULE (every node, edge, and hyperedge): set source_file to the path of the originating file EXACTLY as it appears in FILE_LIST — verbatim and absolute. Do NOT shorten to a basename, do NOT re-relativize, do NOT strip any directory prefix, and do NOT change separators (the engine canonicalizes separators and relativizes against the build root downstream). Copy the FILE_LIST entry character-for-character. This keeps the full build and incremental --update on the same base, so build_merge's replace-on-re-extract matches the existing node instead of accumulating a duplicate.
|
||||
|
||||
Then write the JSON to disk using the Write tool at this exact absolute path (no relative paths — Write resolves relative paths against an undefined cwd and the file will be silently lost):
|
||||
CHUNK_PATH
|
||||
|
||||
@@ -93,11 +93,13 @@ from graphify.detect import save_manifest
|
||||
new_extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\"))
|
||||
incremental = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\"))
|
||||
deleted = list(incremental.get('deleted_files', []))
|
||||
# Also prune old nodes for re-extracted (changed) files before inserting fresh AST.
|
||||
# Without this, build_merge's dedup pass tries to reconcile old and new versions of
|
||||
# the same file's nodes and can collapse same-named symbols across files (#1178).
|
||||
changed = [f for files in incremental.get('new_files', {}).values() for f in files]
|
||||
prune = list(dict.fromkeys(deleted + changed)) or None
|
||||
# prune_sources is ONLY for genuinely DELETED files. Changed/re-extracted files are
|
||||
# handled by build_merge's replace-on-re-extract (#1344): every source_file in
|
||||
# new_chunks is dropped from the base before merge, so old/stale nodes don't survive.
|
||||
# Do NOT add `changed` here: with root= passed, prune_set relativizes to the same base
|
||||
# as the freshly merged nodes and would DELETE the re-extracted content (#1178 is moot
|
||||
# now that replace — not the dedup pass — reconciles changed files).
|
||||
prune = list(deleted) or None
|
||||
|
||||
# Use build_merge() — reads graph.json directly without NetworkX round-trip
|
||||
# so edge direction (calls, implements, imports) is always preserved (#801).
|
||||
|
||||
Reference in New Issue
Block a user