diff --git a/CHANGELOG.md b/CHANGELOG.md index cd371ff..446ed9e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,11 @@ Full release notes with details on each version: [GitHub Releases](https://github.com/safishamsi/graphify/releases) +## 0.9.18 (unreleased) + +- Fix: the semantic cache no longer replays extractions from an older prompt after an upgrade (#1939, thanks @HunterMcGrew and @SinghAman21). Entries were keyed on `sha256(file content + path)` alone, with no component for the extraction prompt that produced them, so a release that changed the prompt left every unchanged file a cache hit: the run exited 0, `cost.json` looked cheap, and the graph silently carried two prompt generations side by side. Semantic entries are now namespaced by a fingerprint of the extraction prompt (`cache/semantic/p{fingerprint}/`, mirroring the AST cache's `v{version}/` layout), keeping both properties #1252 wanted — entries survive releases that don't touch the prompt, and invalidate only when it actually changed. The fingerprint normalizes line endings so a CRLF checkout doesn't look like a prompt change. Both extraction paths pass their prompt: the Python/CLI path (`llm.py`'s `_EXTRACTION_SYSTEM`, all backends) automatically, and the skill path via a new `prompt_file` argument in Step B0/B3 pointing at the `references/extraction-spec.md` the subagents were handed. Pre-existing entries predate fingerprinting and have unknowable vintage: they are still served rather than re-billing a whole corpus, but `check_semantic_cache` now warns with the count, so the "no signal at all" the report describes becomes a visible one; `--force` (or `GRAPHIFY_FORCE=1`) re-extracts them. Old-fingerprint entries are pruned by liveness only, never swept wholesale the way stale AST versions are — two hosts with different prompts can share one `graphify-out/`, and a wholesale sweep would have each run delete the other's entries. (The two monolith skills, aider and devin, inline their prompt instead of shipping a spec sidecar and stay on the unfingerprinted path for now.) +- Fix: PostgreSQL foreign-key `references` edges are no longer dropped when a routine in the same schema is unparseable (#1854, thanks @sekmur). `pg_introspect` builds one synthetic DDL document and parsed it with the function stubs emitted before the FK `ALTER TABLE`s, so a C-language (or otherwise unparseable) routine's stub parsed as a tree-sitter ERROR node that swallowed the trailing FK statements into the error region, losing every FK edge after it. The FK DDL is now emitted before the function stubs, so table-to-table `references` edges are produced first and can't be eaten by a later unparseable routine. + ## 0.9.17 (2026-07-16) - Acknowledgement: Amp (ampcode.com) platform support, which shipped earlier in v8, was contributed by @zuwasi in #948 (the `skill-amp.md` skill and the `graphify amp install` wiring). Belated credit for the work. diff --git a/graphify/cache.py b/graphify/cache.py index 15795d4..7122b5c 100644 --- a/graphify/cache.py +++ b/graphify/cache.py @@ -63,6 +63,99 @@ def _cleanup_stale_ast_entries(ast_base: Path, current_dir: Path) -> None: pass +# Semantic cache entries are LLM output, so they depend on the extraction prompt +# that produced them, not just on file contents. Keying purely on content means a +# release that changes the prompt keeps replaying entries from the older prompt on +# every unchanged file, silently mixing extraction vintages in one graph (#1939). +# Versioning them by package version (as the AST cache does) would re-bill LLM +# extraction on every patch release — the reason #1252 deliberately left them +# unversioned. Fingerprinting the prompt itself keeps both properties: entries +# survive releases that don't touch the prompt, and invalidate only when it +# actually changed. Entries live under cache/semantic/p{fingerprint}/ when the +# caller supplies its prompt; callers that don't keep the historical flat layout. +_PROMPT_FP_LEN = 12 + +# Count of pre-fingerprint (flat-layout) entries served this process, so +# check_semantic_cache can report N to the user (#1939). +_legacy_semantic_hits = 0 + +# Prompt-file fingerprints already computed, keyed by (path, size, mtime_ns) — +# the same stat signature the hash index uses. check_semantic_cache resolves the +# prompt once per FILE in the corpus, so without this a 500-doc run re-reads and +# re-hashes the same spec 500 times (and warns 500 times when it is unreadable). +_prompt_fp_cache: dict[tuple, str] = {} + + +def prompt_fingerprint(prompt: "str | Path") -> str: + """Return a short stable fingerprint of an extraction prompt. + + ``prompt`` is either the prompt text itself (the Python extraction path owns + its system prompt, :func:`graphify.llm._extraction_system`) or a Path to the + prompt file an agent loaded (the skill path's + ``references/extraction-spec.md``). + + Line endings and trailing whitespace are normalized before hashing: the same + spec file checked out with CRLF on Windows must not fingerprint differently + from the LF checkout that wrote the cache, or every Windows run would look + like a prompt change and re-bill extraction. + """ + if isinstance(prompt, Path): + text = prompt.read_text(encoding="utf-8", errors="replace") + else: + text = prompt + normalized = "\n".join( + line.rstrip() for line in text.replace("\r\n", "\n").replace("\r", "\n").split("\n") + ).strip() + return hashlib.sha256(normalized.encode()).hexdigest()[:_PROMPT_FP_LEN] + + +def _resolve_prompt_fp(prompt: "str | Path | None" = None, + prompt_file: "str | Path | None" = None) -> str | None: + """Fingerprint the caller's extraction prompt, or None when it supplied none. + + ``prompt`` is prompt TEXT; ``prompt_file`` is a path to a file CONTAINING the + prompt. They are separate parameters rather than one overloaded argument + because the skill-driven callers are markdown snippets an agent copies with a + path substituted in — passing that path as ``prompt`` would hash the path + string itself, yielding a fingerprint that is stable, plausible, and tracks + nothing about the prompt. A silent wrong fingerprint is the exact failure + class #1939 is about, so the two are not inferred from each other. + + Best-effort: an unreadable ``prompt_file`` falls back to the flat, unattributed + layout rather than failing the run — a cache is never worth aborting an + extraction over. It warns rather than falling back quietly, because that + fallback silently restores the very behavior this fixes, and the skill-side + caller substitutes this path by hand. + """ + memo_key = None + if prompt_file is not None: + prompt = Path(prompt_file) + try: + st = prompt.stat() + memo_key = (str(prompt), st.st_size, st.st_mtime_ns) + if memo_key in _prompt_fp_cache: + return _prompt_fp_cache[memo_key] + except OSError: + pass # unreadable — fall through to the warning below + if prompt is None: + return None + try: + fp = prompt_fingerprint(prompt) + if memo_key is not None: + _prompt_fp_cache[memo_key] = fp + return fp + except (OSError, UnicodeError) as exc: + warnings.warn( + f"could not read extraction prompt {str(prompt)!r} ({exc}); semantic cache " + "entries cannot be attributed to a prompt version and fall back to the " + "unversioned layout, so this run may replay entries from an older " + "extraction prompt (#1939).", + RuntimeWarning, + stacklevel=3, + ) + return None + + # A frontmatter delimiter is a whole line of exactly three dashes (optional # trailing whitespace). Substring checks like startswith("---") / # find("\n---") also match `----` thematic breaks and `--- text` prose, @@ -338,7 +431,8 @@ def _absolutize_source_files_in(payload: dict, root: Path) -> None: continue -def cache_dir(root: Path = Path("."), kind: str = "ast") -> Path: +def cache_dir(root: Path = Path("."), kind: str = "ast", + prompt_fp: str | None = None) -> Path: """Returns the cache directory for ``kind`` - creates it if needed. kind is "ast", "semantic", or a mode-namespaced semantic kind such as @@ -347,9 +441,14 @@ def cache_dir(root: Path = Path("."), kind: str = "ast") -> Path: AST entries live in graphify-out/cache/ast/v{version}/ — namespaced by graphify version because they depend on extractor code, not just file - contents. Semantic entries live unversioned in graphify-out/cache/semantic/ - (re-extraction costs LLM calls); deep-mode entries live beside them in - graphify-out/cache/semantic-deep/. + contents. Semantic entries are still NOT version-namespaced (re-extraction + costs LLM calls, #1252): they live in graphify-out/cache/semantic/, with + deep-mode entries beside them in graphify-out/cache/semantic-deep/. + + ``prompt_fp`` (semantic kinds only) adds a p{fingerprint}/ subdirectory so + entries are attributed to the extraction prompt that produced them (#1939). + Omitting it yields the historical flat layout, where entries of unknown + vintage live. """ _out = Path(_GRAPHIFY_OUT) base = _out if _out.is_absolute() else Path(root).resolve() / _out @@ -357,12 +456,16 @@ def cache_dir(root: Path = Path("."), kind: str = "ast") -> Path: if kind == "ast": d = d / f"v{_EXTRACTOR_VERSION}" _cleanup_stale_ast_entries(d.parent, d) + elif prompt_fp: + d = d / f"p{prompt_fp}" d.mkdir(parents=True, exist_ok=True) return d def load_cached(path: Path, root: Path = Path("."), kind: str = "ast", - cache_root: Path | None = None) -> dict | None: + cache_root: Path | None = None, prompt: "str | Path | None" = None, + prompt_file: "str | Path | None" = None, + allow_legacy: bool = True) -> dict | None: """Return cached extraction for this file if hash matches, else None. Cache key: SHA256 of file contents. @@ -380,19 +483,39 @@ def load_cached(path: Path, root: Path = Path("."), kind: str = "ast", flat cache/ layout (pre-0.5.3) and the unversioned cache/ast/ layout — are deliberately not consulted: they were produced by a different extractor and may be stale. + + ``prompt`` (semantic kinds) is the extraction prompt — text, or a Path to + the prompt file — that the caller is about to extract with. It selects the + p{fingerprint}/ namespace, so an entry produced by a different prompt is a + miss rather than a silent stale hit (#1939). When it is given and the + fingerprinted namespace misses, ``allow_legacy`` (default True) falls back + to a flat-layout entry: those predate fingerprinting, so their vintage is + unknowable — they are served rather than re-billed, and the hit is counted + so :func:`check_semantic_cache` can report N to the user. Callers that must + not mix vintages within one entry (see :func:`save_semantic_cache`'s + ``merge_existing``) pass allow_legacy=False. Returns None if no cache entry or file has changed. """ + global _legacy_semantic_hits location = cache_root if cache_root is not None else root try: h = file_hash(path, root, cache_root=cache_root) except OSError: return None - entry = cache_dir(location, kind) / f"{h}.json" + prompt_fp = _resolve_prompt_fp(prompt, prompt_file) + entry = cache_dir(location, kind, prompt_fp) / f"{h}.json" + legacy_hit = False + if prompt_fp and not entry.exists() and allow_legacy: + legacy = cache_dir(location, kind) / f"{h}.json" + if legacy.exists(): + entry, legacy_hit = legacy, True if entry.exists(): try: result = json.loads(entry.read_text(encoding="utf-8")) except (json.JSONDecodeError, OSError): return None + if legacy_hit: + _legacy_semantic_hits += 1 # Re-anchor relative source_file fields so callers see the same # absolute-path shape that a fresh in-process extraction produces # (#777). Legacy entries with absolute source_file pass through. @@ -403,7 +526,8 @@ def load_cached(path: Path, root: Path = Path("."), kind: str = "ast", def save_cached(path: Path, result: dict, root: Path = Path("."), kind: str = "ast", - cache_root: Path | None = None) -> None: + cache_root: Path | None = None, prompt: "str | Path | None" = None, + prompt_file: "str | Path | None" = None) -> None: """Save extraction result for this file. Stores as graphify-out/cache/{kind}/{hash}.json where hash = SHA256 of current file contents. @@ -413,6 +537,13 @@ def save_cached(path: Path, result: dict, root: Path = Path("."), kind: str = "a ``cache_root`` (when given) is where the cache directory is written, decoupled from ``root`` so the cache never lands inside the analyzed source tree (#1774). + ``prompt`` (semantic kinds) is the extraction prompt that produced ``result`` + — text, or a Path to the prompt file. It stamps the entry into the + p{fingerprint}/ namespace so a later run under a different prompt does not + replay it (#1939). Writes always land in the fingerprinted namespace when a + prompt is given: an entry of known vintage is never written back into the + flat unknown-vintage layout. + No-ops if `path` is not a regular file. Subagent-produced semantic fragments occasionally carry a directory path in `source_file`; skipping them prevents IsADirectoryError from aborting the whole batch. @@ -437,7 +568,7 @@ def save_cached(path: Path, result: dict, root: Path = Path("."), kind: str = "a _relativize_source_files_in(on_disk, root) h = file_hash(p, root, cache_root=cache_root) location = cache_root if cache_root is not None else root - target_dir = cache_dir(location, kind) + target_dir = cache_dir(location, kind, _resolve_prompt_fp(prompt, prompt_file)) entry = target_dir / f"{h}.json" fd, tmp_path = tempfile.mkstemp(dir=target_dir, prefix=f"{h}.", suffix=".tmp") try: @@ -470,13 +601,14 @@ def cached_files(root: Path = Path(".")) -> set[str]: # Legacy flat entries if base.is_dir(): hashes.update(p.stem for p in base.glob("*.json")) - # Namespaced entries (ast/ recursively, covering per-version subdirs; - # semantic-deep/ holds --mode deep entries, #1894) - for kind, pattern in (("ast", "**/*.json"), ("semantic", "*.json"), - ("semantic-deep", "*.json")): + # Namespaced entries, all globbed recursively: ast/ has per-version subdirs, + # semantic-deep/ holds --mode deep entries (#1894), and both semantic kinds + # have per-prompt-fingerprint subdirs alongside pre-fingerprint flat entries + # (#1939). + for kind in ("ast", "semantic", "semantic-deep"): d = base / kind if d.is_dir(): - hashes.update(p.stem for p in d.glob(pattern)) + hashes.update(p.stem for p in d.glob("**/*.json")) return hashes @@ -488,13 +620,13 @@ def clear_cache(root: Path = Path(".")) -> None: if base.is_dir(): for f in base.glob("*.json"): f.unlink() - # Namespaced entries (ast/ recursively, covering per-version subdirs; - # semantic-deep/ holds --mode deep entries, #1894) - for kind, pattern in (("ast", "**/*.json"), ("semantic", "*.json"), - ("semantic-deep", "*.json")): + # Namespaced entries, all globbed recursively: ast/ has per-version subdirs, + # semantic-deep/ holds --mode deep entries (#1894), and both semantic kinds + # have per-prompt-fingerprint subdirs (#1939). + for kind in ("ast", "semantic", "semantic-deep"): d = base / kind if d.is_dir(): - for f in d.glob(pattern): + for f in d.glob("**/*.json"): f.unlink() @@ -519,6 +651,16 @@ def prune_semantic_cache(root: Path, live_hashes: set[str]) -> int: touched (never ``cache/ast/**`` or anything else). The unversioned design is preserved: we prune by liveness, not by version. + The sweep recurses into the per-prompt-fingerprint subdirs (#1939) for the + same reason it covers the deep namespace: a glob that stopped at the top + level would leave every fingerprinted entry permanently unprunable. Entries + under a fingerprint other than the current one are pruned by liveness only, + never swept wholesale the way :func:`_cleanup_stale_ast_entries` sweeps old + AST versions — two hosts with different prompts (verbose vs compact + extraction-spec) can share one graphify-out/, and a wholesale sweep would + have each run delete the other's entries and re-bill extraction on every + alternation. Liveness keeps the total bounded by live docs × prompts seen. + Best-effort, mirroring :func:`_cleanup_stale_ast_entries`: each unlink is wrapped in ``try/except OSError`` and a failure is ignored. The worst-case failure mode is benign — a surviving orphan costs only one re-extraction of @@ -531,7 +673,7 @@ def prune_semantic_cache(root: Path, live_hashes: set[str]) -> int: semantic_dir = base / "cache" / kind if not semantic_dir.is_dir(): continue - for entry in semantic_dir.glob("*.json"): + for entry in semantic_dir.glob("**/*.json"): if entry.stem in live_hashes: continue try: @@ -546,6 +688,8 @@ def check_semantic_cache( files: list[str], root: Path = Path("."), mode: str | None = None, + prompt: "str | Path | None" = None, + prompt_file: "str | Path | None" = None, ) -> tuple[list[dict], list[dict], list[dict], list[str]]: """Check semantic extraction cache for a list of absolute file paths. @@ -558,18 +702,30 @@ def check_semantic_cache( are unaffected. A non-None mode (e.g. ``"deep"``) reads ``cache/semantic-{mode}/`` instead, so deep-mode results never shadow (or get shadowed by) standard-mode entries for the same content (#1894). + + ``prompt`` is the extraction prompt this run will use for the uncached + files — the prompt text (Python path) or a Path to the prompt file the + agent loaded (skill path, ``references/extraction-spec.md``). Supplying it + restricts hits to entries produced by that same prompt, so an upgrade that + changed the prompt re-extracts instead of replaying the older vintage + (#1939). Entries written before fingerprinting existed still hit — their + vintage is unknowable and dropping them would re-bill a whole corpus — but + a warning reports how many were served. Omitting ``prompt`` keeps the + historical behavior for existing callers. """ + global _legacy_semantic_hits kind = "semantic" if mode is None else f"semantic-{mode}" cached_nodes: list[dict] = [] cached_edges: list[dict] = [] cached_hyperedges: list[dict] = [] uncached: list[str] = [] + legacy_before = _legacy_semantic_hits for fpath in files: p = Path(fpath) if not p.is_absolute(): p = Path(root) / p - result = load_cached(p, root, kind=kind) + result = load_cached(p, root, kind=kind, prompt=prompt, prompt_file=prompt_file) if result is not None: cached_nodes.extend(result.get("nodes", [])) cached_edges.extend(result.get("edges", [])) @@ -577,6 +733,18 @@ def check_semantic_cache( else: uncached.append(fpath) + legacy = _legacy_semantic_hits - legacy_before + if legacy: + warnings.warn( + f"{legacy} semantic cache entr{'y' if legacy == 1 else 'ies'} predate " + "extraction-prompt fingerprinting and were written by an unknown prompt " + "version; they were replayed as-is, so this graph may mix extraction " + "vintages. Re-run with --force (or GRAPHIFY_FORCE=1) to re-extract them " + "with the current prompt (#1939).", + RuntimeWarning, + stacklevel=2, + ) + return cached_nodes, cached_edges, cached_hyperedges, uncached @@ -588,6 +756,8 @@ def save_semantic_cache( merge_existing: bool = False, allowed_source_files: Iterable[str | Path] | None = None, mode: str | None = None, + prompt: "str | Path | None" = None, + prompt_file: "str | Path | None" = None, ) -> int: """Save semantic extraction results to cache, keyed by source_file. @@ -611,6 +781,13 @@ def save_semantic_cache( cache-write keys. Semantic nodes can legitimately mention another corpus file, but a model must not be able to replace that file's complete cache entry unless the file was part of the current extraction batch (#1757). + + ``prompt`` is the extraction prompt that produced these results — text, or + a Path to the prompt file. It stamps entries into the p{fingerprint}/ + namespace so a later run under a different prompt re-extracts rather than + replaying them (#1939). Pass the same prompt here as to + :func:`check_semantic_cache`, or the write lands in a namespace the next + read won't consult. Returns the number of files cached. """ from collections import defaultdict @@ -716,13 +893,19 @@ def save_semantic_cache( ) continue if merge_existing: - prev = load_cached(p, root, kind=kind) + # allow_legacy=False: merging a pre-fingerprint entry into this + # write would fuse two prompt vintages inside a single entry and + # then stamp the result as current-vintage — the exact mixing + # #1939 is about, made unfixable because the entry now claims a + # prompt that only produced half of it. + prev = load_cached(p, root, kind=kind, prompt=prompt, + prompt_file=prompt_file, allow_legacy=False) if prev: result = { "nodes": (prev.get("nodes", []) or []) + result["nodes"], "edges": (prev.get("edges", []) or []) + result["edges"], "hyperedges": (prev.get("hyperedges", []) or []) + result["hyperedges"], } - save_cached(p, result, root, kind=kind) + save_cached(p, result, root, kind=kind, prompt=prompt, prompt_file=prompt_file) saved += 1 return saved diff --git a/graphify/cli.py b/graphify/cli.py index d817401..2700f68 100644 --- a/graphify/cli.py +++ b/graphify/cli.py @@ -2553,6 +2553,12 @@ def dispatch_command(cmd: str) -> None: # Deep mode uses its own namespace (cache/semantic-deep/) so deep and # standard results for the same content never shadow each other (#1894). sem_cache_mode = "deep" if deep_mode else None + # Entries are attributed to the extraction prompt that produced them, so + # a release that changes the prompt re-extracts rather than replaying the + # older vintage alongside the new one (#1939). Read and write must pass + # the same prompt, or the write lands where the next read won't look. + from graphify.llm import _extraction_system as _sem_prompt_for + sem_prompt = _sem_prompt_for(deep=deep_mode) if semantic_files: sem_paths_str = [str(p) for p in semantic_files] if force: @@ -2563,7 +2569,8 @@ def dispatch_command(cmd: str) -> None: uncached_paths = list(sem_paths_str) else: cached_nodes, cached_edges, cached_hyperedges, uncached_paths = ( - _check_semantic_cache(sem_paths_str, root=out_root, mode=sem_cache_mode) + _check_semantic_cache(sem_paths_str, root=out_root, mode=sem_cache_mode, + prompt=sem_prompt) ) sem_cache_hits = len(semantic_files) - len(uncached_paths) sem_cache_misses = len(uncached_paths) @@ -2636,6 +2643,7 @@ def dispatch_command(cmd: str) -> None: root=out_root, allowed_source_files=uncached_paths, mode=sem_cache_mode, + prompt=sem_prompt, ) except Exception as exc: print(f"[graphify extract] warning: could not write semantic cache: {exc}", file=sys.stderr) @@ -2947,20 +2955,27 @@ def dispatch_command(cmd: str) -> None: elif cmd == "cache-check": # graphify cache-check [--root ] [--mode | --deep] + # [--prompt-file ] # Reads file paths (one per line) from , checks semantic cache. # --mode deep (or --deep) checks the cache/semantic-deep/ namespace # written by `extract --mode deep` instead of cache/semantic/ (#1894). + # --prompt-file names the extraction prompt the caller will use (an agent's + # references/extraction-spec.md), restricting hits to entries produced by + # that same prompt (#1939). Omitting it reads the unattributed layout, which + # cannot see entries a fingerprinted run wrote. # Writes: # graphify-out/.graphify_cached.json — already-cached nodes/edges/hyperedges # graphify-out/.graphify_uncached.txt — paths that need extraction # Stdout: "Cache: N hit, M miss" from graphify.cache import check_semantic_cache if len(sys.argv) < 3: - print("Usage: graphify cache-check [--root ] [--mode | --deep]", file=sys.stderr) + print("Usage: graphify cache-check [--root ] " + "[--mode | --deep] [--prompt-file ]", file=sys.stderr) sys.exit(1) files_from = Path(sys.argv[2]) root = Path(".") cache_mode: str | None = None + prompt_file: str | None = None i = 3 while i < len(sys.argv): if sys.argv[i] == "--root" and i + 1 < len(sys.argv): @@ -2975,11 +2990,17 @@ def dispatch_command(cmd: str) -> None: elif sys.argv[i] == "--deep": cache_mode = "deep" i += 1 + elif sys.argv[i] == "--prompt-file" and i + 1 < len(sys.argv): + prompt_file = sys.argv[i + 1] + i += 2 + elif sys.argv[i].startswith("--prompt-file="): + prompt_file = sys.argv[i].split("=", 1)[1] + i += 1 else: i += 1 files = [f for f in files_from.read_text(encoding="utf-8").splitlines() if f.strip()] cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache( - files, root, mode=cache_mode + files, root, mode=cache_mode, prompt_file=prompt_file ) out = root / _GRAPHIFY_OUT out.mkdir(parents=True, exist_ok=True) diff --git a/graphify/llm.py b/graphify/llm.py index e3dbf68..3ce10f2 100644 --- a/graphify/llm.py +++ b/graphify/llm.py @@ -1935,6 +1935,10 @@ def extract_corpus_parallel( merge_existing=True, allowed_source_files=allowed, mode="deep" if deep_mode else None, + # Stamp the entry with the prompt that produced it, so a release + # that changes _EXTRACTION_SYSTEM re-extracts instead of replaying + # this vintage forever (#1939). + prompt=_extraction_system(deep=deep_mode), ) except Exception as _exc: # noqa: BLE001 — checkpoint is best-effort print(f"[graphify] incremental cache checkpoint failed: {_exc}", file=sys.stderr) diff --git a/graphify/skill-agents.md b/graphify/skill-agents.md index 4969929..19d5bea 100644 --- a/graphify/skill-agents.md +++ b/graphify/skill-agents.md @@ -214,6 +214,8 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: +SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. + ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -226,7 +228,7 @@ detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encodin # every source file (#1392). Video is transcribed to a document in Step 2.5 first. all_files = [f for cat in ('document', 'paper', 'image') for f in detect['files'].get(cat, [])] -cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH') +cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH', prompt_file='SPEC_PATH') # Always (re)write the cache file: write hits, else DELETE any leftover from a prior # run so Part C never merges a stale .graphify_cached.json (#1392). @@ -302,7 +304,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' " ``` -Save new results to cache: +Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -311,7 +313,7 @@ from pathlib import Path new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} uncached = [line for line in Path('graphify-out/.graphify_uncached.txt').read_text(encoding=\"utf-8\").splitlines() if line] -saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached) +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached, prompt_file='SPEC_PATH') print(f'Cached {saved} files') " ``` diff --git a/graphify/skill-amp.md b/graphify/skill-amp.md index 4969929..19d5bea 100644 --- a/graphify/skill-amp.md +++ b/graphify/skill-amp.md @@ -214,6 +214,8 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: +SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. + ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -226,7 +228,7 @@ detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encodin # every source file (#1392). Video is transcribed to a document in Step 2.5 first. all_files = [f for cat in ('document', 'paper', 'image') for f in detect['files'].get(cat, [])] -cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH') +cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH', prompt_file='SPEC_PATH') # Always (re)write the cache file: write hits, else DELETE any leftover from a prior # run so Part C never merges a stale .graphify_cached.json (#1392). @@ -302,7 +304,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' " ``` -Save new results to cache: +Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -311,7 +313,7 @@ from pathlib import Path new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} uncached = [line for line in Path('graphify-out/.graphify_uncached.txt').read_text(encoding=\"utf-8\").splitlines() if line] -saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached) +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached, prompt_file='SPEC_PATH') print(f'Cached {saved} files') " ``` diff --git a/graphify/skill-claw.md b/graphify/skill-claw.md index bafbca0..bf9dd34 100644 --- a/graphify/skill-claw.md +++ b/graphify/skill-claw.md @@ -214,6 +214,8 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: +SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. + ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -226,7 +228,7 @@ detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encodin # every source file (#1392). Video is transcribed to a document in Step 2.5 first. all_files = [f for cat in ('document', 'paper', 'image') for f in detect['files'].get(cat, [])] -cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH') +cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH', prompt_file='SPEC_PATH') # Always (re)write the cache file: write hits, else DELETE any leftover from a prior # run so Part C never merges a stale .graphify_cached.json (#1392). @@ -305,7 +307,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' " ``` -Save new results to cache: +Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -314,7 +316,7 @@ from pathlib import Path new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} uncached = [line for line in Path('graphify-out/.graphify_uncached.txt').read_text(encoding=\"utf-8\").splitlines() if line] -saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached) +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached, prompt_file='SPEC_PATH') print(f'Cached {saved} files') " ``` diff --git a/graphify/skill-codex.md b/graphify/skill-codex.md index d71f459..04eb570 100644 --- a/graphify/skill-codex.md +++ b/graphify/skill-codex.md @@ -214,6 +214,8 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: +SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. + ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -226,7 +228,7 @@ detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encodin # every source file (#1392). Video is transcribed to a document in Step 2.5 first. all_files = [f for cat in ('document', 'paper', 'image') for f in detect['files'].get(cat, [])] -cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH') +cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH', prompt_file='SPEC_PATH') # Always (re)write the cache file: write hits, else DELETE any leftover from a prior # run so Part C never merges a stale .graphify_cached.json (#1392). @@ -302,7 +304,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' " ``` -Save new results to cache: +Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -311,7 +313,7 @@ from pathlib import Path new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} uncached = [line for line in Path('graphify-out/.graphify_uncached.txt').read_text(encoding=\"utf-8\").splitlines() if line] -saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached) +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached, prompt_file='SPEC_PATH') print(f'Cached {saved} files') " ``` diff --git a/graphify/skill-copilot.md b/graphify/skill-copilot.md index bafbca0..bf9dd34 100644 --- a/graphify/skill-copilot.md +++ b/graphify/skill-copilot.md @@ -214,6 +214,8 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: +SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. + ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -226,7 +228,7 @@ detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encodin # every source file (#1392). Video is transcribed to a document in Step 2.5 first. all_files = [f for cat in ('document', 'paper', 'image') for f in detect['files'].get(cat, [])] -cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH') +cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH', prompt_file='SPEC_PATH') # Always (re)write the cache file: write hits, else DELETE any leftover from a prior # run so Part C never merges a stale .graphify_cached.json (#1392). @@ -305,7 +307,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' " ``` -Save new results to cache: +Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -314,7 +316,7 @@ from pathlib import Path new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} uncached = [line for line in Path('graphify-out/.graphify_uncached.txt').read_text(encoding=\"utf-8\").splitlines() if line] -saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached) +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached, prompt_file='SPEC_PATH') print(f'Cached {saved} files') " ``` diff --git a/graphify/skill-droid.md b/graphify/skill-droid.md index ae1d493..ffa9499 100644 --- a/graphify/skill-droid.md +++ b/graphify/skill-droid.md @@ -214,6 +214,8 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: +SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. + ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -226,7 +228,7 @@ detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encodin # every source file (#1392). Video is transcribed to a document in Step 2.5 first. all_files = [f for cat in ('document', 'paper', 'image') for f in detect['files'].get(cat, [])] -cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH') +cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH', prompt_file='SPEC_PATH') # Always (re)write the cache file: write hits, else DELETE any leftover from a prior # run so Part C never merges a stale .graphify_cached.json (#1392). @@ -302,7 +304,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' " ``` -Save new results to cache: +Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -311,7 +313,7 @@ from pathlib import Path new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} uncached = [line for line in Path('graphify-out/.graphify_uncached.txt').read_text(encoding=\"utf-8\").splitlines() if line] -saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached) +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached, prompt_file='SPEC_PATH') print(f'Cached {saved} files') " ``` diff --git a/graphify/skill-kilo.md b/graphify/skill-kilo.md index 6aa87a8..96dfc2e 100644 --- a/graphify/skill-kilo.md +++ b/graphify/skill-kilo.md @@ -214,6 +214,8 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: +SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. + ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -226,7 +228,7 @@ detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encodin # every source file (#1392). Video is transcribed to a document in Step 2.5 first. all_files = [f for cat in ('document', 'paper', 'image') for f in detect['files'].get(cat, [])] -cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH') +cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH', prompt_file='SPEC_PATH') # Always (re)write the cache file: write hits, else DELETE any leftover from a prior # run so Part C never merges a stale .graphify_cached.json (#1392). @@ -305,7 +307,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' " ``` -Save new results to cache: +Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -314,7 +316,7 @@ from pathlib import Path new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} uncached = [line for line in Path('graphify-out/.graphify_uncached.txt').read_text(encoding=\"utf-8\").splitlines() if line] -saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached) +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached, prompt_file='SPEC_PATH') print(f'Cached {saved} files') " ``` diff --git a/graphify/skill-kiro.md b/graphify/skill-kiro.md index bafbca0..bf9dd34 100644 --- a/graphify/skill-kiro.md +++ b/graphify/skill-kiro.md @@ -214,6 +214,8 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: +SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. + ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -226,7 +228,7 @@ detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encodin # every source file (#1392). Video is transcribed to a document in Step 2.5 first. all_files = [f for cat in ('document', 'paper', 'image') for f in detect['files'].get(cat, [])] -cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH') +cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH', prompt_file='SPEC_PATH') # Always (re)write the cache file: write hits, else DELETE any leftover from a prior # run so Part C never merges a stale .graphify_cached.json (#1392). @@ -305,7 +307,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' " ``` -Save new results to cache: +Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -314,7 +316,7 @@ from pathlib import Path new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} uncached = [line for line in Path('graphify-out/.graphify_uncached.txt').read_text(encoding=\"utf-8\").splitlines() if line] -saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached) +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached, prompt_file='SPEC_PATH') print(f'Cached {saved} files') " ``` diff --git a/graphify/skill-opencode.md b/graphify/skill-opencode.md index 8c3ce3e..6613434 100644 --- a/graphify/skill-opencode.md +++ b/graphify/skill-opencode.md @@ -214,6 +214,8 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: +SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. + ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -226,7 +228,7 @@ detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encodin # every source file (#1392). Video is transcribed to a document in Step 2.5 first. all_files = [f for cat in ('document', 'paper', 'image') for f in detect['files'].get(cat, [])] -cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH') +cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH', prompt_file='SPEC_PATH') # Always (re)write the cache file: write hits, else DELETE any leftover from a prior # run so Part C never merges a stale .graphify_cached.json (#1392). @@ -297,7 +299,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' " ``` -Save new results to cache: +Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -306,7 +308,7 @@ from pathlib import Path new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} uncached = [line for line in Path('graphify-out/.graphify_uncached.txt').read_text(encoding=\"utf-8\").splitlines() if line] -saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached) +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached, prompt_file='SPEC_PATH') print(f'Cached {saved} files') " ``` diff --git a/graphify/skill-pi.md b/graphify/skill-pi.md index bafbca0..bf9dd34 100644 --- a/graphify/skill-pi.md +++ b/graphify/skill-pi.md @@ -214,6 +214,8 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: +SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. + ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -226,7 +228,7 @@ detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encodin # every source file (#1392). Video is transcribed to a document in Step 2.5 first. all_files = [f for cat in ('document', 'paper', 'image') for f in detect['files'].get(cat, [])] -cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH') +cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH', prompt_file='SPEC_PATH') # Always (re)write the cache file: write hits, else DELETE any leftover from a prior # run so Part C never merges a stale .graphify_cached.json (#1392). @@ -305,7 +307,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' " ``` -Save new results to cache: +Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -314,7 +316,7 @@ from pathlib import Path new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} uncached = [line for line in Path('graphify-out/.graphify_uncached.txt').read_text(encoding=\"utf-8\").splitlines() if line] -saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached) +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached, prompt_file='SPEC_PATH') print(f'Cached {saved} files') " ``` diff --git a/graphify/skill-trae.md b/graphify/skill-trae.md index af59166..407f091 100644 --- a/graphify/skill-trae.md +++ b/graphify/skill-trae.md @@ -214,6 +214,8 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: +SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. + ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -226,7 +228,7 @@ detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encodin # every source file (#1392). Video is transcribed to a document in Step 2.5 first. all_files = [f for cat in ('document', 'paper', 'image') for f in detect['files'].get(cat, [])] -cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH') +cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH', prompt_file='SPEC_PATH') # Always (re)write the cache file: write hits, else DELETE any leftover from a prior # run so Part C never merges a stale .graphify_cached.json (#1392). @@ -303,7 +305,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' " ``` -Save new results to cache: +Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -312,7 +314,7 @@ from pathlib import Path new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} uncached = [line for line in Path('graphify-out/.graphify_uncached.txt').read_text(encoding=\"utf-8\").splitlines() if line] -saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached) +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached, prompt_file='SPEC_PATH') print(f'Cached {saved} files') " ``` diff --git a/graphify/skill-vscode.md b/graphify/skill-vscode.md index 11a2837..f7e7f84 100644 --- a/graphify/skill-vscode.md +++ b/graphify/skill-vscode.md @@ -214,6 +214,8 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: +SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. + ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -226,7 +228,7 @@ detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encodin # every source file (#1392). Video is transcribed to a document in Step 2.5 first. all_files = [f for cat in ('document', 'paper', 'image') for f in detect['files'].get(cat, [])] -cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH') +cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH', prompt_file='SPEC_PATH') # Always (re)write the cache file: write hits, else DELETE any leftover from a prior # run so Part C never merges a stale .graphify_cached.json (#1392). @@ -301,7 +303,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' " ``` -Save new results to cache: +Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -310,7 +312,7 @@ from pathlib import Path new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} uncached = [line for line in Path('graphify-out/.graphify_uncached.txt').read_text(encoding=\"utf-8\").splitlines() if line] -saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached) +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached, prompt_file='SPEC_PATH') print(f'Cached {saved} files') " ``` diff --git a/graphify/skill-windows.md b/graphify/skill-windows.md index 0dc7394..9c7ab74 100644 --- a/graphify/skill-windows.md +++ b/graphify/skill-windows.md @@ -236,6 +236,8 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: +SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. + ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -248,7 +250,7 @@ detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encodin # every source file (#1392). Video is transcribed to a document in Step 2.5 first. all_files = [f for cat in ('document', 'paper', 'image') for f in detect['files'].get(cat, [])] -cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH') +cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH', prompt_file='SPEC_PATH') # Always (re)write the cache file: write hits, else DELETE any leftover from a prior # run so Part C never merges a stale .graphify_cached.json (#1392). @@ -327,7 +329,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' " ``` -Save new results to cache: +Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -336,7 +338,7 @@ from pathlib import Path new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} uncached = [line for line in Path('graphify-out/.graphify_uncached.txt').read_text(encoding=\"utf-8\").splitlines() if line] -saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached) +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached, prompt_file='SPEC_PATH') print(f'Cached {saved} files') " ``` diff --git a/graphify/skill.md b/graphify/skill.md index bafbca0..bf9dd34 100644 --- a/graphify/skill.md +++ b/graphify/skill.md @@ -214,6 +214,8 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: +SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. + ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -226,7 +228,7 @@ detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encodin # every source file (#1392). Video is transcribed to a document in Step 2.5 first. all_files = [f for cat in ('document', 'paper', 'image') for f in detect['files'].get(cat, [])] -cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH') +cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH', prompt_file='SPEC_PATH') # Always (re)write the cache file: write hits, else DELETE any leftover from a prior # run so Part C never merges a stale .graphify_cached.json (#1392). @@ -305,7 +307,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' " ``` -Save new results to cache: +Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -314,7 +316,7 @@ from pathlib import Path new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} uncached = [line for line in Path('graphify-out/.graphify_uncached.txt').read_text(encoding=\"utf-8\").splitlines() if line] -saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached) +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached, prompt_file='SPEC_PATH') print(f'Cached {saved} files') " ``` diff --git a/tests/test_cache.py b/tests/test_cache.py index ff89a32..993c5a0 100644 --- a/tests/test_cache.py +++ b/tests/test_cache.py @@ -903,3 +903,234 @@ def test_save_semantic_cache_merge_existing_prunes_only_incoming(tmp_path): assert ("a", "a") in pairs, "prior entry's valid edge must survive the union" assert ("a", "b") in pairs, "incoming valid edge must be kept" assert not any("stray" in p for p in pairs) + + +# --- extraction-prompt fingerprinting (#1939) ------------------------------- + + +def test_prompt_fingerprint_stable_and_prompt_sensitive(tmp_path): + """The fingerprint is stable for identical prompts and differs when the + prompt text changes — the whole invalidation signal rests on this.""" + from graphify.cache import prompt_fingerprint + + assert prompt_fingerprint("extract a graph") == prompt_fingerprint("extract a graph") + assert prompt_fingerprint("extract a graph") != prompt_fingerprint("extract a graph v2") + + # A Path is read and hashed as its contents, so the skill path (which loads + # references/extraction-spec.md) and the Python path agree on the same text. + spec = tmp_path / "extraction-spec.md" + spec.write_text("extract a graph", encoding="utf-8") + assert prompt_fingerprint(spec) == prompt_fingerprint("extract a graph") + + +def test_prompt_fingerprint_ignores_line_endings(tmp_path): + """A CRLF checkout of the same spec must not look like a prompt change — + otherwise every Windows run re-bills the whole corpus.""" + from graphify.cache import prompt_fingerprint + + assert prompt_fingerprint("a\r\nb\r\n") == prompt_fingerprint("a\nb\n") + assert prompt_fingerprint("a \nb\n") == prompt_fingerprint("a\nb\n") + + +def test_semantic_cache_prompt_change_invalidates(tmp_path): + """The reported bug (#1939): after the extraction prompt changes, an + unchanged file must MISS instead of replaying the older vintage.""" + from graphify.cache import check_semantic_cache, save_semantic_cache + + f = tmp_path / "doc.md" + f.write_text("# Doc\n\nBody.\n") + save_semantic_cache([{"id": "old_vintage", "source_file": "doc.md"}], [], + root=tmp_path, prompt="PROMPT V1") + + # Same prompt: hit. + nodes, _, _, uncached = check_semantic_cache([str(f)], root=tmp_path, prompt="PROMPT V1") + assert [n["id"] for n in nodes] == ["old_vintage"] + assert uncached == [] + + # Prompt changed (an upgrade shipped a new extraction-spec): must re-extract. + nodes, _, _, uncached = check_semantic_cache([str(f)], root=tmp_path, prompt="PROMPT V2") + assert nodes == [] + assert uncached == [str(f)], "a new prompt must not replay the old prompt's entry" + + # V2's results land in their own namespace and do not clobber V1's, so + # rolling back to V1 still hits rather than re-billing. + save_semantic_cache([{"id": "new_vintage", "source_file": "doc.md"}], [], + root=tmp_path, prompt="PROMPT V2") + nodes, _, _, _ = check_semantic_cache([str(f)], root=tmp_path, prompt="PROMPT V2") + assert [n["id"] for n in nodes] == ["new_vintage"] + nodes, _, _, _ = check_semantic_cache([str(f)], root=tmp_path, prompt="PROMPT V1") + assert [n["id"] for n in nodes] == ["old_vintage"] + + +def test_semantic_cache_prompt_namespaced_layout(tmp_path): + """Fingerprinted entries live under cache/semantic/p{fp}/, never flat.""" + from graphify.cache import prompt_fingerprint, save_semantic_cache + + f = tmp_path / "doc.md" + f.write_text("# Doc\n") + save_semantic_cache([{"id": "n", "source_file": "doc.md"}], [], + root=tmp_path, prompt="PROMPT V1") + + sem = tmp_path / "graphify-out" / "cache" / "semantic" + h = file_hash(f, tmp_path) + assert (sem / f"p{prompt_fingerprint('PROMPT V1')}" / f"{h}.json").exists() + assert not (sem / f"{h}.json").exists(), ( + "a known-vintage entry must never be written into the flat unknown-vintage layout" + ) + + +def test_semantic_cache_prompt_and_mode_compose(tmp_path): + """The prompt fingerprint nests inside the deep namespace (#1894), so the + two dimensions are independent.""" + from graphify.cache import check_semantic_cache, save_semantic_cache + + f = tmp_path / "doc.md" + f.write_text("# Doc\n") + save_semantic_cache([{"id": "d", "source_file": "doc.md"}], [], + root=tmp_path, mode="deep", prompt="PROMPT V1") + + deep = tmp_path / "graphify-out" / "cache" / "semantic-deep" + assert list(deep.glob("p*/*.json")), "deep + prompt must nest under semantic-deep/p{fp}/" + + # Right mode, wrong prompt -> miss. Right prompt, wrong mode -> miss. + _, _, _, uncached = check_semantic_cache([str(f)], root=tmp_path, mode="deep", + prompt="PROMPT V2") + assert uncached == [str(f)] + _, _, _, uncached = check_semantic_cache([str(f)], root=tmp_path, prompt="PROMPT V1") + assert uncached == [str(f)] + # Both right -> hit. + nodes, _, _, uncached = check_semantic_cache([str(f)], root=tmp_path, mode="deep", + prompt="PROMPT V1") + assert [n["id"] for n in nodes] == ["d"] and uncached == [] + + +def test_semantic_cache_legacy_entries_served_with_warning(tmp_path): + """Entries written before fingerprinting have unknowable vintage. They are + still served — dropping them would re-bill a whole corpus on upgrade — but + the user is told how many, which is the signal #1939 says is missing today.""" + from graphify.cache import check_semantic_cache, save_semantic_cache + + a = tmp_path / "a.md" + a.write_text("# A\n") + b = tmp_path / "b.md" + b.write_text("# B\n") + # Pre-fingerprint writes: the historical flat layout. + save_semantic_cache([{"id": "a_old", "source_file": "a.md"}, + {"id": "b_old", "source_file": "b.md"}], [], root=tmp_path) + + with pytest.warns(RuntimeWarning, match="2 semantic cache entries predate"): + nodes, _, _, uncached = check_semantic_cache( + [str(a), str(b)], root=tmp_path, prompt="PROMPT V1" + ) + assert {n["id"] for n in nodes} == {"a_old", "b_old"} + assert uncached == [] + + +def test_semantic_cache_fingerprinted_entry_beats_legacy(tmp_path): + """Once a file is re-extracted under the current prompt, its fingerprinted + entry wins and the stale flat one is no longer consulted (no warning).""" + import warnings as _warnings + from graphify.cache import check_semantic_cache, save_semantic_cache + + f = tmp_path / "doc.md" + f.write_text("# Doc\n") + save_semantic_cache([{"id": "unknown_vintage", "source_file": "doc.md"}], [], + root=tmp_path) # legacy flat + save_semantic_cache([{"id": "current", "source_file": "doc.md"}], [], + root=tmp_path, prompt="PROMPT V1") + + with _warnings.catch_warnings(): + _warnings.simplefilter("error") # any legacy warning would raise here + nodes, _, _, uncached = check_semantic_cache([str(f)], root=tmp_path, + prompt="PROMPT V1") + assert [n["id"] for n in nodes] == ["current"] + assert uncached == [] + + +def test_semantic_cache_merge_existing_never_fuses_legacy_vintage(tmp_path): + """merge_existing must not union a pre-fingerprint entry into a write it is + about to stamp as current-vintage — that would mix two prompts inside one + entry and then attest the result to a prompt that produced half of it.""" + from graphify.cache import load_cached, save_semantic_cache + + f = tmp_path / "doc.md" + f.write_text("# Doc\n") + save_semantic_cache([{"id": "unknown_vintage", "source_file": "doc.md"}], [], + root=tmp_path) # legacy flat + save_semantic_cache([{"id": "current", "source_file": "doc.md"}], [], + root=tmp_path, merge_existing=True, prompt="PROMPT V1") + + entry = load_cached(f, root=tmp_path, kind="semantic", prompt="PROMPT V1") + assert [n["id"] for n in entry["nodes"]] == ["current"] + + # Within one prompt, merge_existing still unions across checkpoints. + save_semantic_cache([{"id": "second_chunk", "source_file": "doc.md"}], [], + root=tmp_path, merge_existing=True, prompt="PROMPT V1") + entry = load_cached(f, root=tmp_path, kind="semantic", prompt="PROMPT V1") + assert {n["id"] for n in entry["nodes"]} == {"current", "second_chunk"} + + +def test_semantic_prune_and_clear_reach_fingerprint_subdirs(tmp_path): + """A glob that stopped at the top level would leave every fingerprinted + entry unprunable, re-growing the unbounded-orphan problem of #1527.""" + from graphify.cache import ( + cached_files, clear_cache, prune_semantic_cache, save_semantic_cache, + ) + + f = tmp_path / "doc.md" + f.write_text("# Doc\n") + save_semantic_cache([{"id": "n", "source_file": "doc.md"}], [], + root=tmp_path, prompt="PROMPT V1") + h = file_hash(f, tmp_path) + assert h in cached_files(tmp_path), "cached_files must see fingerprinted entries" + + # Live: kept. + assert prune_semantic_cache(tmp_path, {h}) == 0 + # Orphaned (content changed / file deleted): pruned. + assert prune_semantic_cache(tmp_path, set()) == 1 + + save_semantic_cache([{"id": "n", "source_file": "doc.md"}], [], + root=tmp_path, prompt="PROMPT V1") + clear_cache(tmp_path) + assert not list((tmp_path / "graphify-out" / "cache" / "semantic").glob("**/*.json")) + + +def test_semantic_cache_unreadable_prompt_file_warns_and_falls_back(tmp_path): + """A skill snippet substitutes SPEC_PATH by hand. If it lands on a path that + isn't there, the fallback to the unattributed layout must be loud: silently + reverting to unversioned keying is exactly the #1939 behavior being fixed.""" + from graphify.cache import check_semantic_cache, save_semantic_cache + + f = tmp_path / "doc.md" + f.write_text("# Doc\n") + save_semantic_cache([{"id": "n", "source_file": "doc.md"}], [], root=tmp_path) + + with pytest.warns(RuntimeWarning, match="could not read extraction prompt"): + nodes, _, _, uncached = check_semantic_cache( + [str(f)], root=tmp_path, prompt_file=str(tmp_path / "nope.md") + ) + # Fell back rather than aborting the run. + assert [n["id"] for n in nodes] == ["n"] and uncached == [] + + +def test_prompt_file_reflects_edited_spec(tmp_path): + """The prompt-file fingerprint is memoized per (path, size, mtime); an edited + spec must still register as a new prompt rather than reusing a stale memo.""" + from graphify.cache import check_semantic_cache, save_semantic_cache + + spec = tmp_path / "extraction-spec.md" + spec.write_text("prompt one", encoding="utf-8") + f = tmp_path / "doc.md" + f.write_text("# Doc\n") + + save_semantic_cache([{"id": "v1", "source_file": "doc.md"}], [], + root=tmp_path, prompt_file=str(spec)) + nodes, _, _, _ = check_semantic_cache([str(f)], root=tmp_path, prompt_file=str(spec)) + assert [n["id"] for n in nodes] == ["v1"] + + # An upgrade rewrites the spec: the same file path is now a different prompt. + import os as _os + spec.write_text("prompt two — rewritten by an upgrade", encoding="utf-8") + _os.utime(spec, ns=(0, 0)) # force a distinct stat signature + _, _, _, uncached = check_semantic_cache([str(f)], root=tmp_path, prompt_file=str(spec)) + assert uncached == [str(f)], "an edited spec must invalidate, not reuse the memo" diff --git a/tests/test_chunking.py b/tests/test_chunking.py index a978a5d..19e65ef 100644 --- a/tests/test_chunking.py +++ b/tests/test_chunking.py @@ -321,8 +321,11 @@ def test_checkpoint_scopes_cache_writes_to_chunk_files(tmp_path): assert [n["id"] for n in after["nodes"]] == ["b_real"], ( f"B.py cache was clobbered by an out-of-chunk node: {after}" ) - # A.py (the actual chunk file) was legitimately cached. - a_cache = load_cached(a, tmp_path, kind="semantic") + # A.py (the actual chunk file) was legitimately cached. The checkpoint stamps + # entries with the prompt that produced them (#1939), so read that namespace. + from graphify.llm import _extraction_system + + a_cache = load_cached(a, tmp_path, kind="semantic", prompt=_extraction_system()) assert a_cache and any(n["id"] == "a_ok" for n in a_cache["nodes"]) @@ -330,7 +333,7 @@ def test_checkpoint_writes_deep_namespace_in_deep_mode(tmp_path): """#1894: the per-chunk checkpoint must follow the run's mode — a deep_mode=True run checkpoints into cache/semantic-deep/, leaving the standard cache/semantic/ namespace untouched (and vice versa).""" - from graphify.llm import extract_corpus_parallel + from graphify.llm import extract_corpus_parallel, _extraction_system from graphify.cache import load_cached doc = tmp_path / "doc.md" @@ -349,11 +352,15 @@ def test_checkpoint_writes_deep_namespace_in_deep_mode(tmp_path): deep_mode=True, ) - deep = load_cached(doc, tmp_path, kind="semantic-deep") + # The checkpoint also stamps entries with the prompt that produced them + # (#1939) — a deep run's prompt carries the deep suffix. + deep = load_cached(doc, tmp_path, kind="semantic-deep", + prompt=_extraction_system(deep=True)) assert deep and [n["id"] for n in deep["nodes"]] == ["d1"], ( "deep-mode checkpoint must land in cache/semantic-deep/" ) - assert load_cached(doc, tmp_path, kind="semantic") is None, ( + assert load_cached(doc, tmp_path, kind="semantic", + prompt=_extraction_system(deep=False)) is None, ( "deep-mode checkpoint must not write the standard semantic namespace" ) @@ -484,7 +491,9 @@ def test_checkpoint_caches_sliced_document_chunks(tmp_path, capsys): split into FileSlice units; before the fix each sliced chunk leaked the FileSlice object into the allowlist, so save_semantic_cache raised TypeError, the best-effort except swallowed it, and the slice was never checkpointed.""" - from graphify.llm import extract_corpus_parallel, expand_oversized_files, _FILE_CHAR_CAP + from graphify.llm import ( + extract_corpus_parallel, expand_oversized_files, _FILE_CHAR_CAP, _extraction_system, + ) from graphify.file_slice import FileSlice from graphify.cache import load_cached @@ -510,7 +519,8 @@ def test_checkpoint_caches_sliced_document_chunks(tmp_path, capsys): assert "incremental cache checkpoint failed" not in capsys.readouterr().err, ( "checkpoint raised on a FileSlice chunk (#1870)" ) - cached = load_cached(doc, tmp_path, kind="semantic") + # The checkpoint stamps entries with the prompt that produced them (#1939). + cached = load_cached(doc, tmp_path, kind="semantic", prompt=_extraction_system()) assert cached and any(n["id"] == "big_title" for n in cached["nodes"]), ( "sliced document was never checkpointed (#1870)" ) diff --git a/tests/test_extract_cli.py b/tests/test_extract_cli.py index 87a65e4..d18bf5a 100644 --- a/tests/test_extract_cli.py +++ b/tests/test_extract_cli.py @@ -1,6 +1,8 @@ """Tests for `graphify extract` CLI dispatch path in graphify.__main__.""" from __future__ import annotations +import os + import pytest import graphify.__main__ as mainmod @@ -360,8 +362,9 @@ def test_extract_mode_deep_dispatches_over_warm_cache(monkeypatch, tmp_path): assert len(calls) == 2, ( "second deep run must be served from cache/semantic-deep/" ) - # The deep entry landed in its own namespace, not cache/semantic/. - assert any((corpus / "graphify-out" / "cache" / "semantic-deep").glob("*.json")) + # The deep entry landed in its own namespace, not cache/semantic/. Entries are + # nested under a p{prompt-fingerprint}/ subdir (#1939), hence the recursive glob. + assert any((corpus / "graphify-out" / "cache" / "semantic-deep").glob("**/*.json")) def test_extract_force_flag_redispatches_and_stamps_manifest(monkeypatch, tmp_path): @@ -393,7 +396,8 @@ def test_extract_force_flag_redispatches_and_stamps_manifest(monkeypatch, tmp_pa assert calls[1]["paths"] == [str(corpus / "README.md")] # The forced run still wrote the semantic cache and stamped the manifest. - assert any((corpus / "graphify-out" / "cache" / "semantic").glob("*.json")) + # Entries nest under a p{prompt-fingerprint}/ subdir (#1939). + assert any((corpus / "graphify-out" / "cache" / "semantic").glob("**/*.json")) manifest = json.loads( (corpus / "graphify-out" / "manifest.json").read_text() ) @@ -824,3 +828,30 @@ def test_no_cluster_incremental_prunes_newly_excluded_file( f"--no-cluster early exit must prune excluded sources, still see {sources}" ) assert any("keep.py" in s for s in sources) + + +def test_cache_check_prompt_file_scopes_hits_to_that_prompt(monkeypatch, tmp_path, capsys): + """#1939: cache-check --prompt-file only counts entries produced by that same + extraction prompt, so an upgraded prompt reports a miss (re-extract) rather + than replaying the older vintage.""" + from graphify.cache import save_semantic_cache + + doc = tmp_path / "doc.md" + doc.write_text("# Doc\n") + spec = tmp_path / "extraction-spec.md" + spec.write_text("PROMPT V1", encoding="utf-8") + save_semantic_cache([{"id": "d", "source_file": "doc.md"}], [], + root=tmp_path, prompt_file=str(spec)) + files_from = tmp_path / "files.txt" + files_from.write_text(str(doc) + "\n") + monkeypatch.setattr(mainmod, "_check_skill_version", lambda _: None) + + base = ["graphify", "cache-check", str(files_from), "--root", str(tmp_path)] + _run_extract(monkeypatch, base + ["--prompt-file", str(spec)]) + assert "Cache: 1 hit, 0 miss" in capsys.readouterr().out + + # An upgrade rewrites the prompt: the entry must no longer satisfy the run. + spec.write_text("PROMPT V2 — rewritten by an upgrade", encoding="utf-8") + os.utime(spec, ns=(0, 0)) + _run_extract(monkeypatch, base + ["--prompt-file", str(spec)]) + assert "Cache: 0 hit, 1 miss" in capsys.readouterr().out diff --git a/tests/test_skillgen.py b/tests/test_skillgen.py index 9797267..0c09e60 100644 --- a/tests/test_skillgen.py +++ b/tests/test_skillgen.py @@ -949,3 +949,31 @@ def test_agents_audit_baseline_is_amps_v8_body(): assert gen._v8_baseline_ref("agents") == "47042beb05d1f6dd2186c0c499ae2840ce604ead:graphify/skill-amp.md" problems = gen.audit_coverage(platforms["agents"]) assert problems == [], "\n".join(problems) + + +def test_semantic_cache_calls_pass_prompt_file_for_every_split_host(): + """#1939: a skill's cache read and write must both name the extraction prompt + they use, or the run replays entries produced by an older prompt (the read) / + strands its results where the next read won't look (the write). + + Locked per host because the two calls live ~80 lines apart in the rendered + body: adding the argument to one and not the other silently disables the + cache rather than failing loudly. The monolith hosts (aider, devin) inline + their prompt instead of shipping references/extraction-spec.md and are + deliberately excluded — they have no spec path to point at. + """ + platforms = gen.load_platforms() + arts = gen.render_all(platforms) + bodies = [a for a in arts + if "check_semantic_cache(" in a.content + and "references/extraction-spec.md" in a.content] + assert bodies, "no rendered split-host skill body calls check_semantic_cache" + for a in bodies: + for call in ("check_semantic_cache(", "save_semantic_cache("): + line = next(ln for ln in a.content.splitlines() if call in ln and "import" not in ln) + assert "prompt_file='SPEC_PATH'" in line, ( + f"{a.path}: {call} must pass prompt_file so entries are attributed " + f"to the extraction prompt (#1939) — got: {line.strip()}" + ) + # The placeholder is inert unless the body tells the agent what to substitute. + assert "SPEC_PATH below is the **absolute** path" in a.content, a.path diff --git a/tools/skillgen/expected/graphify__skill-agents.md b/tools/skillgen/expected/graphify__skill-agents.md index 4969929..19d5bea 100644 --- a/tools/skillgen/expected/graphify__skill-agents.md +++ b/tools/skillgen/expected/graphify__skill-agents.md @@ -214,6 +214,8 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: +SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. + ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -226,7 +228,7 @@ detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encodin # every source file (#1392). Video is transcribed to a document in Step 2.5 first. all_files = [f for cat in ('document', 'paper', 'image') for f in detect['files'].get(cat, [])] -cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH') +cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH', prompt_file='SPEC_PATH') # Always (re)write the cache file: write hits, else DELETE any leftover from a prior # run so Part C never merges a stale .graphify_cached.json (#1392). @@ -302,7 +304,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' " ``` -Save new results to cache: +Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -311,7 +313,7 @@ from pathlib import Path new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} uncached = [line for line in Path('graphify-out/.graphify_uncached.txt').read_text(encoding=\"utf-8\").splitlines() if line] -saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached) +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached, prompt_file='SPEC_PATH') print(f'Cached {saved} files') " ``` diff --git a/tools/skillgen/expected/graphify__skill-amp.md b/tools/skillgen/expected/graphify__skill-amp.md index 4969929..19d5bea 100644 --- a/tools/skillgen/expected/graphify__skill-amp.md +++ b/tools/skillgen/expected/graphify__skill-amp.md @@ -214,6 +214,8 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: +SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. + ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -226,7 +228,7 @@ detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encodin # every source file (#1392). Video is transcribed to a document in Step 2.5 first. all_files = [f for cat in ('document', 'paper', 'image') for f in detect['files'].get(cat, [])] -cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH') +cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH', prompt_file='SPEC_PATH') # Always (re)write the cache file: write hits, else DELETE any leftover from a prior # run so Part C never merges a stale .graphify_cached.json (#1392). @@ -302,7 +304,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' " ``` -Save new results to cache: +Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -311,7 +313,7 @@ from pathlib import Path new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} uncached = [line for line in Path('graphify-out/.graphify_uncached.txt').read_text(encoding=\"utf-8\").splitlines() if line] -saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached) +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached, prompt_file='SPEC_PATH') print(f'Cached {saved} files') " ``` diff --git a/tools/skillgen/expected/graphify__skill-claw.md b/tools/skillgen/expected/graphify__skill-claw.md index bafbca0..bf9dd34 100644 --- a/tools/skillgen/expected/graphify__skill-claw.md +++ b/tools/skillgen/expected/graphify__skill-claw.md @@ -214,6 +214,8 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: +SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. + ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -226,7 +228,7 @@ detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encodin # every source file (#1392). Video is transcribed to a document in Step 2.5 first. all_files = [f for cat in ('document', 'paper', 'image') for f in detect['files'].get(cat, [])] -cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH') +cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH', prompt_file='SPEC_PATH') # Always (re)write the cache file: write hits, else DELETE any leftover from a prior # run so Part C never merges a stale .graphify_cached.json (#1392). @@ -305,7 +307,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' " ``` -Save new results to cache: +Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -314,7 +316,7 @@ from pathlib import Path new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} uncached = [line for line in Path('graphify-out/.graphify_uncached.txt').read_text(encoding=\"utf-8\").splitlines() if line] -saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached) +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached, prompt_file='SPEC_PATH') print(f'Cached {saved} files') " ``` diff --git a/tools/skillgen/expected/graphify__skill-codex.md b/tools/skillgen/expected/graphify__skill-codex.md index d71f459..04eb570 100644 --- a/tools/skillgen/expected/graphify__skill-codex.md +++ b/tools/skillgen/expected/graphify__skill-codex.md @@ -214,6 +214,8 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: +SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. + ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -226,7 +228,7 @@ detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encodin # every source file (#1392). Video is transcribed to a document in Step 2.5 first. all_files = [f for cat in ('document', 'paper', 'image') for f in detect['files'].get(cat, [])] -cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH') +cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH', prompt_file='SPEC_PATH') # Always (re)write the cache file: write hits, else DELETE any leftover from a prior # run so Part C never merges a stale .graphify_cached.json (#1392). @@ -302,7 +304,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' " ``` -Save new results to cache: +Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -311,7 +313,7 @@ from pathlib import Path new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} uncached = [line for line in Path('graphify-out/.graphify_uncached.txt').read_text(encoding=\"utf-8\").splitlines() if line] -saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached) +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached, prompt_file='SPEC_PATH') print(f'Cached {saved} files') " ``` diff --git a/tools/skillgen/expected/graphify__skill-copilot.md b/tools/skillgen/expected/graphify__skill-copilot.md index bafbca0..bf9dd34 100644 --- a/tools/skillgen/expected/graphify__skill-copilot.md +++ b/tools/skillgen/expected/graphify__skill-copilot.md @@ -214,6 +214,8 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: +SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. + ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -226,7 +228,7 @@ detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encodin # every source file (#1392). Video is transcribed to a document in Step 2.5 first. all_files = [f for cat in ('document', 'paper', 'image') for f in detect['files'].get(cat, [])] -cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH') +cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH', prompt_file='SPEC_PATH') # Always (re)write the cache file: write hits, else DELETE any leftover from a prior # run so Part C never merges a stale .graphify_cached.json (#1392). @@ -305,7 +307,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' " ``` -Save new results to cache: +Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -314,7 +316,7 @@ from pathlib import Path new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} uncached = [line for line in Path('graphify-out/.graphify_uncached.txt').read_text(encoding=\"utf-8\").splitlines() if line] -saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached) +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached, prompt_file='SPEC_PATH') print(f'Cached {saved} files') " ``` diff --git a/tools/skillgen/expected/graphify__skill-droid.md b/tools/skillgen/expected/graphify__skill-droid.md index ae1d493..ffa9499 100644 --- a/tools/skillgen/expected/graphify__skill-droid.md +++ b/tools/skillgen/expected/graphify__skill-droid.md @@ -214,6 +214,8 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: +SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. + ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -226,7 +228,7 @@ detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encodin # every source file (#1392). Video is transcribed to a document in Step 2.5 first. all_files = [f for cat in ('document', 'paper', 'image') for f in detect['files'].get(cat, [])] -cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH') +cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH', prompt_file='SPEC_PATH') # Always (re)write the cache file: write hits, else DELETE any leftover from a prior # run so Part C never merges a stale .graphify_cached.json (#1392). @@ -302,7 +304,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' " ``` -Save new results to cache: +Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -311,7 +313,7 @@ from pathlib import Path new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} uncached = [line for line in Path('graphify-out/.graphify_uncached.txt').read_text(encoding=\"utf-8\").splitlines() if line] -saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached) +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached, prompt_file='SPEC_PATH') print(f'Cached {saved} files') " ``` diff --git a/tools/skillgen/expected/graphify__skill-kilo.md b/tools/skillgen/expected/graphify__skill-kilo.md index 6aa87a8..96dfc2e 100644 --- a/tools/skillgen/expected/graphify__skill-kilo.md +++ b/tools/skillgen/expected/graphify__skill-kilo.md @@ -214,6 +214,8 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: +SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. + ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -226,7 +228,7 @@ detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encodin # every source file (#1392). Video is transcribed to a document in Step 2.5 first. all_files = [f for cat in ('document', 'paper', 'image') for f in detect['files'].get(cat, [])] -cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH') +cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH', prompt_file='SPEC_PATH') # Always (re)write the cache file: write hits, else DELETE any leftover from a prior # run so Part C never merges a stale .graphify_cached.json (#1392). @@ -305,7 +307,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' " ``` -Save new results to cache: +Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -314,7 +316,7 @@ from pathlib import Path new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} uncached = [line for line in Path('graphify-out/.graphify_uncached.txt').read_text(encoding=\"utf-8\").splitlines() if line] -saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached) +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached, prompt_file='SPEC_PATH') print(f'Cached {saved} files') " ``` diff --git a/tools/skillgen/expected/graphify__skill-kiro.md b/tools/skillgen/expected/graphify__skill-kiro.md index bafbca0..bf9dd34 100644 --- a/tools/skillgen/expected/graphify__skill-kiro.md +++ b/tools/skillgen/expected/graphify__skill-kiro.md @@ -214,6 +214,8 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: +SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. + ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -226,7 +228,7 @@ detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encodin # every source file (#1392). Video is transcribed to a document in Step 2.5 first. all_files = [f for cat in ('document', 'paper', 'image') for f in detect['files'].get(cat, [])] -cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH') +cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH', prompt_file='SPEC_PATH') # Always (re)write the cache file: write hits, else DELETE any leftover from a prior # run so Part C never merges a stale .graphify_cached.json (#1392). @@ -305,7 +307,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' " ``` -Save new results to cache: +Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -314,7 +316,7 @@ from pathlib import Path new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} uncached = [line for line in Path('graphify-out/.graphify_uncached.txt').read_text(encoding=\"utf-8\").splitlines() if line] -saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached) +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached, prompt_file='SPEC_PATH') print(f'Cached {saved} files') " ``` diff --git a/tools/skillgen/expected/graphify__skill-opencode.md b/tools/skillgen/expected/graphify__skill-opencode.md index 8c3ce3e..6613434 100644 --- a/tools/skillgen/expected/graphify__skill-opencode.md +++ b/tools/skillgen/expected/graphify__skill-opencode.md @@ -214,6 +214,8 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: +SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. + ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -226,7 +228,7 @@ detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encodin # every source file (#1392). Video is transcribed to a document in Step 2.5 first. all_files = [f for cat in ('document', 'paper', 'image') for f in detect['files'].get(cat, [])] -cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH') +cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH', prompt_file='SPEC_PATH') # Always (re)write the cache file: write hits, else DELETE any leftover from a prior # run so Part C never merges a stale .graphify_cached.json (#1392). @@ -297,7 +299,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' " ``` -Save new results to cache: +Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -306,7 +308,7 @@ from pathlib import Path new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} uncached = [line for line in Path('graphify-out/.graphify_uncached.txt').read_text(encoding=\"utf-8\").splitlines() if line] -saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached) +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached, prompt_file='SPEC_PATH') print(f'Cached {saved} files') " ``` diff --git a/tools/skillgen/expected/graphify__skill-pi.md b/tools/skillgen/expected/graphify__skill-pi.md index bafbca0..bf9dd34 100644 --- a/tools/skillgen/expected/graphify__skill-pi.md +++ b/tools/skillgen/expected/graphify__skill-pi.md @@ -214,6 +214,8 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: +SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. + ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -226,7 +228,7 @@ detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encodin # every source file (#1392). Video is transcribed to a document in Step 2.5 first. all_files = [f for cat in ('document', 'paper', 'image') for f in detect['files'].get(cat, [])] -cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH') +cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH', prompt_file='SPEC_PATH') # Always (re)write the cache file: write hits, else DELETE any leftover from a prior # run so Part C never merges a stale .graphify_cached.json (#1392). @@ -305,7 +307,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' " ``` -Save new results to cache: +Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -314,7 +316,7 @@ from pathlib import Path new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} uncached = [line for line in Path('graphify-out/.graphify_uncached.txt').read_text(encoding=\"utf-8\").splitlines() if line] -saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached) +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached, prompt_file='SPEC_PATH') print(f'Cached {saved} files') " ``` diff --git a/tools/skillgen/expected/graphify__skill-trae.md b/tools/skillgen/expected/graphify__skill-trae.md index af59166..407f091 100644 --- a/tools/skillgen/expected/graphify__skill-trae.md +++ b/tools/skillgen/expected/graphify__skill-trae.md @@ -214,6 +214,8 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: +SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. + ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -226,7 +228,7 @@ detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encodin # every source file (#1392). Video is transcribed to a document in Step 2.5 first. all_files = [f for cat in ('document', 'paper', 'image') for f in detect['files'].get(cat, [])] -cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH') +cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH', prompt_file='SPEC_PATH') # Always (re)write the cache file: write hits, else DELETE any leftover from a prior # run so Part C never merges a stale .graphify_cached.json (#1392). @@ -303,7 +305,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' " ``` -Save new results to cache: +Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -312,7 +314,7 @@ from pathlib import Path new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} uncached = [line for line in Path('graphify-out/.graphify_uncached.txt').read_text(encoding=\"utf-8\").splitlines() if line] -saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached) +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached, prompt_file='SPEC_PATH') print(f'Cached {saved} files') " ``` diff --git a/tools/skillgen/expected/graphify__skill-vscode.md b/tools/skillgen/expected/graphify__skill-vscode.md index 11a2837..f7e7f84 100644 --- a/tools/skillgen/expected/graphify__skill-vscode.md +++ b/tools/skillgen/expected/graphify__skill-vscode.md @@ -214,6 +214,8 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: +SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. + ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -226,7 +228,7 @@ detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encodin # every source file (#1392). Video is transcribed to a document in Step 2.5 first. all_files = [f for cat in ('document', 'paper', 'image') for f in detect['files'].get(cat, [])] -cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH') +cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH', prompt_file='SPEC_PATH') # Always (re)write the cache file: write hits, else DELETE any leftover from a prior # run so Part C never merges a stale .graphify_cached.json (#1392). @@ -301,7 +303,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' " ``` -Save new results to cache: +Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -310,7 +312,7 @@ from pathlib import Path new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} uncached = [line for line in Path('graphify-out/.graphify_uncached.txt').read_text(encoding=\"utf-8\").splitlines() if line] -saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached) +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached, prompt_file='SPEC_PATH') print(f'Cached {saved} files') " ``` diff --git a/tools/skillgen/expected/graphify__skill-windows.md b/tools/skillgen/expected/graphify__skill-windows.md index 0dc7394..9c7ab74 100644 --- a/tools/skillgen/expected/graphify__skill-windows.md +++ b/tools/skillgen/expected/graphify__skill-windows.md @@ -236,6 +236,8 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: +SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. + ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -248,7 +250,7 @@ detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encodin # every source file (#1392). Video is transcribed to a document in Step 2.5 first. all_files = [f for cat in ('document', 'paper', 'image') for f in detect['files'].get(cat, [])] -cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH') +cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH', prompt_file='SPEC_PATH') # Always (re)write the cache file: write hits, else DELETE any leftover from a prior # run so Part C never merges a stale .graphify_cached.json (#1392). @@ -327,7 +329,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' " ``` -Save new results to cache: +Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -336,7 +338,7 @@ from pathlib import Path new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} uncached = [line for line in Path('graphify-out/.graphify_uncached.txt').read_text(encoding=\"utf-8\").splitlines() if line] -saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached) +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached, prompt_file='SPEC_PATH') print(f'Cached {saved} files') " ``` diff --git a/tools/skillgen/expected/graphify__skill.md b/tools/skillgen/expected/graphify__skill.md index bafbca0..bf9dd34 100644 --- a/tools/skillgen/expected/graphify__skill.md +++ b/tools/skillgen/expected/graphify__skill.md @@ -214,6 +214,8 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: +SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. + ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -226,7 +228,7 @@ detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encodin # every source file (#1392). Video is transcribed to a document in Step 2.5 first. all_files = [f for cat in ('document', 'paper', 'image') for f in detect['files'].get(cat, [])] -cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH') +cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH', prompt_file='SPEC_PATH') # Always (re)write the cache file: write hits, else DELETE any leftover from a prior # run so Part C never merges a stale .graphify_cached.json (#1392). @@ -305,7 +307,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' " ``` -Save new results to cache: +Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -314,7 +316,7 @@ from pathlib import Path new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} uncached = [line for line in Path('graphify-out/.graphify_uncached.txt').read_text(encoding=\"utf-8\").splitlines() if line] -saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached) +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached, prompt_file='SPEC_PATH') print(f'Cached {saved} files') " ``` diff --git a/tools/skillgen/fragments/core/core.md b/tools/skillgen/fragments/core/core.md index a5e117b..cf8f030 100644 --- a/tools/skillgen/fragments/core/core.md +++ b/tools/skillgen/fragments/core/core.md @@ -173,6 +173,8 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: +SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. + ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -185,7 +187,7 @@ detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encodin # every source file (#1392). Video is transcribed to a document in Step 2.5 first. all_files = [f for cat in ('document', 'paper', 'image') for f in detect['files'].get(cat, [])] -cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH') +cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH', prompt_file='SPEC_PATH') # Always (re)write the cache file: write hits, else DELETE any leftover from a prior # run so Part C never merges a stale .graphify_cached.json (#1392). @@ -240,7 +242,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' " ``` -Save new results to cache: +Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash $(cat graphify-out/.graphify_python) -c " import json @@ -249,7 +251,7 @@ from pathlib import Path new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} uncached = [line for line in Path('graphify-out/.graphify_uncached.txt').read_text(encoding=\"utf-8\").splitlines() if line] -saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached) +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH', allowed_source_files=uncached, prompt_file='SPEC_PATH') print(f'Cached {saved} files') " ```