jrgemignani opened a new pull request, #2376:
URL: https://github.com/apache/age/pull/2376
Replace AGE's snapshot-based VLE cache invalidation with per-graph monotonic
version counters in shared memory. The old code compared PostgreSQL's global
xmin/xmax/curcid, causing false cache invalidation whenever ANY transaction ran
on the server — even unrelated ones. This forced a full hash table rebuild
(~138s at SF3) on every VLE query in any multi-connection environment.
The fix uses three invalidation paths with automatic detection:
- DSM (PG 17+): GetNamedDSMSegment — works without shared_preload_libraries
- SHMEM (PG <17): shmem_request/startup hooks — needs
shared_preload_libraries
- SNAPSHOT: fallback to original behavior when shared memory unavailable
Version counter increment points:
- Cypher CREATE/DELETE/SET/MERGE via executor hooks
- SQL INSERT/UPDATE/DELETE via auto-installed per-table triggers
- TRUNCATE via ProcessUtility hook interception
Additional optimizations:
- Thin entries: vertex/edge hash table entries store 6-byte TID instead of
copied property Datum; properties fetched on demand via heap_fetch only during
result construction. Reduces hash table memory by ~77%.
- Fast path in is_an_edge_match: skip property access for label-only VLE
patterns (e.g., [:KNOWS*1..2]), avoiding unnecessary heap_fetch during DFS
traversal.
- Defensive elog(ERROR) on stale TID in lazy property fetch to catch
invalidation logic bugs.
- Trigger install is conditional — checks if the trigger function exists in
the catalog before attempting installation, ensuring backward compatibility
with older extension SQL versions.
Test results (LDBC SNB benchmark, SF3 — 52.7M edges, 9.3M vertices):
Production simulation (VLE with concurrent background transactions):
Before: 177,188 ms avg per query (full rebuild every time)
After: 15.7 ms avg per query (cache hit)
Speedup: 11,299x
Cold build time:
Before: 186,275 ms
After: 108,955 ms (41% faster — no datumCopy)
LDBC IC1 warm (3-hop VLE, single session):
Before: 219,385 ms
After: 175,249 ms (20% faster — better cache utilization)
Hash table memory (SF3):
Before: ~9 GB
After: ~2.1 GB (77% reduction)
New regression tests in age_global_graph.sql verify VLE cache invalidation
after CREATE, DELETE, and SET operations, plus thin entry property fetch.
Regression tests: 32/32 pass
Files changed (14):
src/backend/age.c — shmem hook registration (PG <17)
src/backend/catalog/ag_catalog.c — TRUNCATE interception
src/backend/commands/label_commands.c — conditional trigger
auto-install on label creation
src/backend/executor/cypher_create.c — increment_graph_version after
CREATE
src/backend/executor/cypher_delete.c — increment_graph_version after
DELETE
src/backend/executor/cypher_merge.c — increment_graph_version after
MERGE
src/backend/executor/cypher_set.c — increment_graph_version after
SET
src/backend/utils/adt/age_global_graph.c — version counter, thin entries,
trigger fn, lazy fetch
src/backend/utils/adt/age_vle.c — is_an_edge_match fast path
src/include/utils/age_global_graph.h — new declarations
sql/age_main.sql — trigger function registration
for next-version SQL
regress/sql/age_global_graph.sql — VLE cache regression tests
regress/expected/age_global_graph.out — expected output for new tests
age--1.7.0--y.y.y.sql — upgrade template: trigger
function for existing installs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
