Hi, >> Is there a reason not to include this information in the scan descriptor of >> particular AM?
Required columns, pushed-down predicates, and limits may still be
scan-lifetime state owned by a particular table AM. That is orthogonal to
this series. On the executor side, tuples are represented by
TupleTableSlots, so v4 makes batching an optional slot capability rather
than adding a table AM callback. Each AM remains free to keep its scan
descriptor and batch representation private.
> I'll try to reply properly on the rest of Denis's points later this week.
There is no need to review v3 at this point: v4 supersedes it.
V3 demonstrated the performance benefit, but its executor code depended
directly on heap pages and BufferHeapTupleTableSlot. V4 moves the batch
interface to TupleTableSlot. The common executor can use a batch exposed by
a slot and otherwise falls back to the existing scalar path.
This was designed so that TimescaleDB slots without stable batches can keep
using the scalar path, while a Citus columnar slot could expose its native
batch and provide only the requested columns. I have not tested either
extension, so these are design goals rather than compatibility claims.
V4 also adds batch tuple deformation. An earlier implementation could
repeatedly walk a wide physical tuple when quals referenced columns in an
unfavorable order, causing regressions of up to 34%. The slot now extracts
the requested attributes in physical order with one tuple walk, while quals
are still evaluated in the order chosen by the planner.
I compared v4 with its exact master base on prewarmed 5M- and 10M-row tables,
both with and without all-visible pages. Each result is a range from two
medians of 31 executions:
all-visible not-all-visible
count(*), no qual -7.96% .. -7.60% -8.03% .. -7.25%
projection -10.49% .. -10.24% -8.82% .. -8.18%
one qual on column a or o -23.57% .. -19.67% -22.11% .. -18.30%
six quals, pass all -15.71% .. -14.25% -14.55% .. -11.00%
six quals, reject rows -18.69% .. -16.56% -15.93% .. -11.52%
six quals, late pass-half -15.62% .. -15.56% -12.61% .. -11.19%
All 56 comparisons favored v4. The geometric mean was 20.61% for the
single-qual tests and 14.23% for the multi-column deformation tests.
Changing the qual order affected patched execution time by at most 0.89%.
v4-0001-Expose-heap-page-batches-through-tuple-slots.patch
Description: Binary data
v4-0002-Evaluate-SeqScan-quals-in-tuple-batches.patch
Description: Binary data
v4-0003-Deform-heap-tuple-batches-in-physical-order.patch
Description: Binary data
