This is an automated email from the ASF dual-hosted git repository.
xiangfu0 pushed a change to branch xiangfu0/data-3221-6-lazy-materialization
in repository https://gitbox.apache.org/repos/asf/pinot.git
omit 73bf3edd64e Keep nested and sparse settings distinct during config
sharing
omit 7677d53bb1d Use current OpenStruct config constructors in materializer
tests
omit 11352519dc4 Keep lazy materialization compatible with the segment read
lifecycle
omit 6a9541b967f Keep rebased source imports in project order
omit 6041cd6972b Serialize v1/v2 index buffer access for lazy
materialization and cover destroy vs in-flight
omit 93f193c7199 Retain compact index configs for segment-specific loads
omit 88f4bcf1282 Reuse resolved index configs for lazy column
materialization
omit c8e5a00705a Materialize immutable-segment columns lazily behind an
opt-in instance config (default off)
add f818bae6d75 Fix controller job-limit leak and multicluster test port
allocation (#19773)
add 763ba2cb7e1 Use Node 24 for compatibility workflow tooling (#19796)
add 6f87b7150f4 Fix JDK 25 compilation in Zstandard decompressor (#19798)
add 85d90cb3db0 Add class-level Javadoc to ExprMinMaxObjectTest (#19802)
add ada7648609a Count TEXT_MATCH filters without materializing matching
doc ids (#19760)
add ee2ee576f8b Persist upsert snapshot diagnostics for replica divergence
triage (#19637)
add 94fc1432751 Normalize signed zero in transform test fixtures (#19803)
add 113f9ed26bc Bump org.ow2.asm:asm from 9.10.1 to 9.11 (#19813)
add f59ce704600 Bump org.xerial.snappy:snappy-java from 1.1.10.10 to
1.1.10.11 (#19812)
add ec8d00a5c44 Bump io.prometheus.jmx:collector from 1.6.0 to 1.7.0
(#19811)
add 62e7a2ae2ec Bump io.netty:netty-bom from 4.1.138.Final to
4.1.139.Final (#19810)
add 4d6e2681d11 Bump org.jline:jline from 4.4.6 to 4.4.7 (#19808)
add 4b7b358b9dd Push AND restrictions into composite filter children
(#19408)
add 910f9d5e494 Bump org.roaringbitmap:RoaringBitmap from 1.6.23 to 1.6.24
(#19809)
add 99251704d38 Materialize immutable-segment columns lazily behind an
opt-in instance config (default off)
add e465b529ece Reuse resolved index configs for lazy column
materialization
add 3dfb91fd9b6 Retain compact index configs for segment-specific loads
add 74f3279936e Serialize v1/v2 index buffer access for lazy
materialization and cover destroy vs in-flight
add ee7356fb5e4 Keep rebased source imports in project order
add 3a3b0933991 Keep lazy materialization compatible with the segment read
lifecycle
add d51a3d62695 Use current OpenStruct config constructors in materializer
tests
add 9576f825d22 Keep nested and sparse settings distinct during config
sharing
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (73bf3edd64e)
\
N -- N -- N refs/heads/xiangfu0/data-3221-6-lazy-materialization
(9576f825d22)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/workflows/pinot_compatibility_checks.yml | 8 +-
.github/workflows/pinot_compatibility_tests.yml | 4 +-
...ulti_stage_query_engine_compatibility_tests.yml | 4 +-
.../common/utils/config/QueryOptionsUtils.java | 21 +
.../core/util/ControllerZkHelixUtilsTest.java | 6 +
.../apache/pinot/core/common/BlockDocIdSet.java | 95 ++++-
.../data/manager/realtime/ConsumerCoordinator.java | 27 ++
.../realtime/RealtimeSegmentDataManager.java | 21 +-
.../dociditerators/MVScanDocIdIterator.java | 55 +--
.../pinot/core/operator/docidsets/AndDocIdSet.java | 121 +++++-
.../core/operator/docidsets/EmptyDocIdSet.java | 7 +
.../operator/docidsets/ExpressionDocIdSet.java | 5 +
.../core/operator/docidsets/MVScanDocIdSet.java | 5 +
.../core/operator/docidsets/MatchAllDocIdSet.java | 7 +
.../pinot/core/operator/docidsets/NotDocIdSet.java | 47 +++
.../pinot/core/operator/docidsets/OrDocIdSet.java | 90 +++-
.../core/operator/docidsets/SVScanDocIdSet.java | 5 +
.../core/operator/filter/AndFilterOperator.java | 12 +-
.../operator/filter/BaseColumnFilterOperator.java | 5 +-
.../operator/filter/CombinedFilterOperator.java | 7 +-
.../core/operator/filter/FilterOperatorUtils.java | 2 +-
.../core/operator/filter/MapFilterOperator.java | 7 +-
.../operator/filter/TextMatchFilterOperator.java | 8 +-
.../core/plan/maker/InstancePlanMakerImplV2.java | 41 ++
.../function/AggregationFunctionUtils.java | 3 +-
.../core/query/request/context/QueryContext.java | 14 +
.../manager/realtime/ConsumerCoordinatorTest.java | 25 ++
.../realtime/RealtimeSegmentDataManagerTest.java | 84 ++++
.../docidsets/AndDocIdSetPushdownTest.java | 465 +++++++++++++++++++++
.../function/BaseTransformFunctionTest.java | 5 +-
.../plan/maker/AndRestrictionPushdownModeTest.java | 129 ++++++
.../utils/exprminmax/ExprMinMaxObjectTest.java | 3 +
.../AndRestrictionPushdownNullQueriesTest.java | 201 +++++++++
.../queries/AndRestrictionPushdownQueriesTest.java | 109 +++++
.../pinot/queries/TextSearchQueriesTest.java | 18 +
.../tests/custom/GroupByTrimmingTest.java | 2 +-
.../BaseMultiClusterIntegrationTest.java | 8 +-
.../perf/BenchmarkAndRestrictionPushdown.java | 208 +++++++++
.../immutable/ImmutableSegmentImpl.java | 22 +-
.../io/compression/ZstandardDecompressor.java | 2 +-
.../index/readers/text/LuceneTextIndexReader.java | 123 ++++--
.../upsert/BasePartitionUpsertMetadataManager.java | 29 +-
.../pinot/segment/local/upsert/DocIdsSnapshot.java | 130 ++++++
.../upsert/PartitionUpsertMetadataManager.java | 5 +
.../segment/local/utils/LuceneTextIndexUtils.java | 89 ++++
.../readers/text/LuceneTextIndexCountTest.java | 289 +++++++++++++
.../BasePartitionUpsertMetadataManagerTest.java | 58 ++-
...rrentMapPartitionUpsertMetadataManagerTest.java | 3 +-
.../segment/local/upsert/DocIdsSnapshotTest.java | 136 ++++++
.../mutable/ThreadSafeMutableRoaringBitmap.java | 16 +-
.../index/reader/MultiColumnTextIndexReader.java | 19 +
.../segment/spi/index/reader/TextIndexReader.java | 18 +
.../pinot/server/api/resources/TablesResource.java | 30 +-
.../pinot/server/api/TablesResourceTest.java | 54 +++
.../apache/pinot/spi/utils/CommonConstants.java | 25 ++
pom.xml | 12 +-
56 files changed, 2797 insertions(+), 147 deletions(-)
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/operator/docidsets/AndDocIdSetPushdownTest.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/plan/maker/AndRestrictionPushdownModeTest.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/queries/AndRestrictionPushdownNullQueriesTest.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/queries/AndRestrictionPushdownQueriesTest.java
create mode 100644
pinot-perf/src/main/java/org/apache/pinot/perf/BenchmarkAndRestrictionPushdown.java
create mode 100644
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/DocIdsSnapshot.java
create mode 100644
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment/index/readers/text/LuceneTextIndexCountTest.java
create mode 100644
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/upsert/DocIdsSnapshotTest.java
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
