This is an automated email from the ASF dual-hosted git repository.
xiangfu0 pushed a change to branch xiangfu0/data-3221-12-parallel-value-pruning
in repository https://gitbox.apache.org/repos/asf/pinot.git
discard 8c6e53471d5 DATA-3221 (14): prune values across the query executor
discard 6f1c69123ae DATA-3221 (13): prune segments from column metadata
instead of materializing them
discard 4a3d0d6549d DATA-3221 (12): make the new column-name accessor sorted
and null-safe
discard 2676ba99227 DATA-3221 (11): list physical columns without building the
segment schema
discard a2313efa6a3 DATA-3221 (10): publish the segment column arrays as one
immutable pair
discard 2db3a4c08b2 DATA-3221 (10): hold segment column metadata in sorted
arrays, derive the map on demand
discard 2c01e87674f DATA-3221 (8+9 follow-up): drop the shared shape, pack
bitsPerElement instead
discard 2af6280d14d DATA-3221 (8+9): shared column shape and primitive min/max
in ColumnMetadataImpl
discard 6a5f050985c Slim ColumnMetadataImpl to 72 bytes and derive the
per-segment Schema lazily
discard 52ee4691f91 Materialize immutable-segment columns lazily behind an
opt-in instance config (default off)
discard 1ad2b25ac36 Share segment-derived FieldSpec instances across segments
through a weak interner
discard 33a59e34a38 Fold PhysicalColumnIndexContainer's IndexTypeMap into a
presence mask plus dense readers
discard 95907890be1 Delegate immutable DataSourceMetadata to ColumnMetadata
instead of snapshotting it
discard c566594f519 Canonicalize the default null value and intern per-column
strings at metadata parse time
discard 89ea2d28abd Allocate ColumnMetadataImpl index sizes lazily and skip
the index_map lookup without an index dir
add 5e05b9cf062 Make segment-replace timeouts and endReplaceSegments
convergence wait configurable (#19278)
add e83b59f0d8e Release MSE operator buffers on error and cancel, not only
on end of stream (#19462)
add a8476f988cf Stop a lingering receiving mailbox from pinning the
receive operator (#19465)
add cab13822b26 [MSE] Add ExplainableOperator so alternative leaf-stage
engines appear in EXPLAIN (#19447)
add 6aaa0405566 Add broker startup pre-connect for broker-to-server
channels (SSE) (#19407)
add 6c773871d9d Bump org.jline:jline from 4.4.1 to 4.4.2 (#19498)
add 32b9d6bf7a5 Bump org.slf4j:slf4j-bom from 2.0.18 to 2.0.19 (#19497)
add ec0507c764d Bump com.mchange:c3p0 from 0.14.1 to 0.14.2 (#19496)
add a88fa8750a2 Bump software.amazon.awssdk:bom from 2.54.9 to 2.54.12
(#19495)
add 16227abd341 Bump org.checkerframework:checker-qual from 4.2.2 to 4.2.3
(#19494)
add b9ee686b945 Bump io.grpc:grpc-bom from 1.83.1 to 1.84.0 (#19491)
add 7712e67bc9c Bump parquet.version from 1.18.0 to 1.18.1 (#19492)
add ff7aefbef30 Bump org.roaringbitmap:RoaringBitmap from 1.6.20 to 1.6.21
(#19493)
add 7f024b6b623 Release the consumer semaphore only after the segment's
upsert metadata is removed on offload (#19444)
add 49cf1be58b6 Bump com.google.cloud:libraries-bom from 26.87.0 to
26.88.0 (#19506)
add 51f24d8a476 Bump com.squareup.okio:okio-bom from 3.18.1 to 3.18.2
(#19507)
add 452f5a686e8 Bump com.h2database:h2 from 2.4.240 to 2.5.250 (#19508)
add 9d91a7bffb1 Bump software.amazon.awssdk:bom from 2.54.12 to 2.54.13
(#19509)
add 0e540de467a Use plain component names for the MultiValueLimit record
(#19482)
add 1f5b06c6001 Add free OpenRouter PR flow diagrams (#19502)
add 6fb1075c4de Sample resource usage while the DISTINCT cross-segment
merge grows (#19416)
add c6116095f7d Log a query on a single line (#19453)
add fb925d71340 Allocate ColumnMetadataImpl index sizes lazily and skip
the index_map lookup without an index dir
add b61c9233453 Canonicalize the default null value and intern per-column
strings at metadata parse time
add 2b73f0db6cf Delegate immutable DataSourceMetadata to ColumnMetadata
instead of snapshotting it
add 0bbde43bad4 Fold PhysicalColumnIndexContainer's IndexTypeMap into a
presence mask plus dense readers
add fdde54d032b Share segment-derived FieldSpec instances across segments
through a weak interner
add 19dbb53b3bd Materialize immutable-segment columns lazily behind an
opt-in instance config (default off)
add 289f83918e5 Slim ColumnMetadataImpl to 72 bytes and derive the
per-segment Schema lazily
add a43da48fc71 DATA-3221 (8+9): shared column shape and primitive min/max
in ColumnMetadataImpl
add 42fc38a698b DATA-3221 (8+9 follow-up): drop the shared shape, pack
bitsPerElement instead
add 624b8118c09 DATA-3221 (10): hold segment column metadata in sorted
arrays, derive the map on demand
add f0a20da161a DATA-3221 (10): publish the segment column arrays as one
immutable pair
add 187be1c87b4 DATA-3221 (11): list physical columns without building the
segment schema
add 1bad6fdfc20 DATA-3221 (12): make the new column-name accessor sorted
and null-safe
add 9a696892abe DATA-3221 (13): prune segments from column metadata
instead of materializing them
add 785926d7367 DATA-3221 (14): prune values across the query executor
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (8c6e53471d5)
\
N -- N -- N
refs/heads/xiangfu0/data-3221-12-parallel-value-pruning (785926d7367)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/scripts/pr_flow/README.md | 121 ++++
.github/scripts/pr_flow/main.py | 524 +++++++++++++++++
.github/scripts/pr_flow/model.py | 470 +++++++++++++++
.github/scripts/pr_flow/test_main.py | 572 ++++++++++++++++++
.github/scripts/pr_flow/test_model.py | 476 +++++++++++++++
.github/workflows/pr-flow-checks.yml | 55 ++
.../workflows/pr-flow-signal.yml | 26 +-
.github/workflows/pr-flow.yml | 134 +++++
.../broker/broker/helix/BaseBrokerStarter.java | 126 +++-
.../apache/pinot/broker/querylog/QueryLogger.java | 16 +-
.../requesthandler/BrokerRequestHandler.java | 10 +
.../BrokerRequestHandlerDelegate.java | 7 +
.../broker/requesthandler/ServerPreConnector.java | 196 +++++++
.../SingleConnectionBrokerRequestHandler.java | 46 ++
.../routing/manager/BaseBrokerRoutingManager.java | 5 +
.../manager/MultiClusterRoutingManager.java | 12 +
.../pinot/broker/querylog/QueryLoggerTest.java | 133 ++++-
.../requesthandler/ServerPreConnectorTest.java | 406 +++++++++++++
.../manager/MultiClusterRoutingManagerTest.java | 14 +
.../apache/pinot/common/metrics/BrokerTimer.java | 9 +
.../apache/pinot/common/utils/http/HttpClient.java | 10 +-
.../apache/pinot/controller/ControllerConf.java | 27 +
.../helix/core/PinotHelixResourceManager.java | 25 +-
.../pinot/controller/ControllerConfTest.java | 24 +
.../apache/pinot/core/common/MinionConstants.java | 4 +
.../realtime/RealtimeSegmentDataManager.java | 32 +-
.../distinct/table/BigDecimalDistinctTable.java | 6 +
.../query/distinct/table/BytesDistinctTable.java | 6 +
.../query/distinct/table/DoubleDistinctTable.java | 6 +
.../query/distinct/table/FloatDistinctTable.java | 6 +
.../query/distinct/table/IntDistinctTable.java | 6 +
.../query/distinct/table/LongDistinctTable.java | 6 +
.../distinct/table/MultiColumnDistinctTable.java | 7 +
.../query/distinct/table/StringDistinctTable.java | 6 +
.../apache/pinot/core/routing/RoutingManager.java | 11 +
.../apache/pinot/core/transport/QueryRouter.java | 26 +
.../pinot/core/transport/ServerChannels.java | 136 ++++-
.../realtime/RealtimeSegmentDataManagerTest.java | 111 ++++
.../table/DistinctTableMergeAccountingTest.java | 292 ++++++++++
.../pinot/core/transport/ServerChannelsTest.java | 95 +++
.../BrokerServerPreConnectIntegrationTest.java | 142 +++++
.../integration/tests/TlsIntegrationTest.java | 29 +
.../java/org/apache/pinot/minion/MinionConf.java | 6 +
.../org/apache/pinot/minion/MinionConfTest.java | 14 +
.../BaseMultipleSegmentsConversionExecutor.java | 3 +-
.../minion/tasks/SegmentConversionUtils.java | 17 +-
.../pinot/query/context/PlannerContextTest.java | 2 +-
.../apache/pinot/query/runtime/QueryRunner.java | 8 +-
.../query/runtime/operator/AggregateOperator.java | 33 +-
.../query/runtime/operator/AsofJoinOperator.java | 7 +-
.../query/runtime/operator/BaseJoinOperator.java | 6 +-
.../operator/BaseMailboxReceiveOperator.java | 21 +-
.../runtime/operator/ExplainableOperator.java | 56 ++
.../query/runtime/operator/HashJoinOperator.java | 7 +-
.../pinot/query/runtime/operator/LeafOperator.java | 12 +-
.../query/runtime/operator/MultiStageOperator.java | 60 ++
.../runtime/operator/NonEquiJoinOperator.java | 13 +-
.../query/runtime/operator/RepeatOperator.java | 26 +
.../pinot/query/runtime/operator/SortOperator.java | 37 +-
.../operator/SortedMailboxReceiveOperator.java | 27 +-
.../runtime/operator/set/BinarySetOperator.java | 19 +-
.../query/runtime/operator/set/UnionOperator.java | 18 +-
.../plan/pipeline/PipelineBreakerOperator.java | 10 +
.../operator/MailboxReceiveOperatorTest.java | 41 ++
.../operator/OperatorBufferReleaseTest.java | 640 +++++++++++++++++++++
.../operator/SortedMailboxReceiveOperatorTest.java | 34 ++
.../operator/set/SetOperatorBufferReleaseTest.java | 228 ++++++++
.../indexsegment/mutable/MutableSegmentImpl.java | 8 +-
.../apache/pinot/spi/utils/CommonConstants.java | 15 +
pom.xml | 22 +-
70 files changed, 5660 insertions(+), 101 deletions(-)
create mode 100644 .github/scripts/pr_flow/README.md
create mode 100644 .github/scripts/pr_flow/main.py
create mode 100644 .github/scripts/pr_flow/model.py
create mode 100644 .github/scripts/pr_flow/test_main.py
create mode 100644 .github/scripts/pr_flow/test_model.py
create mode 100644 .github/workflows/pr-flow-checks.yml
copy docker/images/pinot-superset/Dockerfile =>
.github/workflows/pr-flow-signal.yml (66%)
create mode 100644 .github/workflows/pr-flow.yml
create mode 100644
pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/ServerPreConnector.java
create mode 100644
pinot-broker/src/test/java/org/apache/pinot/broker/requesthandler/ServerPreConnectorTest.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/query/distinct/table/DistinctTableMergeAccountingTest.java
create mode 100644
pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/BrokerServerPreConnectIntegrationTest.java
create mode 100644
pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/ExplainableOperator.java
create mode 100644
pinot-query-runtime/src/test/java/org/apache/pinot/query/runtime/operator/OperatorBufferReleaseTest.java
create mode 100644
pinot-query-runtime/src/test/java/org/apache/pinot/query/runtime/operator/set/SetOperatorBufferReleaseTest.java
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
