gortiz commented on code in PR #12704:
URL: https://github.com/apache/pinot/pull/12704#discussion_r1584415369
##########
pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/LeafStageTransferableBlockOperator.java:
##########
@@ -135,14 +160,128 @@ protected TransferableBlock getNextBlock()
}
}
- private TransferableBlock constructMetadataBlock() {
- // All data blocks have been returned. Record the stats and return EOS.
- Map<String, String> executionStats = _executionStats;
+ private void mergeExecutionStats(@Nullable Map<String, String>
executionStats) {
if (executionStats != null) {
- OperatorStats operatorStats = _opChainStats.getOperatorStats(_context,
getOperatorId());
- operatorStats.recordExecutionStats(executionStats);
+ for (Map.Entry<String, String> entry : executionStats.entrySet()) {
+ DataTable.MetadataKey key =
DataTable.MetadataKey.getByName(entry.getKey());
+ if (key == null) {
+ LOGGER.debug("Skipping unknown execution stat: {}", entry.getKey());
+ continue;
+ }
+ switch (key) {
+ case UNKNOWN:
+ LOGGER.debug("Skipping unknown execution stat: {}",
entry.getKey());
+ break;
+ case TABLE:
Review Comment:
I've removed OPERATOR_ID because there was no use for it.
Right now the TABLE stat is used in `MultiStageBrokerRequestHandler` to
build the `stageStats` json object (the one that is shown in the description of
this PR). There we use the Table stat to introduce the name of the table
scanned when the json node of the leaf stage is created. In theory we don't
need this here, we can infer the name of the table from the PlanNode, but there
are cases where the PlanNode tree is quite different to the actual physical
plan, so it is more difficult to do than expected.
There was another place where it was used, in
`ResourceBasedQueriesTest.testQueryTestCasesWithMetadata`, but I ended up
finding a way to do not need it. The previous version of the code was honestly
simpler, but I guess the new version is better because ideally we would like to
remove the TABLE stat.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
