ankitsultana commented on code in PR #17502:
URL: https://github.com/apache/pinot/pull/17502#discussion_r2692123467
##########
pinot-core/src/main/java/org/apache/pinot/core/operator/blocks/results/DistinctResultsBlock.java:
##########
@@ -33,6 +33,8 @@ public class DistinctResultsBlock extends BaseResultsBlock {
private final QueryContext _queryContext;
private DistinctTable _distinctTable;
+ private boolean _liteLeafLimitReached;
Review Comment:
I think adding this in individual results block will not work.
Instead can you consider tracking this in the InstanceResponseOperator? I'd
recommend running a quickstart and setting breakpoints to understand which
exact operators are called for the leaf stage in lite mode in the servers.
##########
pinot-common/src/main/java/org/apache/pinot/common/datatable/DataTable.java:
##########
@@ -151,11 +151,13 @@ enum MetadataKey {
// Needed so that we can track workload name in Netty channel response.
WORKLOAD_NAME(40, "workloadName", MetadataValueType.STRING),
// Needed so that we can track query id in Netty channel response.
- QUERY_ID(41, "queryId", MetadataValueType.STRING);
+ QUERY_ID(41, "queryId", MetadataValueType.STRING),
Review Comment:
datatable is used by v1 engine so we can leave it be
##########
pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/plan/server/ServerPlanRequestUtils.java:
##########
@@ -148,11 +148,85 @@ private static void
constructPinotQueryPlan(ServerPlanRequestContext serverConte
PinotQuery pinotQuery = serverContext.getPinotQuery();
// attach leaf node limit it not set
Integer leafNodeLimit =
QueryOptionsUtils.getMultiStageLeafLimit(requestMetadata);
+ int prevLimit = pinotQuery.getLimit();
pinotQuery.setLimit(leafNodeLimit != null ? leafNodeLimit :
DEFAULT_LEAF_NODE_LIMIT);
+ // Tag provenance if a leaf cap (explicit or lite-mode fallback) tightened
the previous limit.
+ tagLeafLimitProvenanceIfTightened(pinotQuery, requestMetadata, prevLimit);
// visit the plan and create PinotQuery and determine the leaf stage
boundary PlanNode.
ServerPlanRequestVisitor.walkPlanNode(stagePlan.getRootNode(),
serverContext);
}
+ /**
+ * Tags queryOptions["leafLimitProvenance"]="LITE_CAP" if either:
+ * - an explicit multiStageLeafLimit (in request metadata or query
options) would tighten the previous LIMIT; or
Review Comment:
this is interesting. I suppose you want to track if results were truncated
by the "multiStageLeafLimit". This will be useful even for non-lite mode
queries. Correct?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
