tarun11Mavani commented on code in PR #18475:
URL: https://github.com/apache/pinot/pull/18475#discussion_r3354908631
##########
pinot-core/src/main/java/org/apache/pinot/core/operator/BaseOperator.java:
##########
@@ -55,10 +59,36 @@ public ExplainInfo getExplainInfo() {
protected void checkTermination() {
QueryThreadContext.checkTermination(this::getExplainName);
+ checkScanBasedKilling();
}
protected void checkTerminationAndSampleUsage() {
QueryThreadContext.checkTerminationAndSampleUsage(this::getExplainName);
+ checkScanBasedKilling();
Review Comment:
Suggestion to explore.
Instead of registering the scan context in each operator, what if we add
that logic inside `checkTerminationAndSampleUsage`, where we first update the
context and than call `checkScanBasedKilling`
##########
pinot-core/src/main/java/org/apache/pinot/core/operator/query/DistinctOperator.java:
##########
@@ -60,6 +61,12 @@ protected DistinctResultsBlock getNextBlock() {
ValueBlock valueBlock;
while ((valueBlock = _projectOperator.nextBlock()) != null) {
_numDocsScanned += valueBlock.getNumDocs();
+ QueryScanCostContext scanCost = getScanCostContext();
+ if (scanCost != null) {
+ scanCost.addDocsScanned(valueBlock.getNumDocs());
+ scanCost.addEntriesScannedPostFilter(
+ (long) valueBlock.getNumDocs() *
_projectOperator.getNumColumnsProjected());
+ }
Review Comment:
:nit: We could move this logic inside BaseOperator and call the method in
each operator. Will make it easier to push more data into context easier in
future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
