xinyiZzz commented on PR #10080: URL: https://github.com/apache/incubator-doris/pull/10080#issuecomment-1153711447
> > @jacktengg hi, why removed the logic that multiple filters only wait 1s at most. Is there a performance loss in the test? > > This is a TODO for the first implementation of the runtime filter. Currently, `ScanNode` will wait at most n seconds, and n is equal to the number of runtime filters, which will cause serious performance loss in extreme cases. > > No enough test have been done for the change of 'multiple filters only wait 1s at most', so the performance impact is not sure for mutiple runtime filters in on ScanNode. > > E.g., waiting a RF for 1.5 seconds and it can filter a lot of rows, VS waiting for 1 seconds and the RF is missed, I have not tested this situation, not sure which is better. > > So in this PR will not include the change of RF waiting. If waiting for 1 second causes RF to be missed, `runtime_filter_wait_time_ms` should be increased. Otherwise it will only wait 1.5 seconds if `ScanNode`apply is greater than 1 RF, which is uncontrollable. `Number of RFs` is not directly related to `Waiting time per RF`. I think 'multiple filters only wait 1s at most' is a viable optimization, maybe do some testing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
