[ https://issues.apache.org/jira/browse/DRILL-6660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565708#comment-16565708 ]
salim achouche commented on DRILL-6660: --------------------------------------- During the analysis it was realized that the Batch Sizing functionality (for exchange operators) is not enough on its own: * Exchanges are usually involved with MxN communication; using a 16MB (default) batch size for each output / input batch will not scaleĀ * Instead, the analysis should include the exchange topology, performance implications, communication timing (should we fill an output batch even if it means a long delay?) Thus, our recommendation is to club Resource Management, Batch Sizing, and Performance Tuning within a single initiative to avoid regression and achieve the desired goal of more scalability. > Exchange operators Analysis > --------------------------- > > Key: DRILL-6660 > URL: https://issues.apache.org/jira/browse/DRILL-6660 > Project: Apache Drill > Issue Type: Sub-task > Components: Execution - Flow > Reporter: salim achouche > Assignee: salim achouche > Priority: Major > Fix For: 1.15.0 > > > Analysis of what will it take to batch size the exchange operators. -- This message was sent by Atlassian JIRA (v7.6.3#76005)