okumin commented on code in PR #5492:
URL: https://github.com/apache/hive/pull/5492#discussion_r1834157590
##########
ql/src/java/org/apache/hadoop/hive/ql/optimizer/SharedWorkOptimizer.java:
##########
@@ -248,6 +249,32 @@ public ParseContext transform(ParseContext pctx) throws
SemanticException {
return pctx;
}
+ private static List<List<TableScanOperator>>
groupTableScanOperators(List<Entry<String, Long>> sortedTables,
+ ArrayListMultimap<String, TableScanOperator> tableNameToOps, int
batchSize) {
+ final List<List<TableScanOperator>> batches = new ArrayList<>();
+ for (Entry<String, Long> tablePair : sortedTables) {
+ final String tableName = tablePair.getKey();
+ final List<TableScanOperator> scans = tableNameToOps.get(tableName);
+ if (batchSize == -1) {
+ batches.add(scans);
+ continue;
+ }
+
+ final int limit = scans.size();
+ int from = 0;
+ while (from != limit) {
+ final int to = Math.min(limit, from + batchSize);
+ // We have to copy the list because it is mutated later
+ final List<TableScanOperator> subList = new
ArrayList<>(scans.subList(from, to));
+ Preconditions.checkState(!subList.isEmpty());
Review Comment:
The level of intention of this assertion might be lower than what you are
thinking. I just added it to ensure I correctly implemented L265-L272 because I
am worried too much. I think we can remove the line basically. Will do
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org
