tkhurana commented on code in PR #2411:
URL: https://github.com/apache/phoenix/pull/2411#discussion_r3113387227
##########
phoenix-core-server/src/main/java/org/apache/phoenix/mapreduce/PhoenixSyncTableInputFormat.java:
##########
@@ -94,15 +101,36 @@ public List<InputSplit> getSplits(JobContext context)
throws IOException, Interr
} catch (SQLException e) {
throw new RuntimeException(e);
}
- if (completedRegions.isEmpty()) {
- LOGGER.info("No completed regions for table {} - processing all {}
splits", tableName,
- allSplits.size());
- return allSplits;
- }
List<InputSplit> unprocessedSplits = filterCompletedSplits(allSplits,
completedRegions);
LOGGER.info("Found {} completed mapper regions for table {}, {}
unprocessed splits remaining",
completedRegions.size(), tableName, unprocessedSplits.size());
+
+ boolean enableSplitCoalescing =
+ conf.getBoolean(PhoenixSyncTableTool.PHOENIX_SYNC_TABLE_SPLIT_COALESCING,
+ PhoenixSyncTableTool.DEFAULT_PHOENIX_SYNC_TABLE_SPLIT_COALESCING);
+ LOGGER.info("Split coalescing enabled: {}, for table {}",
enableSplitCoalescing, tableName);
+
+ if (enableSplitCoalescing && unprocessedSplits.size() > 1) {
+ try (Connection conn = ConnectionUtil.getInputConnection(conf)) {
+ PhoenixConnection pConn = conn.unwrap(PhoenixConnection.class);
+ byte[] physicalTableName =
pConn.getTable(tableName).getPhysicalName().getBytes();
+ org.apache.hadoop.hbase.client.Connection hbaseConn =
+ pConn.getQueryServices().getAdmin().getConnection();
Review Comment:
There is a resource leak here. Admin object is not being closed. Also
hconnection is also not being closed. We already have similar APIs available to
us in ConnectionQueryServicesImpl class. My suggestion would be to use
`getTableRegionLocation`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
