ishitadixit-dot opened a new pull request, #2623: URL: https://github.com/apache/phoenix/pull/2623
## Problem `PhoenixInputFormat.getSplits()` emits one PhoenixInputSplit per region (or per stats guidepost). On large or salted tables this fans out into a very large number of mappers, and because a table's regions for a given RegionServer are scanned together, it can hot-spot individual RegionServers. The PhoenixSyncTable tool already solved this inline, but the logic wasn't reusable and made a live region-location RPC per split. ## Change Introduced `RegionServerSplitCoalescer`, which merges all of a table's region-boundary splits that live on the same RegionServer into a single PhoenixInputSplit (one mapper per RegionServer), concatenating their scans. - Opt-in, off by default: Added flag `PhoenixInputFormat.SPLIT_COALESCING_ENABLED `(phoenix.mapreduce.split.coalescing.enabled, default false). Behavior is unchanged for every existing consumer unless the flag is set on the job's Configuration. - Grouping by RegionServer identity: Grouped on the host:port identity now stamped on each split at generation time (PhoenixInputSplit#getRegionServerName()) — no region-location RPCs. Used host:port rather than the bare hostname so two RegionServers on the same host stay in separate splits. Fallbacks: bare hostname → UNKNOWN_SERVER. - Stats-based splitting disabled when coalescing is on: Turned off stats splitting before coalescing (mirroring PhoenixSyncTableTool), since fragmenting regions into sub-scans and then coalescing them would be self-defeating. - Correctness guard: Added `coalesceWithGuard`, which verifies the total scan count is preserved and falls back to the original region-granular splits on any mismatch or exception (InterruptedException is re-propagated). Coalescing is an optimization; it never changes which rows are scanned. - Refactor: Moved `PhoenixSyncTableInputFormat` onto the shared coalescer (gated on its existing key PHOENIX_SYNC_TABLE_SPLIT_COALESCING), replacing its inline implementation and removing a per-split region-location RPC. Behavior is equivalent, plus the new scan-count guard. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
