[jira] [Updated] (ARROW-17115) [C++] HashJoin fails if it encounters a batch with more than 32Ki rows
[ https://issues.apache.org/jira/browse/ARROW-17115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Keane updated ARROW-17115: --- Fix Version/s: 9.0.0 > [C++] HashJoin fails if it encounters a batch with more than 32Ki rows > -- > > Key: ARROW-17115 > URL: https://issues.apache.org/jira/browse/ARROW-17115 > Project: Apache Arrow > Issue Type: Bug > Components: C++ >Reporter: Weston Pace >Assignee: Weston Pace >Priority: Blocker > Fix For: 9.0.0 > > > The new swiss join assumes that batches are being broken according to the > morsel/batch model and it assumes those batches have, at most, 32Ki rows > (signed 16-bit indices are used in various places). > However, we are not currently slicing all of our inputs to batches this > small. This is causing conbench to fail and would likely be a problem with > any large inputs. > We should fix this by slicing batches in the engine to the appropriate > maximum size. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ARROW-17115) [C++] HashJoin fails if it encounters a batch with more than 32Ki rows
[ https://issues.apache.org/jira/browse/ARROW-17115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-17115: --- Labels: pull-request-available (was: ) > [C++] HashJoin fails if it encounters a batch with more than 32Ki rows > -- > > Key: ARROW-17115 > URL: https://issues.apache.org/jira/browse/ARROW-17115 > Project: Apache Arrow > Issue Type: Bug > Components: C++ >Reporter: Weston Pace >Assignee: Weston Pace >Priority: Blocker > Labels: pull-request-available > Fix For: 9.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > The new swiss join assumes that batches are being broken according to the > morsel/batch model and it assumes those batches have, at most, 32Ki rows > (signed 16-bit indices are used in various places). > However, we are not currently slicing all of our inputs to batches this > small. This is causing conbench to fail and would likely be a problem with > any large inputs. > We should fix this by slicing batches in the engine to the appropriate > maximum size. -- This message was sent by Atlassian Jira (v8.20.10#820010)
