[ https://issues.apache.org/jira/browse/IMPALA-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989357#comment-16989357 ]
ASF subversion and git services commented on IMPALA-9126: --------------------------------------------------------- Commit 17e534e3164a88c4f1da85b39e8245d1ef079bd6 in impala's branch refs/heads/master from Tim Armstrong [ https://gitbox.apache.org/repos/asf?p=impala.git;h=17e534e ] IMPALA-9126: part 4: hash join builder manages spilling This is the final patch for IMPALA-9126. This will allow the many:1 relationship of probe:build partitions that we need for the shared join build. Key changes: * Builder picks the next spilled partition to process. * Partitions are identified by unique ID so can be decoupled between build and probe. * unique_ptr is used to manage build partitions. This helps document the lifecycle of the partitions better, particularly when they are handed off to PartitionedHashJoinNode. Testing: * Ran exhaustive tests. * Ran a single node TPC-H and TPC-DS stress test with 1000 queries. Perf: Ran a single node TPC-H 30 test against master from before IMPALA-9126 changes. No significant perf change. Change-Id: I6de5f62e3eacf80f72c8ea0ed8cba012f0f53c90 Reviewed-on: http://gerrit.cloudera.org:8080/14790 Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> > Cleanly separate build and probe state in hash join node > -------------------------------------------------------- > > Key: IMPALA-9126 > URL: https://issues.apache.org/jira/browse/IMPALA-9126 > Project: IMPALA > Issue Type: Improvement > Components: Backend > Reporter: Tim Armstrong > Assignee: Tim Armstrong > Priority: Major > Labels: multithreading > > As a precursor to IMPALA-4224, we should clean up the hash join > implementation so that the build and probe state is better separated. The > builder should not deal with probe side data structures (like the probe > streams that it allocates) and all accesses to the build-side data structures > should go through as narrow APIs as possible. > The nested loop join is already pretty clean. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org