[
https://issues.apache.org/jira/browse/IMPALA-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16983082#comment-16983082
]
ASF subversion and git services commented on IMPALA-9126:
---------------------------------------------------------
Commit b571aa9b745c4c27542d509b31e6748e10838fca in impala's branch
refs/heads/master from Tim Armstrong
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=b571aa9 ]
IMPALA-9126: part 2: no hash join probe structures in build
This is actually independent of part 1, and can
be merged ahead of it if needed.
This cleans a up a bit of tech debt, where the hash
join builder allocated probe-side streams. This was
implemented before we had reliable memory reservations.
Now we can simply transfer reservation.
The reason things are this way is because the separation
of PhjBuilder from PartitionedHashJoinNode (IMPALA-3567)
happened before we switched to the new BufferPool
(IMPALA-4674). It wasn't possible to reliably
transfer reservations, instead the workaround of
allocating and transferring probe streams was
necessary.
After this change, PartitionedHashJoinBuilder does
not explicitly touch any probe-side data structures.
There is still some implicit sharing of things like
the buffer pool client, which is expected as long
as the builder belongs to the ExecNode.
Testing:
Ran exhaustive tests. We should already have adequate coverage for
spilling and non-spilling hash joins.
Change-Id: I0065f7f44f44f02b7616b1f694178ca42341c42d
Reviewed-on: http://gerrit.cloudera.org:8080/14716
Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
> Cleanly separate build and probe state in hash join node
> --------------------------------------------------------
>
> Key: IMPALA-9126
> URL: https://issues.apache.org/jira/browse/IMPALA-9126
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Major
> Labels: multithreading
>
> As a precursor to IMPALA-4224, we should clean up the hash join
> implementation so that the build and probe state is better separated. The
> builder should not deal with probe side data structures (like the probe
> streams that it allocates) and all accesses to the build-side data structures
> should go through as narrow APIs as possible.
> The nested loop join is already pretty clean.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
