OrderedPartitionedKVEdgeConfig is only for ScatterGather.
UnorderedKVEdgeConfig is for broadcast and 1-1 edge.
Usually broadcast output is not sorted
OrderedPartitionedKVEdgeConfig.java
public EdgeProperty createDefaultEdgeProperty() {
EdgeProperty edgeProperty =
EdgeProperty.create
Thanks Jianfeng.
if I use OrderedPartitionedKVEdgeConfig , and use broadcast edge property
with parallism 1, does that Tez also do shuffle and group before broadcast?
On Fri, Mar 27, 2015 at 12:47 PM, Jianfeng (Jeff) Zhang <
jzh...@hortonworks.com> wrote:
>
> Hi Azuryy,
>
> I check the sourc
Hi Azuryy,
I check the source code, the last vertex Writer do an in-memory sort in its
processor. So in this case broadcast is possible (also require parallelism to
be 1)
If the edge is scatter-gather and use OrderedPartitionedKVEdgeConfig, then the
in-memory sort is not necessary because the
Hi,
please look through this simple code:
https://github.com/sequenceiq/sequenceiq-samples/blob/master/tez-topk/src/main/java/com/sequenceiq/tez/topk/TopK.java
why they create a broadcast edge property from SUM to WRITER? what about
default edge property? (scatter-gather)
Yes. Hitesh, I mainly refer to the Tez docs, just found this blog
deprecated occasionaly.
On Thu, Mar 26, 2015 at 11:41 PM, Hitesh Shah wrote:
> Thanks for the heads up, Azuryy. I will pass the information along to the
> relevant folks at Hortonworks. And yes, you are right. At some point, we
Thanks for the heads up, Azuryy. I will pass the information along to the
relevant folks at Hortonworks. And yes, you are right. At some point, we merged
the code into the common TezClient so that it became easier for the user to
toggle session-mode on/off via configuration and not have to write
Hi,
I'm the original victim :) just sent up TEZ-2237.
Sent as much logs as was practical up to this point; can supply on a
direct basis as much as required to nail the issue.
To give some context: these two failing DAG are part of a meta-DAG
comprised of 20 distinct DAG, all generated throug
Hi,
Please review this blog,
http://www.hortonworks.com/blog/introducing-tez-sessions/
espcially for the section
Using Tez Sessions
TezSession is removed, right?
if we want to re-use session in Tez, just do like this:
TezConfiguration tezConf = new TezConfiguration();
tezConf.setBoolean(TezConfi
