Ok, I got it, thanks Bikas.
On Sat, Mar 28, 2015 at 7:41 AM, Bikas Saha <[email protected]> wrote: > Hi, > > > > In practice, if the consumer vertex parallelism = 1 then scatter-gather > and broadcast will behave the same since a 1-partition scatter-gather is > equivalent to a broadcast J > > > > So if your consumer vertex parallelism = 1, then you can use > OrderedPartitionedKVEdgeConfig and it will work like a broadcast even > though it is using scatter-gather. You don’t need to specify broadcast > property. > > > > Bikas > > > > *From:* Azuryy Yu [mailto:[email protected]] > *Sent:* Thursday, March 26, 2015 11:59 PM > *To:* [email protected] > *Subject:* Re: Why broadcast Edge property? > > > > Sorry I didn't make my question clear, I mean if I use > OrderedPartitionedKVEdgeConfig > , and use broadcast edge property with parallism 1, can I get a correct > result or Tez will complain exception? > > > > On Fri, Mar 27, 2015 at 2:52 PM, Jianfeng (Jeff) Zhang < > [email protected]> wrote: > > > > OrderedPartitionedKVEdgeConfig is only for ScatterGather. > UnorderedKVEdgeConfig > is for broadcast and 1-1 edge. > > Usually broadcast output is not sorted > > > > > > OrderedPartitionedKVEdgeConfig.java > > > > public EdgeProperty createDefaultEdgeProperty() { > > EdgeProperty edgeProperty = > EdgeProperty.create(EdgeProperty.DataMovementType.SCATTER_GATHER, > > EdgeProperty.DataSourceType.PERSISTED, > EdgeProperty.SchedulingType.SEQUENTIAL, > > OutputDescriptor.create( > > getOutputClassName()).setUserPayload(getOutputPayload()), > > InputDescriptor.create( > > getInputClassName()).setUserPayload(getInputPayload())); > > Utils.setEdgePropertyHistoryText(this, edgeProperty); > > return edgeProperty; > > } > > > > Best Regard, > > Jeff Zhang > > > > > > *From: *Azuryy Yu <[email protected]> > *Reply-To: *"[email protected]" <[email protected]> > *Date: *Friday, March 27, 2015 at 2:45 PM > *To: *"[email protected]" <[email protected]> > *Subject: *Re: Why broadcast Edge property? > > > > OrderedPartitionedKVEdgeConfig > > >
