In scenario “b”, set the parallelism for the spout before the `shuffle()` operation.
Trident topologies compile down to regular spouts and bolts. Partitioning
operations like `shuffle()`, etc. define the bolt boundaries and hence where
parallelism hints take effect.
-Taylor
On Apr 29, 2015, at 6:52 AM, nitin sharma <kumarsharma.ni...@gmail.com> wrote:
> Hi Team,
>
> I am trying to understand ParallelismHint in Trident Topology but somehow not
> getting anywhere close.. It will be great if someone can help me.... Also,
> kindly explain the difference between MasterCorrdinator and
> SpoutCoordinator...
>
> Things that i have tried so far:
>
> a. I created a simple trident topology with following configuration.. When
> checked the UI, i see --
> $mastercoord-bg0 1 1
>
> $spoutcoord-spout0 1 1
> __acker 3 3
> b-0 4 4
> spout 1 1
>
> topology.newStream("spout1", spout).parallelismHint(2)
> .each(new Fields(new String[] { "sentence" }),
> new Split(), new Fields(new String[] { "word" })).parallelismHint(4);
>
> Query: Ideally i should see 2 executor for spout and 4 executor for bolt.. i
> can see proper setting for bolt but not for Spout
>
>
> b. then i modified my code to create another topology with following
> configuration to increase spout count. But, no changes were observed in spout
> count
> $mastercoord-bg0 1 1
>
> $spoutcoord-spout0 1 1
> __acker 3 3
> b-0 2 2
> spout 1 1
>
>
> Stream st = topology.newStream("spout1", spout)
> .shuffle()
> .each(new Fields("sentence"),
> new Split(), new Fields("word")).parallelismHint(4);
> st.parallelismHint(2);
>
> Regards,
> Nitin Kumar Sharma.
>
signature.asc
Description: Message signed with OpenPGP using GPGMail
