[ https://issues.apache.org/jira/browse/STORM-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans resolved STORM-2210. ---------------------------------------- Resolution: Fixed Assignee: Kevin Peek Fix Version/s: 1.0.3 1.1.0 2.0.0 Thanks [~kevpeek], I merged this into master, 1.x-branch and 1.0.x-branch. Keep up the good work. > ShuffleGrouping does not produce even distribution > -------------------------------------------------- > > Key: STORM-2210 > URL: https://issues.apache.org/jira/browse/STORM-2210 > Project: Apache Storm > Issue Type: Bug > Components: storm-core > Affects Versions: 1.0.2 > Reporter: Kevin Peek > Assignee: Kevin Peek > Priority: Critical > Fix For: 2.0.0, 1.1.0, 1.0.3 > > Time Spent: 3.5h > Remaining Estimate: 0h > > When testing the ShuffleGrouping in a multithreaded environment, it produces > an extremely uneven distribution. > This appears to be a result of the Collection.shuffle call here. > https://github.com/apache/storm/blob/1.0.x-branch/storm-core/src/jvm/org/apache/storm/grouping/ShuffleGrouping.java#L58 > Because current was set to zero before the shuffle, other threads are able to > access the arrayList while it is being shuffled. > Stephen's gist here includes a test that results in a very uneven > distribution of taskIds from the ShuffleGrouping: > https://gist.github.com/Crim/61537958df65a5e13b3844b2d5e28cde > I would have expected the taskIds from the ShuffleGrouping to be almost > uniformly distributed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)