[jira] [Updated] (MAPREDUCE-6245) Fixed split shuffling.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
lbkzman updated MAPREDUCE-6245:
---
Affects Version/s: 2.6.0
Release Note:
index 72b47f2..8b89782 100644
--- src/java/org/apache/hadoop/mapreduce/lib/partition/InputSampler.java
+++ src/java/org/apache/hadoop/mapreduce/lib/partition/InputSampler.java
@@ -203,12 +203,8 @@ public class InputSampler extends Configured implement
s Tool {
r.setSeed(seed);
LOG.debug("seed: " + seed);
// shuffle splits
- for (int i = 0; i < splits.size(); ++i) {
-InputSplit tmp = splits.get(i);
-int j = r.nextInt(splits.size());
-splits.set(i, splits.get(j));
-splits.set(j, tmp);
- }
+ Collections.shuffle(splits);
+
// our target rate is in terms of the maximum number of sample splits,
// but we accept the possibility of sampling additional splits to hit
// the target sample keyset
Status: Patch Available (was: Open)
> Fixed split shuffling.
> --
>
> Key: MAPREDUCE-6245
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6245
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 2.6.0
>Reporter: lbkzman
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6245) Fixed split shuffling.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
lbkzman updated MAPREDUCE-6245:
---
Release Note: (was: index 72b47f2..8b89782 100644
--- src/java/org/apache/hadoop/mapreduce/lib/partition/InputSampler.java
+++ src/java/org/apache/hadoop/mapreduce/lib/partition/InputSampler.java
@@ -203,12 +203,8 @@ public class InputSampler extends Configured implement
s Tool {
r.setSeed(seed);
LOG.debug("seed: " + seed);
// shuffle splits
- for (int i = 0; i < splits.size(); ++i) {
-InputSplit tmp = splits.get(i);
-int j = r.nextInt(splits.size());
-splits.set(i, splits.get(j));
-splits.set(j, tmp);
- }
+ Collections.shuffle(splits);
+
// our target rate is in terms of the maximum number of sample splits,
// but we accept the possibility of sampling additional splits to hit
// the target sample keyset
)
> Fixed split shuffling.
> --
>
> Key: MAPREDUCE-6245
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6245
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 2.6.0
>Reporter: lbkzman
>Assignee: lbkzman
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6245) Fixed split shuffling.
[ https://issues.apache.org/jira/browse/MAPREDUCE-6245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lbkzman updated MAPREDUCE-6245: --- Assignee: lbkzman Target Version/s: 2.6.0 Status: Open (was: Patch Available) > Fixed split shuffling. > -- > > Key: MAPREDUCE-6245 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6245 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.6.0 >Reporter: lbkzman >Assignee: lbkzman > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6245) Fixed split shuffling.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
lbkzman updated MAPREDUCE-6245:
---
Status: Patch Available (was: Open)
index 72b47f2..8b89782 100644
--- src/java/org/apache/hadoop/mapreduce/lib/partition/InputSampler.java
+++ src/java/org/apache/hadoop/mapreduce/lib/partition/InputSampler.java
@@ -203,12 +203,8 @@ public class InputSampler extends Configured implement
s Tool {
r.setSeed(seed);
LOG.debug("seed: " + seed);
// shuffle splits
- for (int i = 0; i < splits.size(); ++i) {
-InputSplit tmp = splits.get(i);
-int j = r.nextInt(splits.size());
-splits.set(i, splits.get(j));
-splits.set(j, tmp);
- }
+ Collections.shuffle(splits);
+
// our target rate is in terms of the maximum number of sample splits,
// but we accept the possibility of sampling additional splits to hit
// the target sample keyset
> Fixed split shuffling.
> --
>
> Key: MAPREDUCE-6245
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6245
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 2.6.0
>Reporter: lbkzman
>Assignee: lbkzman
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6245) Fixed split shuffling.
[ https://issues.apache.org/jira/browse/MAPREDUCE-6245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lbkzman updated MAPREDUCE-6245: --- Status: Open (was: Patch Available) > Fixed split shuffling. > -- > > Key: MAPREDUCE-6245 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6245 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.6.0 >Reporter: lbkzman >Assignee: lbkzman > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6245) Fixed split shuffling.
[ https://issues.apache.org/jira/browse/MAPREDUCE-6245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lbkzman updated MAPREDUCE-6245: --- Status: Patch Available (was: Open) > Fixed split shuffling. > -- > > Key: MAPREDUCE-6245 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6245 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.6.0 >Reporter: lbkzman >Assignee: lbkzman > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6245) Fixed split shuffling.
[ https://issues.apache.org/jira/browse/MAPREDUCE-6245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lbkzman updated MAPREDUCE-6245: --- Status: Open (was: Patch Available) > Fixed split shuffling. > -- > > Key: MAPREDUCE-6245 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6245 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.6.0 >Reporter: lbkzman >Assignee: lbkzman > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
