[jira] [Work logged] (BEAM-5272) Randomize the reduced splits in BigtableIO so that multiple workers may not hit the same tablet server

2018-09-28 Thread ASF GitHub Bot (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5272?focusedWorklogId=149137=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-149137
 ]

ASF GitHub Bot logged work on BEAM-5272:


Author: ASF GitHub Bot
Created on: 28/Sep/18 12:06
Start Date: 28/Sep/18 12:06
Worklog Time Spent: 10m 
  Work Description: kevinsi4508 commented on issue #6503: [BEAM-5272] 
Randomize the reduced splits in BigtableIO so that multiple workers may not hit 
the same tablet server
URL: https://github.com/apache/beam/pull/6503#issuecomment-425414669
 
 
   Thanks!


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 149137)
Time Spent: 1h 40m  (was: 1.5h)

> Randomize the reduced splits in BigtableIO so that multiple workers may not 
> hit the same tablet server
> --
>
> Key: BEAM-5272
> URL: https://issues.apache.org/jira/browse/BEAM-5272
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Reporter: Kevin Si
>Assignee: Chamikara Jayalath
>Priority: Minor
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> Randomize the reduced splits in BigtableIO so that multiple workers may not 
> hit the same tablet server.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-5272) Randomize the reduced splits in BigtableIO so that multiple workers may not hit the same tablet server

2018-09-27 Thread ASF GitHub Bot (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5272?focusedWorklogId=148879=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-148879
 ]

ASF GitHub Bot logged work on BEAM-5272:


Author: ASF GitHub Bot
Created on: 27/Sep/18 20:17
Start Date: 27/Sep/18 20:17
Worklog Time Spent: 10m 
  Work Description: aaltay closed pull request #6503: [BEAM-5272] Randomize 
the reduced splits in BigtableIO so that multiple workers may not hit the same 
tablet server
URL: https://github.com/apache/beam/pull/6503
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIO.java
 
b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIO.java
index edad185323c..755d889b491 100644
--- 
a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIO.java
+++ 
b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIO.java
@@ -848,18 +848,25 @@ protected BigtableSource withEstimatedSizeBytes(Long 
estimatedSizeBytes) {
   // Delegate to testable helper.
   List splits =
   splitBasedOnSamples(desiredBundleSizeBytes, 
getSampleRowKeys(options));
-  return reduceSplits(splits, options, MAX_SPLIT_COUNT);
+
+  // Reduce the splits.
+  List reduced = reduceSplits(splits, options, 
MAX_SPLIT_COUNT);
+  // Randomize the result before returning an immutable copy of the 
splits, the default behavior
+  // may lead to multiple workers hitting the same tablet.
+  Collections.shuffle(reduced);
+  return ImmutableList.copyOf(reduced);
 }
 
+/** Returns a mutable list of reduced splits. */
 @VisibleForTesting
 protected List reduceSplits(
 List splits, PipelineOptions options, long 
maxSplitCounts)
 throws IOException {
   int numberToCombine = (int) ((splits.size() + maxSplitCounts - 1) / 
maxSplitCounts);
   if (splits.size() < maxSplitCounts || numberToCombine < 2) {
-return splits;
+return new ArrayList<>(splits);
   }
-  ImmutableList.Builder reducedSplits = 
ImmutableList.builder();
+  List reducedSplits = new ArrayList<>();
   List previousSourceRanges = new ArrayList();
   int counter = 0;
   long size = 0;
@@ -879,7 +886,7 @@ protected BigtableSource withEstimatedSizeBytes(Long 
estimatedSizeBytes) {
   if (size > 0) {
 reducedSplits.add(new BigtableSource(config, filter, 
previousSourceRanges, size));
   }
-  return reducedSplits.build();
+  return reducedSplits;
 }
 
 /**
diff --git 
a/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIOTest.java
 
b/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIOTest.java
index cadb908be5a..54a2fee99b0 100644
--- 
a/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIOTest.java
+++ 
b/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIOTest.java
@@ -102,7 +102,7 @@
 import org.apache.beam.sdk.values.PCollection;
 import org.apache.beam.sdk.values.TypeDescriptor;
 import org.hamcrest.Matchers;
-import org.hamcrest.collection.IsIterableContainingInOrder;
+import org.hamcrest.collection.IsIterableContainingInAnyOrder;
 import org.junit.Before;
 import org.junit.Rule;
 import org.junit.Test;
@@ -736,10 +736,10 @@ public void testReduceSplitsWithSomeNonAdjacentRanges() 
throws Exception {
 keyRanges,
 null /*size*/);
 
-List splits =
-source.split(numRows * bytesPerRow / numSamples, null /* options */);
-
-assertThat(splits, hasSize(keyRanges.size()));
+List splits = new ArrayList<>();
+for (ByteKeyRange range : keyRanges) {
+  splits.add(source.withSingleRange(range));
+}
 
 List reducedSplits = source.reduceSplits(splits, null, 
maxSplit);
 
@@ -753,7 +753,8 @@ public void testReduceSplitsWithSomeNonAdjacentRanges() 
throws Exception {
 
 assertThat(
 actualRangesAfterSplit,
-
IsIterableContainingInOrder.contains(expectedKeyRangesAfterReducedSplits.toArray()));
+IsIterableContainingInAnyOrder.containsInAnyOrder(
+expectedKeyRangesAfterReducedSplits.toArray()));
   }
 
   /** Tests reduce split with all non adjacent ranges. */
@@ -786,10 +787,10 @@ public void testReduceSplitsWithAllNonAdjacentRange() 
throws Exception {
 keyRanges,
 null /*size*/);
 
-List splits =
-source.split(numRows * 

[jira] [Work logged] (BEAM-5272) Randomize the reduced splits in BigtableIO so that multiple workers may not hit the same tablet server

2018-09-27 Thread ASF GitHub Bot (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5272?focusedWorklogId=148745=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-148745
 ]

ASF GitHub Bot logged work on BEAM-5272:


Author: ASF GitHub Bot
Created on: 27/Sep/18 13:43
Start Date: 27/Sep/18 13:43
Worklog Time Spent: 10m 
  Work Description: kevinsi4508 closed pull request #6308: [BEAM-5272] 
Randomize the reduced splits in BigtableIO so that multiple workers may not hit 
the same tablet server
URL: https://github.com/apache/beam/pull/6308
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIO.java
 
b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIO.java
index ae8fe7d04d9..cb5a174713e 100644
--- 
a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIO.java
+++ 
b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIO.java
@@ -848,18 +848,27 @@ protected BigtableSource withEstimatedSizeBytes(Long 
estimatedSizeBytes) {
   // Delegate to testable helper.
   List splits =
   splitBasedOnSamples(desiredBundleSizeBytes, 
getSampleRowKeys(options));
-  return reduceSplits(splits, options, MAX_SPLIT_COUNT);
+
+  // Reduce the splits.
+  List reduced = reduceSplits(splits, options, 
MAX_SPLIT_COUNT);
+  // Randomize the result before returning an immutable copy of the 
splits, the default behavior
+  // may lead to multiple workers hitting the same tablet.
+  Collections.shuffle(reduced);
+  return ImmutableList.copyOf(reduced);
 }
 
+/**
+ * Returns a mutable list of reduced splits.
+ */
 @VisibleForTesting
 protected List reduceSplits(
 List splits, PipelineOptions options, long 
maxSplitCounts)
 throws IOException {
   int numberToCombine = (int) ((splits.size() + maxSplitCounts - 1) / 
maxSplitCounts);
   if (splits.size() < maxSplitCounts || numberToCombine < 2) {
-return splits;
+return new ArrayList<>(splits);
   }
-  ImmutableList.Builder reducedSplits = 
ImmutableList.builder();
+  List reducedSplits = new ArrayList<>();
   List previousSourceRanges = new ArrayList();
   int counter = 0;
   long size = 0;
@@ -879,7 +888,7 @@ protected BigtableSource withEstimatedSizeBytes(Long 
estimatedSizeBytes) {
   if (size > 0) {
 reducedSplits.add(new BigtableSource(config, filter, 
previousSourceRanges, size));
   }
-  return reducedSplits.build();
+  return reducedSplits;
 }
 
 /**
diff --git 
a/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIOTest.java
 
b/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIOTest.java
index 47727e5b8a1..518dc104c4e 100644
--- 
a/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIOTest.java
+++ 
b/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIOTest.java
@@ -100,7 +100,7 @@
 import org.apache.beam.sdk.values.PCollection;
 import org.apache.beam.sdk.values.TypeDescriptor;
 import org.hamcrest.Matchers;
-import org.hamcrest.collection.IsIterableContainingInOrder;
+import org.hamcrest.collection.IsIterableContainingInAnyOrder;
 import org.junit.Before;
 import org.junit.Rule;
 import org.junit.Test;
@@ -680,10 +680,10 @@ public void testReduceSplitsWithSomeNonAdjacentRanges() 
throws Exception {
 keyRanges,
 null /*size*/);
 
-List splits =
-source.split(numRows * bytesPerRow / numSamples, null /* options */);
-
-assertThat(splits, hasSize(keyRanges.size()));
+List splits = new ArrayList<>();
+for (ByteKeyRange range : keyRanges) {
+  splits.add(source.withSingleRange(range));
+}
 
 List reducedSplits = source.reduceSplits(splits, null, 
maxSplit);
 
@@ -697,7 +697,8 @@ public void testReduceSplitsWithSomeNonAdjacentRanges() 
throws Exception {
 
 assertThat(
 actualRangesAfterSplit,
-
IsIterableContainingInOrder.contains(expectedKeyRangesAfterReducedSplits.toArray()));
+IsIterableContainingInAnyOrder.containsInAnyOrder(
+expectedKeyRangesAfterReducedSplits.toArray()));
   }
 
   /** Tests reduce split with all non adjacent ranges. */
@@ -730,10 +731,10 @@ public void testReduceSplitsWithAllNonAdjacentRange() 
throws Exception {
 keyRanges,
 null /*size*/);
 
-List splits =
-

[jira] [Work logged] (BEAM-5272) Randomize the reduced splits in BigtableIO so that multiple workers may not hit the same tablet server

2018-09-27 Thread ASF GitHub Bot (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5272?focusedWorklogId=148744=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-148744
 ]

ASF GitHub Bot logged work on BEAM-5272:


Author: ASF GitHub Bot
Created on: 27/Sep/18 13:43
Start Date: 27/Sep/18 13:43
Worklog Time Spent: 10m 
  Work Description: kevinsi4508 commented on issue #6503: [BEAM-5272] 
Randomize the reduced splits in BigtableIO so that multiple workers may not hit 
the same tablet server
URL: https://github.com/apache/beam/pull/6503#issuecomment-425097437
 
 
   @chamikaramj, could you take a look? Thanks!


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 148744)
Time Spent: 1h 10m  (was: 1h)

> Randomize the reduced splits in BigtableIO so that multiple workers may not 
> hit the same tablet server
> --
>
> Key: BEAM-5272
> URL: https://issues.apache.org/jira/browse/BEAM-5272
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Reporter: Kevin Si
>Assignee: Chamikara Jayalath
>Priority: Minor
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Randomize the reduced splits in BigtableIO so that multiple workers may not 
> hit the same tablet server.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-5272) Randomize the reduced splits in BigtableIO so that multiple workers may not hit the same tablet server

2018-09-27 Thread ASF GitHub Bot (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5272?focusedWorklogId=148742=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-148742
 ]

ASF GitHub Bot logged work on BEAM-5272:


Author: ASF GitHub Bot
Created on: 27/Sep/18 13:42
Start Date: 27/Sep/18 13:42
Worklog Time Spent: 10m 
  Work Description: kevinsi4508 opened a new pull request #6503: 
[BEAM-5272] Randomize the reduced splits in BigtableIO so that multiple workers 
may not hit the same tablet server
URL: https://github.com/apache/beam/pull/6503
 
 
   Randomize the reduced splits in BigtableIO so that multiple workers may not 
hit the same tablet server.
   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/)
 | --- | --- | ---
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 148742)
Time Spent: 50m  (was: 40m)

> Randomize the reduced splits in BigtableIO so that multiple workers may not 
> hit the same tablet server
> --
>
> Key: BEAM-5272
> URL: https://issues.apache.org/jira/browse/BEAM-5272
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Reporter: Kevin Si
>Assignee: Chamikara Jayalath
>Priority: Minor
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Randomize the reduced splits in BigtableIO so that multiple workers may not 
> hit the same tablet server.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-5272) Randomize the reduced splits in BigtableIO so that multiple workers may not hit the same tablet server

2018-09-27 Thread ASF GitHub Bot (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5272?focusedWorklogId=148743=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-148743
 ]

ASF GitHub Bot logged work on BEAM-5272:


Author: ASF GitHub Bot
Created on: 27/Sep/18 13:42
Start Date: 27/Sep/18 13:42
Worklog Time Spent: 10m 
  Work Description: kevinsi4508 commented on issue #6308: [BEAM-5272] 
Randomize the reduced splits in BigtableIO so that multiple workers may not hit 
the same tablet server
URL: https://github.com/apache/beam/pull/6308#issuecomment-425097332
 
 
   Created a new PR: https://github.com/apache/beam/pull/6503


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 148743)
Time Spent: 1h  (was: 50m)

> Randomize the reduced splits in BigtableIO so that multiple workers may not 
> hit the same tablet server
> --
>
> Key: BEAM-5272
> URL: https://issues.apache.org/jira/browse/BEAM-5272
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Reporter: Kevin Si
>Assignee: Chamikara Jayalath
>Priority: Minor
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> Randomize the reduced splits in BigtableIO so that multiple workers may not 
> hit the same tablet server.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-5272) Randomize the reduced splits in BigtableIO so that multiple workers may not hit the same tablet server

2018-09-26 Thread ASF GitHub Bot (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5272?focusedWorklogId=148557=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-148557
 ]

ASF GitHub Bot logged work on BEAM-5272:


Author: ASF GitHub Bot
Created on: 27/Sep/18 01:47
Start Date: 27/Sep/18 01:47
Worklog Time Spent: 10m 
  Work Description: aaltay commented on issue #6308: [BEAM-5272] Randomize 
the reduced splits in BigtableIO so that multiple workers may not hit the same 
tablet server
URL: https://github.com/apache/beam/pull/6308#issuecomment-424928687
 
 
   What is the status of this PR?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 148557)
Time Spent: 40m  (was: 0.5h)

> Randomize the reduced splits in BigtableIO so that multiple workers may not 
> hit the same tablet server
> --
>
> Key: BEAM-5272
> URL: https://issues.apache.org/jira/browse/BEAM-5272
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Reporter: Kevin Si
>Assignee: Chamikara Jayalath
>Priority: Minor
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Randomize the reduced splits in BigtableIO so that multiple workers may not 
> hit the same tablet server.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-5272) Randomize the reduced splits in BigtableIO so that multiple workers may not hit the same tablet server

2018-08-31 Thread ASF GitHub Bot (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5272?focusedWorklogId=140150=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-140150
 ]

ASF GitHub Bot logged work on BEAM-5272:


Author: ASF GitHub Bot
Created on: 31/Aug/18 16:34
Start Date: 31/Aug/18 16:34
Worklog Time Spent: 10m 
  Work Description: kevinsi4508 commented on issue #6308: [BEAM-5272] 
Randomize the reduced splits in BigtableIO so that multiple workers may not hit 
the same tablet server
URL: https://github.com/apache/beam/pull/6308#issuecomment-417719812
 
 
   @chamikaramj, could you take a loook?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 140150)
Time Spent: 20m  (was: 10m)

> Randomize the reduced splits in BigtableIO so that multiple workers may not 
> hit the same tablet server
> --
>
> Key: BEAM-5272
> URL: https://issues.apache.org/jira/browse/BEAM-5272
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Reporter: Kevin Si
>Assignee: Chamikara Jayalath
>Priority: Minor
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Randomize the reduced splits in BigtableIO so that multiple workers may not 
> hit the same tablet server.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Work logged] (BEAM-5272) Randomize the reduced splits in BigtableIO so that multiple workers may not hit the same tablet server

2018-08-30 Thread ASF GitHub Bot (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5272?focusedWorklogId=139887=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-139887
 ]

ASF GitHub Bot logged work on BEAM-5272:


Author: ASF GitHub Bot
Created on: 30/Aug/18 21:14
Start Date: 30/Aug/18 21:14
Worklog Time Spent: 10m 
  Work Description: kevinsi4508 opened a new pull request #6308: 
[BEAM-5272] Randomize the reduced splits in BigtableIO so that multiple workers 
may not hit the same tablet server
URL: https://github.com/apache/beam/pull/6308
 
 
   Randomize the reduced splits so that multiple workers may not hit the same 
tablet server
   
   
   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | --- | --- | --- | ---
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 139887)
Time Spent: 10m
Remaining Estimate: 0h

> Randomize the reduced splits in BigtableIO so that multiple workers may not 
> hit the same tablet server
> --
>
> Key: BEAM-5272
> URL: https://issues.apache.org/jira/browse/BEAM-5272
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Reporter: Kevin Si
>Assignee: Chamikara Jayalath
>Priority: Minor
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Randomize the reduced splits in BigtableIO so that multiple workers may not 
> hit the same tablet server.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)