jkff commented on a change in pull request #4175: [BEAM-3247] fix Sample.any
performance
URL: https://github.com/apache/beam/pull/4175#discussion_r153348529
##########
File path:
sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Sample.java
##########
@@ -209,29 +202,67 @@ public void populateDisplayData(DisplayData.Builder
builder) {
}
/**
- * A {@link DoFn} that returns up to limit elements from the side input
PCollection.
+ * A {@link DoFn} that outputs up to limit elements.
*/
- private static class SampleAnyDoFn<T> extends DoFn<Void, T> {
- long limit;
- final PCollectionView<Iterable<T>> iterableView;
+ private static class SampleAnyDoFn<T> extends DoFn<T, T> {
Review comment:
Oh sorry, I looked in the wrong place in the code. Yeah, this implementation
is wrong: the main element (null in this case) is in the global window, so of
course it's not allowed to access a side input that is windowed non-globally,
because it's ambiguous which window should be accessed. It plays even more
poorly with triggering. See
https://beam.apache.org/documentation/programming-guide/#side-inputs-and-windowing
. All the more reason to fix this!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
