I think this can answer my question under Flink-runner
https://github.com/apache/beam/blob/019d3002b0e2a7db9c5c2e84a0a95fad60f16422/runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/SourceInputFormat.java
the desiredBundleSizeBytes is determined by
source.getEstimatedSizeBytes()/numSplits.
On 11/07/2017 11:39 AM, Ivan wrote:
Hi, we are trying to build a custom BoundedSource based on gRPC call.
in class BoundedSource<T> we got method below
public abstract java.util.List<? extendsBoundedSource
<https://beam.apache.org/documentation/sdks/javadoc/2.0.0/org/apache/beam/sdk/io/BoundedSource.html><T
<https://beam.apache.org/documentation/sdks/javadoc/2.0.0/org/apache/beam/sdk/io/BoundedSource.html>>> split(long desiredBundleSizeBytes,
PipelineOptions
<https://beam.apache.org/documentation/sdks/javadoc/2.0.0/org/apache/beam/sdk/options/PipelineOptions.html> options)
throws
java.lang.Exception
at runtime, how does the parameter desiredBundleSizeBytes get determined, is it
different under different runner? what's the behavior in Flink-Runner? what's
the relational ship of this parameter and the parallelism we specify when in
theFlinkPipelieOptions?