reuvenlax commented on a change in pull request #4145: Many simplifications to
WriteFiles
URL: https://github.com/apache/beam/pull/4145#discussion_r152652760
##########
File path: sdks/java/core/src/main/java/org/apache/beam/sdk/io/WriteFiles.java
##########
@@ -824,177 +826,78 @@ public void startBundle() {
public void processElement(ProcessContext c) {
fileResults.add(c.element());
if (fixedNumShards == null) {
- if (numShardsView != null) {
- fixedNumShards = c.sideInput(numShardsView);
- } else if (numShardsProvider != null) {
- fixedNumShards = numShardsProvider.get();
- } else {
- throw new IllegalStateException(
- "When finalizing a windowed write, should have set fixed
sharding");
- }
+ fixedNumShards = getFixedNumShards.apply(c);
+ checkState(fixedNumShards != null, "Windowed write should have set
fixed sharding");
Review comment:
Windowed (non triggered) writes in batch do not need fixed sharding
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
