shpark commented on a change in pull request #91: [NEMO-3] Bump up the Beam
version to 2.5.0
URL: https://github.com/apache/incubator-nemo/pull/91#discussion_r208205196
##########
File path:
examples/beam/src/main/java/edu/snu/nemo/examples/beam/GenericSourceSink.java
##########
@@ -94,7 +94,10 @@ public static PDone write(final PCollection<String>
dataToWrite,
dataToWrite.apply(ParDo.of(new HDFSWrite(path)));
return PDone.in(dataToWrite.getPipeline());
} else {
- return dataToWrite.apply(TextIO.write().to(path));
+ // Added withWindowedWrites() to local file writes. This is necessary
for FileResult coders.
+ // If not specified, FileResultCoder#encode will be blocked. See
windowCoder in FileResultCoder#encode
Review comment:
Since this problem is caused from Beam, It was difficult to fully specify
the cause. The execution is blocked at
[here](https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/io/FileBasedSink.java#L1141)
when trying to encode FileResult.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
