[
https://issues.apache.org/jira/browse/BEAM-849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890885#comment-15890885
]
Amit Sela edited comment on BEAM-849 at 3/1/17 7:35 PM:
--------------------------------------------------------
I disagree with stating that "Create.of(filename) + ParDo(tail file) +
ParDo(process records)" in streaming leverages "low-latency", that's very
runner specific, and generally inaccurate - how is sending a "filename" to
worker/s + data locality slower than streaming the file to processors ?
was (Author: amitsela):
I disagree with stating that "Create.of(filename) + ParDo(tail file) +
ParDo(process records)" in streaming leverages "low-latency", that's very
runner specific, and generally inaccurate - how is sending a "filename" to
worker/s + data locality faster than streaming the file to processors ?
> Redesign PipelineResult API
> ---------------------------
>
> Key: BEAM-849
> URL: https://issues.apache.org/jira/browse/BEAM-849
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-core
> Reporter: Pei He
>
> Current state:
> Jira https://issues.apache.org/jira/browse/BEAM-443 addresses
> waitUntilFinish() and cancel().
> However, there are additional work around PipelineResult:
> need clearly defined contract and verification across all runners
> need to revisit how to handle metrics/aggregators
> need to be able to get logs
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
