[ https://issues.apache.org/jira/browse/BEAM-1048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stas Levin reassigned BEAM-1048: -------------------------------- Assignee: Stas Levin (was: Kobi Salant) > Spark Runner streaming batch duration does not include duration of reading > from source > --------------------------------------------------------------------------------------- > > Key: BEAM-1048 > URL: https://issues.apache.org/jira/browse/BEAM-1048 > Project: Beam > Issue Type: Bug > Components: runner-spark > Affects Versions: 0.4.0 > Reporter: Kobi Salant > Assignee: Stas Levin > > Spark Runner streaming batch duration does not include duration of reading > from source this is because we perform rdd.count in SparkUnboundedSourcewhich > that invokes a regular spark job outside the streaming context. > We do it for reporting the batch size both for UI and back pressure -- This message was sent by Atlassian JIRA (v6.3.15#6346)