[
https://issues.apache.org/jira/browse/BEAM-2072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stas Levin resolved BEAM-2072.
------------------------------
Resolution: Fixed
Fix Version/s: First stable release
> MicrobatchSource.reader stops reading after reaching maxNumRecords for the
> first time
> -------------------------------------------------------------------------------------
>
> Key: BEAM-2072
> URL: https://issues.apache.org/jira/browse/BEAM-2072
> Project: Beam
> Issue Type: Bug
> Components: runner-spark
> Affects Versions: Not applicable
> Reporter: Stas Levin
> Assignee: Stas Levin
> Fix For: First stable release
>
>
> {{MicrobatchSource.Reader}} stops reading further data after the first batch
> in which it has reached {{maxNumRecords}}. Subsequent read request will check
> the {{recordsRead}} and conclude that no data can be read.
> The root cause being that {{recordsRead}} is maintained throughout the
> reader's lifecycle, and can span multiple (spark) batches.
> We should reset {{recordsRead}} every time the reader starts reading a fresh
> batch.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
