[ https://issues.apache.org/jira/browse/BEAM-1182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15941180#comment-15941180 ]
ASF GitHub Bot commented on BEAM-1182: -------------------------------------- GitHub user tgroh opened a pull request: https://github.com/apache/beam/pull/2323 [BEAM-1182] Clone Checkpoints before resuming in the DirectRunner Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-<Jira issue #>] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `<Jira issue #>` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/tgroh/beam clone_before_resume Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2323.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2323 ---- commit 04fe4b63e84b56c455ccc966b591ae5e24d67149 Author: Thomas Groh <tg...@google.com> Date: 2017-03-24T21:36:12Z Always require an UnboundedSource to provide a Checkpoint Coder The coder can do no work, but should always be specified. commit 0de77171d5cd273e7915cf875d796b9550daa71e Author: Thomas Groh <tg...@google.com> Date: 2017-03-24T21:36:46Z Clone before Resume in DirectRunner Unbounded Reads This exercises the CheckpointMarkCoder of all Unbounded Sources in the DirectRunner. ---- > Direct runner should enforce encodability of unbounded source checkpoints > ------------------------------------------------------------------------- > > Key: BEAM-1182 > URL: https://issues.apache.org/jira/browse/BEAM-1182 > Project: Beam > Issue Type: Improvement > Components: runner-direct > Reporter: Eugene Kirpichov > Assignee: Thomas Groh > > As far as I can tell, > https://github.com/apache/incubator-beam/blob/master/runners/direct-java/src/main/java/org/apache/beam/runners/direct/UnboundedReadEvaluatorFactory.java > currently uses the reader's getCheckpoint() only as an in-memory object, > i.e. it's not exercising that 1) the checkpoint can be encoded at all, and 2) > that the reader can be resumed from an encoded/decoded checkpoint. > I've seen cases in code reviews where people implemented a non-serializable > checkpoint, and direct runner tests didn't catch that because of this issue. -- This message was sent by Atlassian JIRA (v6.3.15#6346)