[ https://issues.apache.org/jira/browse/BEAM-461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15383977#comment-15383977 ]
ASF GitHub Bot commented on BEAM-461: ------------------------------------- GitHub user ppastuszkaocado opened a pull request: https://github.com/apache/incubator-beam/pull/687 [BEAM-461] Create custom Kinesis source This adds custom Kinesis source as new beam-sdks-java-io-kinesis module. Build on Travis has passed. This has been tested with unit tests (included in the pull request), but also using DirectRunner (I've confirmed, that source is able to successfully read data produced by Kinesis Producer Library). I've also tried to run this on real dataflow, but it doesn't seem to work (the pipeline starts, but it doesn't read anything at all) - I'm going to investigate the issue, but I'd like you to do the review in the meantime. Speaking about tests... I've got bunch of integration / end-to-end level tests (using DirectRunner and DataflowRunner), which are using real kinesis to read data from. I'm not sure where should I put them and how would they be executed in apache CI environment? BTW. Is this "large" enough, so that I should sign ICLA? @dhalperi, @jbonofre - can you please review this? You can merge this pull request into a Git repository by running: $ git pull https://github.com/ocadotechnology/incubator-beam beam_461_kinesis_io Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/687.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #687 ---- commit 76b302e1d011e3c1acc9bd2e4e348520f1d7e1d2 Author: Pastuszka Przemysław <pastuszka.przemys...@gmail.com> Date: 2016-07-18T18:03:37Z Moving to apache beam commit fd6195d32936febe1acf20ea4f8b4a7c4f93f053 Author: Przemysław Pastuszka <p.pastus...@ocado.com> Date: 2016-07-19T08:04:09Z Fix build commit fb70c6699c95a69a448b3d8d26da0ba381a9e188 Author: Przemysław Pastuszka <p.pastus...@ocado.com> Date: 2016-07-19T08:56:02Z Get rid of dependency on Java 8 ---- > Create custom Kinesis source > ---------------------------- > > Key: BEAM-461 > URL: https://issues.apache.org/jira/browse/BEAM-461 > Project: Beam > Issue Type: Improvement > Affects Versions: 0.2.0-incubating > Reporter: Przemyslaw Pastuszka > Labels: newbie > > Design doc is here: > https://docs.google.com/document/d/1O_m9albjxzBdGf94gbU1682kSwGEKjsQ0AMF5FBsjUU/edit?usp=sharing > This has been already proposed for contribution to DataflowJavaSDK: > https://github.com/GoogleCloudPlatform/DataflowJavaSDK/pull/149 -- This message was sent by Atlassian JIRA (v6.3.4#6332)