GitHub user ppastuszkaocado opened a pull request: https://github.com/apache/incubator-beam/pull/687
[BEAM-461] Create custom Kinesis source This adds custom Kinesis source as new beam-sdks-java-io-kinesis module. Build on Travis has passed. This has been tested with unit tests (included in the pull request), but also using DirectRunner (I've confirmed, that source is able to successfully read data produced by Kinesis Producer Library). I've also tried to run this on real dataflow, but it doesn't seem to work (the pipeline starts, but it doesn't read anything at all) - I'm going to investigate the issue, but I'd like you to do the review in the meantime. Speaking about tests... I've got bunch of integration / end-to-end level tests (using DirectRunner and DataflowRunner), which are using real kinesis to read data from. I'm not sure where should I put them and how would they be executed in apache CI environment? BTW. Is this "large" enough, so that I should sign ICLA? @dhalperi, @jbonofre - can you please review this? You can merge this pull request into a Git repository by running: $ git pull https://github.com/ocadotechnology/incubator-beam beam_461_kinesis_io Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/687.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #687 ---- commit 76b302e1d011e3c1acc9bd2e4e348520f1d7e1d2 Author: Pastuszka PrzemysÅaw <pastuszka.przemys...@gmail.com> Date: 2016-07-18T18:03:37Z Moving to apache beam commit fd6195d32936febe1acf20ea4f8b4a7c4f93f053 Author: PrzemysÅaw Pastuszka <p.pastus...@ocado.com> Date: 2016-07-19T08:04:09Z Fix build commit fb70c6699c95a69a448b3d8d26da0ba381a9e188 Author: PrzemysÅaw Pastuszka <p.pastus...@ocado.com> Date: 2016-07-19T08:56:02Z Get rid of dependency on Java 8 ---- --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---