[ https://issues.apache.org/jira/browse/BEAM-6?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15189165#comment-15189165 ]
ASF GitHub Bot commented on BEAM-6: ----------------------------------- GitHub user tomwhite opened a pull request: https://github.com/apache/incubator-beam/pull/37 Import Spark Runner code This addresses https://issues.apache.org/jira/browse/BEAM-6. I've preserved git history (using @mxm's amazing git rewriting trick from #12). This is just an initial import - the Spark runner build is not yet integrated with the main build, packages need changing, etc. That's going to take more work, so it might be a good idea to get this merged first. You can merge this pull request into a Git repository by running: $ git pull https://github.com/tomwhite/incubator-beam beam-6-import-spark-runner Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/37.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #37 ---- commit ad444470e97f7a1a9bc4582831f9d6df334e9949 Author: Josh Wills <jwi...@cloudera.com> Date: 2014-07-11T01:59:48Z Initial commit commit 845a8178ab8bb620d0278b8312f04dce3f1348d8 Author: Josh Wills <jwi...@cloudera.com> Date: 2014-07-11T04:25:35Z Dumbest proof of concept possible commit cb7c86642ea9839e3638c16849e113cb3536604a Author: Josh Wills <jwi...@cloudera.com> Date: 2014-11-04T00:40:14Z First bit of work to get this running against the new Dataflow API commit dce03e4b7d607bf8ac9846fa4bfb26a81d747a63 Author: Josh Wills <jwi...@cloudera.com> Date: 2014-12-02T01:58:30Z Update version of dataflow to get new API method access commit 08e94b28ea06667049817def8aa5aee7aa31ebe1 Author: Josh Wills <jwi...@cloudera.com> Date: 2014-12-02T02:23:05Z Add support for getters and a Flatten impl commit 9fdac6ccb750488cdff8e3a696d114911d385ee4 Author: Josh Wills <jwi...@cloudera.com> Date: 2014-12-02T06:54:28Z Such code. Much features. commit deca2c0bdbe699099717591fc89afa4175c5438c Author: Josh Wills <jwi...@cloudera.com> Date: 2014-12-02T23:28:15Z Adding some more operators: toiterable, seqdo commit 6ee38b2a0677fc8eb77b4cd0aca64cfec2ec7fd9 Author: Josh Wills <jwi...@cloudera.com> Date: 2014-12-03T00:31:31Z Support for ParDo.BoundMulti commit 64c6d8d723023aafc2ee5754d41e00eca1524ddf Author: Josh Wills <jwi...@cloudera.com> Date: 2014-12-03T00:54:14Z Fix bug in deserializing side inputs commit 40adbeceb32b5d8a2a01bb5156c4e97ae9cfc58c Author: Josh Wills <jwi...@cloudera.com> Date: 2014-12-03T03:13:24Z Add SparkRuntimeContext for handling shared runtime objects commit 565509dc50a0f216665e6900a65842509b63e4d5 Author: Josh Wills <jwi...@cloudera.com> Date: 2014-12-03T04:12:49Z First cut at aggregators commit bb219d48a89f4f61d090b7cf47566dcee85adcf9 Author: Josh Wills <jwi...@cloudera.com> Date: 2014-12-03T05:03:35Z First minimally working aggregators commit 9152769c79c00aa8c9332c23ffcdd42a9876f402 Author: Josh Wills <jwi...@cloudera.com> Date: 2014-12-09T06:40:34Z Updates for 141206 SDK release commit 3bd04ae9ebcd89c93e5e635d806b613c5c5438ae Author: Josh Wills <jwi...@cloudera.com> Date: 2014-12-09T20:06:28Z Dummy impls of windowing-related ProcContext functions commit ba74f19ba6bda500a9541bbd84655336f1cff149 Author: Juliet Hougland <jul...@cloudera.com> Date: 2014-12-09T20:16:13Z Simplify pom.xml commit 6aa08e067d87ddb896df5a1a828bc0c526b9964f Author: Josh Wills <jwi...@cloudera.com> Date: 2014-12-09T23:30:29Z Add proper coder handling to RDD retrieval commit 45be5086820ba71dbee81bd7785f81e38f899fe8 Author: Juliet Hougland <jul...@cloudera.com> Date: 2014-12-10T00:35:25Z Refactor aggregation related classes. commit 137d54a1efe4e134cc7645c01eb013db10d2be30 Author: Juliet Hougland <jul...@cloudera.com> Date: 2014-12-10T00:37:46Z Adds Javadoc and Tests to project. commit 67cf364252fcf1df66ad97dac12d0cc006aa5d46 Author: Juliet Hougland <jul...@cloudera.com> Date: 2014-12-11T20:12:45Z Add README.md and update project version in pom.xml. commit b9545895738856693f3b5a40e127e348763a91f8 Author: Juliet Hougland <jul...@cloudera.com> Date: 2014-12-11T22:28:00Z Add apache2 license and cloudera copyright. commit 1523ffdae2a2fdf8029dfab68da1430edc6fe951 Author: Juliet Hougland <jul...@cloudera.com> Date: 2014-12-12T00:40:28Z Adds custom checkstyle. more checkstyle improvements commit 2992838fe35e06f280fbbf2eb419973be7699281 Author: Juliet Hougland <jul...@cloudera.com> Date: 2014-12-12T00:56:01Z Factor out tranform translation logic in to its own class. commit 2e3fe1ab9412be1b10659cd273a4de0ec5f103cd Author: Sean Owen <sro...@gmail.com> Date: 2014-12-13T16:03:39Z Specify and rationalize generic types in State, CoderHelpers to start commit 74892633fa1c5a66f2f05cbce4b0e571ed72d71f Author: Juliet Hougland <jul...@cloudera.com> Date: 2014-12-16T21:15:05Z Add simple word count test. commit f9e8fab69223582ab5b68cdfb634fad4c0c525ef Author: Juliet Hougland <jul...@cloudera.com> Date: 2014-12-17T01:34:57Z Factor out spark pipeline options. commit ec172bac7b77774f86f0982d0261e1b7bb6d52e6 Author: Sean Owen <sro...@gmail.com> Date: 2014-12-19T21:37:23Z Miscellaneous inspection changes from IntelliJ commit ed1e2f7fb66f0746c1a8511b398f41a6df0e1513 Author: Sean Owen <sro...@gmail.com> Date: 2014-12-20T08:36:13Z Issue #13 : attempt to remove all generics warnings, or handle them explicitly commit 225f6c0121e6171b02097cac86724920e0e32b50 Author: Sean Owen <sro...@gmail.com> Date: 2014-12-21T18:31:44Z Update and specify POM plugin config; Update Spark to 1.1.1, JUnit to 4.12, Spark 1.2; Add source, javadoc plugins and other info; Fix javadoc errors and a few typos commit 1f9cd04591497231a65c83d6d3f4a27428be79ac Author: Juliet Hougland <jul...@cloudera.com> Date: 2014-12-24T02:55:18Z Improve readme to explain current state of the repo, and to encourage outside contributions. commit ba4b32687c1c1bbf86bec6fc3a87e9c90c078ad7 Author: Juliet Hougland <jul...@cloudera.com> Date: 2014-12-23T18:44:46Z Update version of dataflow we depend on. The primary change needed to accomodate the new dataflow api is to how we handle side inputs. ---- > Import Spark Runner code > ------------------------ > > Key: BEAM-6 > URL: https://issues.apache.org/jira/browse/BEAM-6 > Project: Beam > Issue Type: Sub-task > Components: runner-spark > Reporter: Frances Perry > Assignee: Tom White > -- This message was sent by Atlassian JIRA (v6.3.4#6332)