[GitHub] beam pull request #3611: [BEAM-79] merge gearpump-runner into master
Github user manuzhang closed the pull request at: https://github.com/apache/beam/pull/3611 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3637: [BEAM-79] Prepare for merging gearpump-runner
GitHub user manuzhang opened a pull request: https://github.com/apache/beam/pull/3637 [BEAM-79] Prepare for merging gearpump-runner Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/manuzhang/beam gearpump-runner Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3637.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3637 commit e655f53c2e732bcd082449ba8d0b551f417e8aaa Author: manuzhang <owenzhang1...@gmail.com> Date: 2017-07-21T06:27:37Z Revert accidental changes to sdks/java/pom.xml commit daa7566939e212095ef745ff004580bfe4209b38 Author: manuzhang <owenzhang1...@gmail.com> Date: 2017-07-21T06:58:13Z Upgrade BEAM version to 2.2.0-SNAPSHOT in gearpump-runner commit 49d4ed59567852e978d14729c89b9ea91bb96c32 Author: manuzhang <owenzhang1...@gmail.com> Date: 2017-07-21T14:33:36Z Add beam-runners-gearpump dependency to javadoc --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3636: [BEAM-79] merge gearpump-runner into master
GitHub user manuzhang opened a pull request: https://github.com/apache/beam/pull/3636 [BEAM-79] merge gearpump-runner into master Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/apache/beam gearpump-runner Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3636.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3636 commit 9478f4117de3a2d0ea40614ed4cb801918610724 Author: manuzhang <owenzhang1...@gmail.com> Date: 2016-03-15T08:15:16Z [BEAM-79] add Gearpump runner commit 02b2248a5b3c8a2c064547d7380bebc97f849bf1 Author: Kenneth Knowles <k...@google.com> Date: 2016-07-20T16:07:48Z This closes #323 commit 2a0ba61e8507e1539115b583749a78f14d577bd8 Author: Kenneth Knowles <k...@google.com> Date: 2016-08-25T18:36:45Z Merge branch master into gearpump-runner commit 1672b5483e029292816397248dc6fe63bf51f4af Author: manuzhang <owenzhang1...@gmail.com> Date: 2016-07-23T06:10:15Z move integration tests to profile commit 276a2e106aa1a573fc2eb2426b640f63cf68 Author: manuzhang <owenzhang1...@gmail.com> Date: 2016-07-28T08:30:13Z add package-info.java commit 40be715a696bb1218b209f7ad9a979b7e5d088d3 Author: Kenneth Knowles <k...@google.com> Date: 2016-08-10T17:26:57Z Update Gearpump runner version to 0.3.0-incubating commit bc1b354949416db3b52c4f37c66968bdb86f0813 Author: manuzhang <owenzhang1...@gmail.com> Date: 2016-08-11T23:22:00Z Rename DoFn to OldDoFn in Gearpump runner commit 091a15a07c7625ae3009cefaecece3a29a34c109 Author: Kenneth Knowles <k...@google.com> Date: 2016-08-25T18:40:03Z This closess #750 commit fb74c936ed92c7a8548c338cc03957794fc60902 Author: Dan Halperin <dhalp...@google.com> Date: 2016-08-26T23:25:58Z gearpump: switch to stable version They have apparently deleted the SNAPSHOT jar and now builds are failing. commit bf0a2edae11416a3cbddeaff2c0a70adc272c5fe Author: Dan Halperin <dhalp...@google.com> Date: 2016-08-27T00:46:42Z Closes #895 commit 89921c41ca9d4c333af45efa32359a631214c1df Author: bchambers <bchamb...@google.com> Date: 2016-07-29T16:41:17Z Remove Counter and associated code Aggregator is the model level concept. Counter was specific to the Dataflow Runner, and is now not needed as part of Beam. commit 7fc2c6848f002ac8b2ccbe35e6b5a576777a7af9 Author: Mark Liu <mark...@markliu-macbookpro.roam.corp.google.com> Date: 2016-08-03T00:25:14Z [BEAM-495] Create General Verifier for File Checksum commit b47549e4893a6d487c00ea0ba02619168a3f19f3 Author: Mark Liu <mark...@markliu-macbookpro.roam.corp.google.com> Date: 2016-08-03T00:47:46Z Add output checksum to WordCountITOptions commit 58cd781c82fa728f34f5ab0641f8f9b6edcf449c Author: Ian Zhou <ianz...@google.com> Date: 2016-08-05T22:31:59Z Added unit tests and error handling in removeTemporaryTables commit 36a9aa232ea56de449930194788becce585212ef Author: Thomas Groh <tg...@google.com> Date: 2016-08-09T02:09:58Z Improve Write Error Message If provided with an Unbounded PCollection, Write will fail due to restriction of calling finalize only once. This error message fails in a deep stack trace based on it not being possible to apply a GroupByKey. Instead, throw immediately on application with a specific error message. commit d5641553cebb02f08ca7c1fe667948d39cb3962c Author: Thomas Groh <tg...@google.com> Date: 2016-08-09T17:47:09Z Remove Streaming Write Overrides in DataflowRunner These writes should be forbidden based on the boundedness of the input PCollection. As Write explicitly f
[GitHub] beam pull request #3611: [BEAM-79] merge gearpump-runner into master
GitHub user manuzhang opened a pull request: https://github.com/apache/beam/pull/3611 [BEAM-79] merge gearpump-runner into master Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/manuzhang/beam gearpump-runner Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3611.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3611 commit 9478f4117de3a2d0ea40614ed4cb801918610724 Author: manuzhang <owenzhang1...@gmail.com> Date: 2016-03-15T08:15:16Z [BEAM-79] add Gearpump runner commit 02b2248a5b3c8a2c064547d7380bebc97f849bf1 Author: Kenneth Knowles <k...@google.com> Date: 2016-07-20T16:07:48Z This closes #323 commit 2a0ba61e8507e1539115b583749a78f14d577bd8 Author: Kenneth Knowles <k...@google.com> Date: 2016-08-25T18:36:45Z Merge branch master into gearpump-runner commit 1672b5483e029292816397248dc6fe63bf51f4af Author: manuzhang <owenzhang1...@gmail.com> Date: 2016-07-23T06:10:15Z move integration tests to profile commit 276a2e106aa1a573fc2eb2426b640f63cf68 Author: manuzhang <owenzhang1...@gmail.com> Date: 2016-07-28T08:30:13Z add package-info.java commit 40be715a696bb1218b209f7ad9a979b7e5d088d3 Author: Kenneth Knowles <k...@google.com> Date: 2016-08-10T17:26:57Z Update Gearpump runner version to 0.3.0-incubating commit bc1b354949416db3b52c4f37c66968bdb86f0813 Author: manuzhang <owenzhang1...@gmail.com> Date: 2016-08-11T23:22:00Z Rename DoFn to OldDoFn in Gearpump runner commit 091a15a07c7625ae3009cefaecece3a29a34c109 Author: Kenneth Knowles <k...@google.com> Date: 2016-08-25T18:40:03Z This closess #750 commit fb74c936ed92c7a8548c338cc03957794fc60902 Author: Dan Halperin <dhalp...@google.com> Date: 2016-08-26T23:25:58Z gearpump: switch to stable version They have apparently deleted the SNAPSHOT jar and now builds are failing. commit bf0a2edae11416a3cbddeaff2c0a70adc272c5fe Author: Dan Halperin <dhalp...@google.com> Date: 2016-08-27T00:46:42Z Closes #895 commit 89921c41ca9d4c333af45efa32359a631214c1df Author: bchambers <bchamb...@google.com> Date: 2016-07-29T16:41:17Z Remove Counter and associated code Aggregator is the model level concept. Counter was specific to the Dataflow Runner, and is now not needed as part of Beam. commit 7fc2c6848f002ac8b2ccbe35e6b5a576777a7af9 Author: Mark Liu <mark...@markliu-macbookpro.roam.corp.google.com> Date: 2016-08-03T00:25:14Z [BEAM-495] Create General Verifier for File Checksum commit b47549e4893a6d487c00ea0ba02619168a3f19f3 Author: Mark Liu <mark...@markliu-macbookpro.roam.corp.google.com> Date: 2016-08-03T00:47:46Z Add output checksum to WordCountITOptions commit 58cd781c82fa728f34f5ab0641f8f9b6edcf449c Author: Ian Zhou <ianz...@google.com> Date: 2016-08-05T22:31:59Z Added unit tests and error handling in removeTemporaryTables commit 36a9aa232ea56de449930194788becce585212ef Author: Thomas Groh <tg...@google.com> Date: 2016-08-09T02:09:58Z Improve Write Error Message If provided with an Unbounded PCollection, Write will fail due to restriction of calling finalize only once. This error message fails in a deep stack trace based on it not being possible to apply a GroupByKey. Instead, throw immediately on application with a specific error message. commit d5641553cebb02f08ca7c1fe667948d39cb3962c Author: Thomas Groh <tg...@google.com> Date: 2016-08-09T17:47:09Z Remove Streaming Write Overrides in DataflowRunner These writes should be forbidden based on the boundedness of the input PCollection. As Write explicitly f
[GitHub] beam pull request #3515: [BEAM-79] Sync gearpump-runner with master and upgr...
GitHub user manuzhang opened a pull request: https://github.com/apache/beam/pull/3515 [BEAM-79] Sync gearpump-runner with master and upgrade to gearpump 0.8.4 Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/manuzhang/beam sync-master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3515.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3515 commit 7b8cd6401cb5ed6e184ed36571a89d3ae324dd5f Author: Jeremie Lenfant-Engelmann <jeremi...@google.com> Date: 2017-06-29T01:32:56Z Properly convert milliseconds whether there's less than 3/more than 9 digits. TimeUtil did not properly convert (and returned null) when the number of digits for fractions of seconds was less than 3 digits or more than 9 digits. The solution is to pad with zeros when there is less than 3 digits and to truncate when there is more than 3. commit f75dfe711b357800762a93dc5059d9a19da9ee61 Author: Luke Cwik <lc...@google.com> Date: 2017-06-30T16:00:13Z Properly convert milliseconds whether there's less than 3/more than 9 digits. This closes #3469 commit 56cb6c51748fde6ad56522733ab10edca062e802 Author: Kenneth Knowles <k...@google.com> Date: 2017-06-13T17:29:50Z Add support for PipelineOptions parameters This is a step towards eliminating catch-all context parameters and making DoFns express their fine-grained data needs. commit 0e429b33ff85eba08da5018c9febd0b99b44f720 Author: Kenneth Knowles <k...@google.com> Date: 2017-06-30T22:39:20Z This closes #3355: Add support for PipelineOptions parameters commit f99ab1a472868e4ce175a86e5d76823b1c09c10b Author: Kenneth Knowles <k...@google.com> Date: 2017-07-01T04:42:17Z Fix DoFn javadoc: StateSpec does not require a key commit ce4e5174750ae2c4b1691c6994bd0f92e161 Author: Ismaël MejÃa <ieme...@apache.org> Date: 2017-06-04T20:55:05Z Make modules that depend on Hadoop and Spark use the same version property commit 11010cf08ec9ca90a843cc15ea8f0fd4910f7fab Author: Jean-Baptiste Onofré <jbono...@apache.org> Date: 2017-07-03T07:26:40Z This closes #3291 commit 75475ef3dc23a09fa9bbba478d6fdbc468f7dd2e Author: Ismaël MejÃa <ieme...@gmail.com> Date: 2017-06-28T14:58:55Z [BEAM-2530] Fix compilation of modules with Java 9 that depend on jdk.tools commit 0bd47c07659909c6b3c2d04fbdb96cc30a6e0243 Author: Jean-Baptiste Onofré <jbono...@apache.org> Date: 2017-07-03T12:23:18Z [BEAM-2530] This closes #3460 commit 68f1fb64fd2565e287e322d715ca778d01e7137b Author: Ahmet Altay <al...@google.com> Date: 2017-07-01T00:37:33Z For GCS operations use an http client with a default timeout value. commit b8ac32641183b06a87517a169c8ecc5371996aed Author: Ahmet Altay <al...@google.com> Date: 2017-07-04T18:07:31Z This closes #3486 commit 51877a3405dbf778c3bb88f19bb194e54c3b3def Author: Jean-Baptiste Onofré <jbono...@apache.org> Date: 2017-07-05T14:47:29Z [maven-release-plugin] prepare branch release-2.1.0 commit 7f0723cf7fd587c4f8dbe8a8b1c9d298a7b1e5e3 Author: Jean-Baptiste Onofré <jbono...@apache.org> Date: 2017-07-05T14:47:38Z [maven-release-plugin] prepare for next development iteration commit 14fa7f79f0830739122b7573e032ad0aea172a98 Author: Jean-Baptiste Onofré <jbono...@apache.org> Date: 2017-07-05T14:52:48Z Update Python SDK version commit 06897b1cc142f658437ac7779c849e5182e331f1 Author: Jason Kuster <jasonkus...@google.com> Date: 2017-06-09T08:39:15Z Website Mergebot Job Signed-off-by: Jason Kuster <jasonkus...@google.com> commit f0549b464737d0d11ae1da8e0534622b10247208 Author: Thomas Groh <tg...@google.com> Date: 2017-07-05T16:22:20Z This closes #3485 commit 6ca410a908f1f4e7ac1e141ee1335f7a537bb150 Author: Luke Cwik <lc...@google.com> Date: 2017-07-05T17:38:44Z [BEAM-2553] Update Maven exec plugin to 1.6.0 to incorporate messaging improvements commit 39c80f640749fed5f55a0c8f25c849e5f7c3d179 Author: Luke Cwik <lc...@google.com> Date: 2017-07-05T19:26:28Z [BEAM-2553] Update Maven exec plugin to 1.6.0 to incorporate messaging improvements This closes #
[GitHub] beam pull request #3479: [BEAM-79] Merge master into gearpump-runner branch
GitHub user manuzhang opened a pull request: https://github.com/apache/beam/pull/3479 [BEAM-79] Merge master into gearpump-runner branch Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/manuzhang/beam sync-master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3479.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3479 commit cab4d8969e7f95b0ece59838ad2d578e75d38823 Author: Kenneth Knowles <k...@google.com> Date: 2017-06-22T03:25:31Z DataflowRunner: Reject merging windowing for stateful ParDo commit 5f8cfa7412e8f4ce97d6df4413fecbea22b4b5e1 Author: Kenneth Knowles <k...@google.com> Date: 2017-06-23T03:05:28Z This closes #3419: DataflowRunner: Reject merging windowing for stateful ParDo commit 649994b353afe28c917969609c7a1a47a4f39aaf Author: Rune Fevang <fev...@exabel.com> Date: 2017-06-15T11:51:12Z Allow output from FinishBundle in DoFnTester commit b3c36256e32f4a4a616d0e152686d29f1174c5cd Author: Kenneth Knowles <k...@google.com> Date: 2017-06-23T03:07:48Z This closes #3367: Allow output from FinishBundle in DoFnTester commit 8dcda6e40355af13f4d92fcd44aae4539a225a4a Author: Sourabh Bajaj <sourabhba...@google.com> Date: 2017-06-23T00:08:20Z [BEAM-2497] Fix the reading of concat gzip files commit 336b7f1cffc8709cd480a243a16de95bce39b53e Author: Chamikara Jayalath <chamik...@google.com> Date: 2017-06-23T05:13:05Z This closes #3428 commit f291713b28e3ba0246a8c0a710c71506cd0a0f91 Author: Etienne Chauchot <echauc...@gmail.com> Date: 2017-06-21T08:39:39Z [BEAM-2489] Use dynamic ES port in HIFIOWithElasticTest commit 6cef5c7b625c6b5c2078084ba611b1c2adccc998 Author: Jean-Baptiste Onofré <jbono...@apache.org> Date: 2017-06-23T13:01:42Z [BEAM-2489] This closes #3411 commit fd8f15f1ac761425dc791a455b042a8846081f48 Author: Mark Liu <mark...@google.com> Date: 2017-06-22T21:04:00Z [BEAM-2745] Add Jenkins Suite for Python Performance Test commit af69e979ab3037274bdd78904bdeb5016563bcad Author: Ahmet Altay <al...@google.com> Date: 2017-06-23T21:03:17Z This closes #3426 commit 32095487e56b63b5c1aa690bb6e098375cb108d5 Author: Vikas Kedigehalli <vika...@google.com> Date: 2017-06-23T18:50:12Z Fix python fn API data plane remote grpc port access commit 9acce7150ee8fa6c9e50049155ad7b85b646f98e Author: Ahmet Altay <al...@google.com> Date: 2017-06-23T22:13:30Z This closes #3431 commit 903da41ac5395e76c44ef8ae1c8a695569e23abb Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-06-23T22:01:42Z Avoid pickling the entire pipeline per-transform. commit a90e40ae9f748c1c8392c1198469f3229a06ed70 Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-06-23T23:39:52Z Closes #3433 commit e45f522d6e945899c20259ebf8faca105c2e552e Author: Valentyn Tymofieiev <valen...@google.com> Date: 2017-06-23T23:44:49Z Fix a typo in function args commit 16f87f49f20796e29d01ed363a9097ea5420583c Author: Ahmet Altay <al...@google.com> Date: 2017-06-24T00:34:21Z This closes #3435 commit 926f949580c3a21df72a8836feda1f6b947850ec Author: Charles Chen <c...@google.com> Date: 2017-06-26T20:00:14Z Remove old deprecated PubSub code commit 1ea1de4aa9d32e3c5a596ccd7d84aff1cc2a7428 Author: Ahmet Altay <al...@google.com> Date: 2017-06-26T21:23:08Z This closes #3440 commit bec32fe93c6b5c16563d7ea4b877a2dee3352fee Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-06-16T21:56:07Z Reintroduces DoFn.ProcessContinuation (Dataflow worker compatibility part) commit 2052cc7689b4ed53f817f56dd71a32235fb083ca Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-06-23T17:16:30Z Bump Dataflow worker to 0623 commit 95e6bbe5065a983f9c71b79b20df1fda83c9fd1b Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-06-27T00:25:14Z This closes #3383: [BEAM-2447] Reintroduces DoFn.ProcessContinuation (Dataflow worker compatibility part) commit eb379e76adaa9c4b4e24a4b3c5757be8523d95c4 Author: Charles Chen <c...@google.com> Date: 2017-06-26T23:54:00Z Implement streaming GroupByKey in Python DirectRunner
[GitHub] beam pull request #3456: [BEAM-2526] Add direct-runner as runtime dependency...
Github user manuzhang closed the pull request at: https://github.com/apache/beam/pull/3456 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3456: [BEAM-2526] Add direct-runner as runtime dependency...
GitHub user manuzhang opened a pull request: https://github.com/apache/beam/pull/3456 [BEAM-2526] Add direct-runner as runtime dependency and default profile Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/manuzhang/beam sql Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3456.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3456 commit 3130f47921b4012d0800bb0ef5ea3efe4b767ff9 Author: manuzhang <owenzhang1...@gmail.com> Date: 2017-06-28T00:56:37Z [BEAM-2526] Add direct-runner as runtime dependency and default profile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3438: [BEAM-2514] Use option name in missing required val...
GitHub user manuzhang opened a pull request: https://github.com/apache/beam/pull/3438 [BEAM-2514] Use option name in missing required value message Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/manuzhang/beam BEAM-2514 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3438.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3438 commit ab85c2ce8e9948e340d2019085b6cbb8e7c4 Author: manuzhang <owenzhang1...@gmail.com> Date: 2017-06-26T12:55:13Z [BEAM-2514] Use option name in missing required value message --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3388: [BEAM-79] Sync gearpump-runner branch with master
GitHub user manuzhang opened a pull request: https://github.com/apache/beam/pull/3388 [BEAM-79] Sync gearpump-runner branch with master Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/manuzhang/beam sync-master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3388.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3388 commit 8362bdb9cd35cc02ed179b3a64fd72f1264a99be Author: JingsongLi <lzljs3620...@aliyun.com> Date: 2017-06-07T17:31:34Z [BEAM-2423] Abstract StateInternalsTest for the different state internals commit f9d51aa5cc718c44c82ec4773dbd16a57e7d394e Author: Aljoscha Krettek <aljoscha.kret...@gmail.com> Date: 2017-06-12T20:35:35Z This closes #3313 commit 809f17876d847002ba76979cb3362451fa01c110 Author: Charles Chen <c...@google.com> Date: 2017-06-12T21:17:50Z Reverse removal of NativeWrite evaluator in Python DirectRunner commit 86e04893a14d9b6aa29b78e25dc064879b712fb3 Author: Ahmet Altay <al...@google.com> Date: 2017-06-12T22:43:44Z This closes #3346 commit f0f98c70b0b01c43ce1b093c2035b20bd90ba907 Author: Thomas Groh <tg...@google.com> Date: 2017-06-12T18:07:47Z Cleanup Combine Tests with Context Split out the "shared" bit of all the accumulators, so they show up as an explicit component of the final result string. Update timestamped creation logic. commit ce3dd4583294fcbb65121f5fecfc49a8b85634ad Author: Thomas Groh <tg...@google.com> Date: 2017-06-12T22:48:38Z This closes #3345 commit 1ac18b2eb1371422e60d50a8c3f37b3b24d59611 Author: Thomas Groh <tg...@google.com> Date: 2017-06-12T23:55:59Z Check for Deferral on Non-additional inputs Because Side Inputs are represented within the expanded inputs, the check that the transform is a Combine with Side Inputs would never be hit. This ensures that we do not consider additional inputs during the check to defer evaluation of the node. commit 32d55323cd5a88f604ad9c763ab265de845df08c Author: Kenneth Knowles <k...@google.com> Date: 2017-06-13T03:20:40Z This closes #3347: Check for Deferral on Non-additional inputs commit 6859f80400bb16bbca34ed282e7a5e8a1328f955 Author: Ismaël MejÃa <ieme...@apache.org> Date: 2017-06-05T21:20:27Z [BEAM-2412] Update HBaseIO to use HBase client 1.2.6 commit fe3d55403334bf235952fff33e6a90c0f4445a0d Author: Jean-Baptiste Onofré <jbono...@apache.org> Date: 2017-06-13T05:59:30Z [BEAM-2412] This closes #3327 commit 4c36508733a69fafce0f7dfb86c71eee5eb6bc84 Author: JingsongLi <lzljs3620...@aliyun.com> Date: 2017-06-07T06:34:25Z Use CoderTypeSerializer and remove unuse code in FlinkStateInternals commit 10b166b355a03daeae78dd1e71016fc72805939d Author: JingsongLi <lzljs3620...@aliyun.com> Date: 2017-06-07T06:40:30Z [BEAM-1483] Support SetState in Flink runner and fix MapState to be consistent with InMemoryStateInternals. commit 4d18606378f43c7b0d3ac05d45ca6e0570e49eef Author: JingsongLi <lzljs3620...@aliyun.com> Date: 2017-06-13T02:15:33Z Add set and map readable test to StateInternalsTest commit 7126fdc6ee5671e99a2dede3f25ba616aa0e8fa4 Author: Aljoscha Krettek <aljoscha.kret...@gmail.com> Date: 2017-06-13T09:35:43Z This closes #3309 commit e71eb66ae319bdf0cdad1fe9b54662962c8e8f16 Author: Robert Bradshaw <rober...@google.com> Date: 2017-06-09T23:44:55Z Actually test the fn_api_runner. The test suite was not being run due to a typo. Fix breakage due to changes in the code in the meantime. commit bd83612f246c1654b80df19890a25f8de48a063a Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-06-13T15:56:36Z Closes #3339 commit b547b5a1eee61e293ff2f8ccfac57f308867328c Author: Maria Garcia Herrero <mari...@google.com> Date: 2017-06-10T06:34:59Z Make unique test names for value-provider arguments commit e906fe9c3a1e12f01d84e05fbb872d47b28d2fe9 Author: Ahmet Altay <al...@google.com> Date: 2017-06-13T16:34:16Z This closes #3341 commit ee728f1b2f617dac8e5cd729cacf1a46911021e0 Author: Vikas Kedigehalli <vika...@google.com> Date: 2017-06-13T06:11:22Z Fix Window
[GitHub] beam pull request #3181: [BEAM-79] Activate pre-commit tests and fix failure
Github user manuzhang closed the pull request at: https://github.com/apache/beam/pull/3181 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3292: [BEAM-79] Merge master into gearpump-runner
GitHub user manuzhang opened a pull request: https://github.com/apache/beam/pull/3292 [BEAM-79] Merge master into gearpump-runner Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/manuzhang/beam sync-master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3292.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3292 commit 66e564e507b1ad5b9e379e043a193b556e1f3e76 Author: Borisa Zivkovic <borisa.zivko...@huawei.com> Date: 2017-05-15T08:24:10Z [BEAM-1702] Document how to use BigtableIO with local emulator commit 8ef812defc5237934f4f3efa6bd8c4c4240cd7d0 Author: Dan Halperin <dhalp...@google.com> Date: 2017-05-18T20:07:13Z This closes #3141 commit a8d2125e5783a556056e88dad8fe3c0a397920d5 Author: huafengw <fvunic...@gmail.com> Date: 2017-05-09T03:21:44Z [BEAM-659] WindowFn#isCompatible should provide a meaningful reason commit 8c572ef0b998a4fa529e2f4ba0e622b3db9f4ec6 Author: Kenneth Knowles <k...@google.com> Date: 2017-05-18T20:27:59Z This closes #2985: WindowFn#isCompatible should provide a meaningful reason commit 26a615e885b39f92dc81d383c8115906c280187a Author: Andrew Martin <andrewsmartin...@gmail.com> Date: 2017-05-18T19:52:09Z Remove incorrect `Message` bound on writeAvros commit 6a7eeeb932ffb33c876c0d11d7f325c47606d8de Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-05-18T20:52:33Z This closes #3176: Remove incorrect `Message` bound on writeAvros commit 790e7fe6653b926044d3dfecdccbc2fda9c998f0 Author: Thomas Groh <tg...@google.com> Date: 2017-03-21T22:06:58Z Add ParDos Add ParDoPayloadTranslator to PTransformTranslator commit 2e0cf008303ff5ed48268550b9fe3cd1aa748c15 Author: Thomas Groh <tg...@google.com> Date: 2017-05-18T22:41:17Z This closes #2597 commit 7a2e829c849d3828353994b8eddca4784a900aca Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-05-15T22:54:03Z Splits SplittableParDo into a core-construction part and a KWI-related part commit c93b3aa3d2081558a776fa12c564efb5f15090ab Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-05-18T23:15:00Z This closes #3156 commit 54808be14ff2e86088807adc3c0dc0965c8a6f03 Author: Dan Halperin <dhalp...@google.com> Date: 2017-05-18T22:43:31Z [BEAM-2318] Ensure that tables that don't exist are not created We use the same test table name in two tests: one that expects the table not to exist, another that creates the table. Obviously, the correctness of this will depend on the order in which unit tests are executed. Fix the flake by using different and more explicit test names. commit f458065dad0429ba2639510ea460cf7f2679ed36 Author: Dan Halperin <dhalp...@google.com> Date: 2017-05-18T23:41:41Z This closes #3179 commit 1da8da79dcc623c53b35d97419566b736447f6b6 Author: Dan Halperin <dhalp...@google.com> Date: 2017-05-18T20:19:29Z [BEAM-2391] Clone Scan in HBaseReader commit 49245080ad6393b5678077caa047f7a6ef0efffc Author: Ismaël MejÃa <ieme...@apache.org> Date: 2017-05-19T08:50:06Z This closes #3178 commit f969ea443c90345cf8c33f953c399e46223e8d03 Author: Luke Cwik <lc...@google.com> Date: 2017-05-15T20:39:12Z [BEAM-2252] Use pluginManagement instead of redefining maven-shade-plugin to inherit build order from root pom.xml commit ea5b9dabbfb55ff47a4f5983a8cbf04b0949c77c Author: Luke Cwik <lc...@google.com> Date: 2017-05-19T17:37:10Z [BEAM-2252] Use pluginManagement instead of redefining maven-shade-plugin to inherit build order from root pom.xml This closes #3152 commit 3efb34b8322ac5933d9d71255d67e99d8e8361d7 Author: Ahmet Altay <al...@google.com> Date: 2017-05-19T02:00:00Z update dataflow python container commit a58275a2f904ab1304de7ca5295f27a4f6df6f1d Author: Ahmet Altay <al...@google.com> Date: 2017-05-19T17:43:15Z This closes #3182 commit bac7bde5f878e6b2104c2e7cf3d01894a8f1b71d Author: Mark Liu <mark...@google.com> Date: 2017-05-17T21:24:25Z [BEAM-2164] Fix generate-sources.sh if directory contains space commit ce00d2466a9495f4eb262b5f7d2afec5788fcfe2 Author: Kenneth
[GitHub] beam pull request #3181: [BEAM-79] Respect WindowFn#getOutputTime in gearpum...
GitHub user manuzhang opened a pull request: https://github.com/apache/beam/pull/3181 [BEAM-79] Respect WindowFn#getOutputTime in gearpump-runner Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/manuzhang/beam fix_windowing_test Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3181.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3181 commit 7328ff21f2b805801123c0458d4fad22563ad384 Author: manuzhang <owenzhang1...@gmail.com> Date: 2017-05-19T01:19:42Z [BEAM-79] Respect WindowFn#getOutputTime in gearpump-runner --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3172: [BEAM-79] Sync gearpump-runner with master
GitHub user manuzhang opened a pull request: https://github.com/apache/beam/pull/3172 [BEAM-79] Sync gearpump-runner with master Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/manuzhang/beam sync-master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3172.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3172 commit 44c9385ddbcfbdf904c5d054754dca3ddde88ec2 Author: Dan Halperin <dhalp...@google.com> Date: 2017-05-06T00:16:34Z [BEAM-2190] pom.xml: do a better job of dependency management Even if Beam appears to have the correct dependencies, we cannot guarantee that modules that depend on us transitively get the right dependencies. For example, even though grpc-protobuf-lite has protobuf-lite excluded, and the Maven Enforcer banned-dependencies check passes... if a user happens to get a transitive dependency on grpc-all first, they may pull in grpc-protobuf from that other source without the exclusion. Thus we need to exclude protobuf-lite from grpc-all as well. While we're here, also add guava-jdk5 to the set of banned dependencies, though (as above) we cannot currently properly identify the places it might be transitively exposed in a users' pom.xml. commit 2b0e699b8f6a0aac26ac57c536e8d45cee1c3d2e Author: Dan Halperin <dhalp...@google.com> Date: 2017-05-08T18:35:56Z This closes #2934 commit 5bac40efcc7bcd9d53183579ec6089c23aeda782 Author: Dan Halperin <dhalp...@google.com> Date: 2017-05-06T02:06:03Z [BEAM-2212] FileBasedSource: improve message when logging. ValueProvider should not be printed, rather the string instead. commit 9423babd8f827e843723c218441e9a91aaa7b361 Author: Dan Halperin <dhalp...@google.com> Date: 2017-05-08T16:59:16Z [BEAM-2212] FileBasedSource: refactor to remove uses of fileOrPatternSpec.get() Makes it less likely to have errors from printing ValueProviders instead of runtime values commit e1791c3f83a007013625b43f9976e7ffa64e8a49 Author: Dan Halperin <dhalp...@google.com> Date: 2017-05-08T18:48:42Z This closes #2956 commit 13e51c9cd114eec73c47b71f46214dd92ee81048 Author: Dan Halperin <dhalp...@google.com> Date: 2017-05-06T02:49:29Z Convert Executor Services to use Daemon Threads This will cause the DirectRunner to automatically shut down when the worker threads are shut down. commit 63c6bea3bbdb52964ac54073326d4ad99ba7432c Author: Thomas Groh <tg...@google.com> Date: 2017-05-08T19:52:09Z This closes #2954 commit 50a4c56de1d3aa0e51e8b2b08b71715be4c2ef80 Author: Sourabh Bajaj <sourabhba...@google.com> Date: 2017-05-07T20:55:20Z [BEAM-2206] Move pipelineOptions into options modules commit 604c2f59e2a928a95106814b0b55b8097dc39faa Author: Ahmet Altay <al...@google.com> Date: 2017-05-08T19:56:15Z This closes #2925 commit d3fc7d18a4a3c394d32d85ef34c08a85712a8b18 Author: Reuven Lax <re...@google.com> Date: 2017-05-08T16:06:55Z TableDescription is allowed to be null. commit c481396edc7a3e906cd79dcc2a356baf9c93895e Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-05-08T20:41:01Z This closes #2953 commit 1fd15bd7441e1be72feb3f9535ffe536d730a531 Author: Reuven Lax <re...@google.com> Date: 2017-05-07T18:50:08Z Shortens PubsubIO names for reading/writing messages. commit d1914ed0c3195e1de2f707e2bb7bc57597b834fe Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-05-08T20:45:25Z This closes #2950 commit b7635aadc2532b5c3bdc38f070865091ebf901c1 Author: Ahmet Altay <al...@google.com> Date: 2017-05-08T20:42:02Z Remove apex README.md. This information is already in the website in quickstart and apex runner pages. https://github.com/apache/beam-site/pull/232 moves small bits of missing content. commit 962e8796d68f9a81b0ae3d1b30d7a4bcadbdaf60 Author: Ahmet Altay <al...@google.com> Date: 2017-05-08T21:28:36Z This closes #2962 commit fba4c010ce90db8b5cdac489d47126ca3ee8db65 Author: Ahmet Altay <al...@google.com> Date: 2017-05-08T22:00:33Z Remove readme.md files that are covered in web site commit 152c5bcc7f6f
[GitHub] beam pull request #2888: [BEAM-79] Merge master into gearpump-runner branch
GitHub user manuzhang opened a pull request: https://github.com/apache/beam/pull/2888 [BEAM-79] Merge master into gearpump-runner branch Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/manuzhang/incubator-beam sync-master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2888.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2888 commit d9595d35176d2572423f6be069982b05eea204dd Author: Maria Garcia Herrero <mari...@google.com> Date: 2017-04-26T23:22:09Z Add extra message to deprecated annotation commit 42e3a6f8588046f257e5c780dec12d16d0314a66 Author: Ahmet Altay <al...@google.com> Date: 2017-04-27T18:25:18Z This closes #2719 commit 9174ebf48391f393761b2ce085d2f165ebb2c7bb Author: Tibor Kiss <tibor.k...@gmail.com> Date: 2017-04-27T12:12:13Z [BEAM-1695] Improve Python-SDK's programming guide commit 884935cb98f0e29bd965fda9182970423d111d7e Author: Ahmet Altay <al...@google.com> Date: 2017-04-27T19:03:22Z This closes #2738 commit e047b69efab9c988011303cf2eda86ac408b38c2 Author: Dan Halperin <dhalp...@google.com> Date: 2017-04-27T15:30:47Z [BEAM-2096] Make DataflowMetrics more resilient DataflowMetrics seems to have many hard-coded assumptions about what will be returned by the Dataflow service that, which will likely break when users use new types of metrics or if the Dataflow service makes minor adjustments in our it sends metrics back to the user. In order for code to continue working in these cases, handle errors by logging and skipping. commit fdf2de9994268f5e1d704aa30a4504ac6ee624b8 Author: Dan Halperin <dhalp...@google.com> Date: 2017-04-27T20:10:32Z This closes #2739 commit f93a27755921e73141fd97de042e6dcd04e10a47 Author: Dan Halperin <dhalp...@google.com> Date: 2017-04-26T00:29:02Z [BEAM-59] Add FileSystems#matchNewResource The new FileSystems API needs a way to generate a ResourceId for a resource that does not exist. This does not come up in sources, because we typically are just matching existing files. However, sinks need the ability to reference a new directory (say, in order to create it). Couldn't think of anything better than a simple function that says "treat this string as a full resource path with the specified type", which is what FileSystems#matchNewResource is. commit 14a90d7c5196d96de66e897687d28202d790439f Author: Dan Halperin <dhalp...@google.com> Date: 2017-04-27T21:56:24Z This closes #2728 commit d3b399fa84bff45b0960368d32d4c69e433f4c94 Author: Thomas Groh <tg...@google.com> Date: 2017-04-26T22:12:54Z Remove Coder Encoding IDs These were never safe to use, and will not be used once Coders have been migrated to the Runner API representation. commit c4936958834aa55f0f1fcb73fe9b8419c4c9eb9b Author: Thomas Groh <tg...@google.com> Date: 2017-04-27T22:22:55Z This closes #2717 commit 1925a50d860b7b8f8422f1c2f251d0ea11def736 Author: Aviem Zur <aviem...@gmail.com> Date: 2017-04-27T20:48:02Z [BEAM-2029] NullPointerException when using multi output ParDo in Spark runner in streaming mode. commit a46eb1a80074dc9c382da4d71725ffc61f2e65ba Author: Dan Halperin <dhalp...@google.com> Date: 2017-04-27T22:33:39Z This closes #2746 commit 9a3b830c8737859c6afaa6e6e1922b037a78741f Author: Dan Halperin <dhalp...@google.com> Date: 2017-04-27T22:35:10Z [BEAM-59] LocalResources: helper utility to create LocalResourceId Primarily for testing. commit f71395bfd8f94cbe8d78098f5bb66a1dce65e74b Author: Dan Halperin <dhalp...@google.com> Date: 2017-04-27T23:30:50Z This closes #2751 commit 95fdb8f4dbb6bb8ec6d80152aa85cc970ab13366 Author: Dan Halperin <dhalp...@google.com> Date: 2017-04-27T20:51:53Z WriteFiles: trivial message fix commit 674ae9a475f1f236230697d5cb1c33d36fec49db Author: Dan Halperin <dhalp...@google.com> Date: 2017-04-28T00:20:54Z This closes #2755 commit 06f4ef40bec7d6d3c5826fb9e0c164eb9c05efe7 Author: Ahmet Altay <al...@google.com> Date: 2017-04-28T00:14:41Z Add a simple snippet for a composite transform.
[GitHub] beam pull request #2610: Merge master into gearpump-runner branch
GitHub user manuzhang opened a pull request: https://github.com/apache/beam/pull/2610 Merge master into gearpump-runner branch Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/manuzhang/incubator-beam merge_master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2610.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2610 commit e0e39a97560933f99eaee98549877fd5f8f49a52 Author: Dan Halperin <dhalp...@google.com> Date: 2017-04-10T17:16:06Z This closes #2481 commit a3a15a5370467db72f8be2fa031483349e49f6ab Author: Ahmet Altay <al...@google.com> Date: 2017-04-08T00:54:14Z enable test_multi_valued_singleton_side_input test commit 80d2548f258074931a62e412609363dfd2305086 Author: Ahmet Altay <al...@google.com> Date: 2017-04-10T17:49:16Z This closes #2467 commit 74e31c350986d093be1a0b53d001b3376def8b69 Author: Thomas Weise <t...@apache.org> Date: 2017-04-08T20:01:01Z BEAM-1053 ApexGroupByKeyOperator serialization issues commit 836e8e4aab239dd41f81de2f1553850b3fd3d716 Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-04-10T17:55:50Z This closes #2473 commit 33d4a02bb28e5a1c09513dc3e7701b30df148943 Author: Vikas Kedigehalli <vika...@google.com> Date: 2017-04-03T17:01:45Z Create as custom source commit fc1006500d14afd01a3a34527c0ea9dc94ac09e9 Author: Chamikara Jayalath <chamik...@google.com> Date: 2017-04-10T21:18:30Z This closes #2446 commit b7d7adc879694cf5b22f80a26a46982730a483ec Author: Thomas Groh <tg...@google.com> Date: 2017-04-07T16:17:19Z Add SdkComponents This takes SDK objects and assigns IDs to them. It is effectively a ComponentsBuilder context where a component is referred to by the Java object which is being translated, rather than by an opaque string or protocol buffer. commit 8afa62398d51e3b11215970b139ab839ccb2e91a Author: Thomas Groh <tg...@google.com> Date: 2017-04-10T22:14:34Z This closes #2459 commit 11a3af8474998bd2b874cfeede82f353999ae3e3 Author: Ahmet Altay <al...@google.com> Date: 2017-04-10T21:13:00Z Upgrade dependencies. Minor version changes to use latest versions of avro, pyyaml, google-apitools, googledatastore. commit 1c7a974ab00b8f4a8c089520aeb15ec29e128977 Author: Ahmet Altay <al...@google.com> Date: 2017-04-10T22:18:37Z This closes #2484 commit 678833096550cf81da0fe026ab978626e395ef11 Author: Maria Garcia Herrero <mari...@google.com> Date: 2017-04-09T06:08:44Z Skip query metrics when creating a template commit 1761d1cabf47e34fd9107c5673339aa966148f0e Author: Ahmet Altay <al...@google.com> Date: 2017-04-10T22:23:21Z This closes #2475 commit 9e453fabe2bf448552ab5706130495e5ea4cf1c2 Author: Vikas Kedigehalli <vika...@google.com> Date: 2017-04-08T02:22:27Z Fix GroupByKeyInputVisitor for Direct Runner commit 7fd9c6516ea5f3d290fd95507559b0c99940 Author: Ahmet Altay <al...@google.com> Date: 2017-04-10T22:28:12Z This closes #2470 commit 40b36686bac65995d606fe5cea7aa957d0ef3f9d Author: Thomas Groh <tg...@google.com> Date: 2017-04-10T21:35:59Z Move WindowingStrategies to runners-core-construction commit 0a0b1c80c97949954add4b4357fe29409a08197d Author: Thomas Groh <tg...@google.com> Date: 2017-04-11T02:24:52Z This closes #2486 commit 62e23b71e3b964033c15707629b560dfc8204654 Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-04-08T00:54:18Z Removes DoFn.ProcessContinuation completely commit 159ac58122258e42b8faa94dad20e9fb82a4fe34 Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-04-10T22:44:14Z Bump Dataflow worker to 20170410 commit c58f4f89b3026e130b63dc14731a8ee8615e92f5 Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-04-11T05:51:54Z This closes #2468 commit 5f7c772cc4d21b220fa3b5dcec8b7d5bdba8685f Author: Dan Halperin <dhalp...@google.com> Date: 2017-04-07T21:50:42Z PubsubIO: remove support for BoundedReader Google Cloud Pub/Sub is not currently that useful in bo
[GitHub] beam pull request #2241: Sync gearpump-runner branch with master
GitHub user manuzhang opened a pull request: https://github.com/apache/beam/pull/2241 Sync gearpump-runner branch with master Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/manuzhang/incubator-beam gearpump-runner-merge Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2241.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2241 commit 3120fd9a499b01b08659edc4e045f5abbcc3ae07 Author: Kenneth Knowles <k...@google.com> Date: 2017-02-24T02:06:56Z Condense FunctionSpec and remove SdkFunctionSpec, merging data and params commit 49c9de1e84e0f322dfd4426d8f1d523aee1529e9 Author: Kenneth Knowles <k...@google.com> Date: 2017-02-24T22:39:19Z This closes #2094: Concretize generic bits of the Runner API graph structure Condense FunctionSpec and remove SdkFunctionSpec, merging data and params Inline PTransform to GraphNode, removing generic design commit 8a06134c6543394849a5f8856418dd699fd2e2a7 Author: Kenneth Knowles <k...@google.com> Date: 2017-02-24T23:07:35Z This closes #2103: Ignore HBaseIOTest temporarily commit 1b1bb0b9aeb8f4b131fbcf62b117ef3f55db2f61 Author: Ahmet Altay <al...@google.com> Date: 2017-02-24T21:29:01Z Change the package name to apache-beam Also, allow direct installations from PyPI (once the package is published.) This will eventually make sdk_location an optional argument. commit b1fd55f0c0a10c9376ee084582532df1e6fd7359 Author: Ahmet Altay <al...@google.com> Date: 2017-02-24T23:13:25Z This closes #2102 commit fa13c7b9362c65d8d61e914d3822dea025400d5b Author: Vikas Kedigehalli <vika...@google.com> Date: 2017-02-24T19:04:55Z Ignore undeclared outputs in the java fn harness commit d79fb3e06ddc8ff7d06645d0c4a2c685a90c250b Author: Dan Halperin <dhalp...@google.com> Date: 2017-02-24T23:16:15Z This closes #2100 commit 7162964f058002db76232fd04d720ce9c3e75585 Author: Ahmet Altay <al...@google.com> Date: 2017-02-24T23:43:17Z Update python sdk readme commit 639f9f4b7195eee3528acabc358f3790f4549c57 Author: Ahmet Altay <al...@google.com> Date: 2017-02-24T23:55:43Z This closes #2105 commit a5b8cb1e11deee7fe7ed152a4718a1f971f58eab Author: Pablo <pabl...@google.com> Date: 2017-02-15T23:16:12Z Making metrics queriable in Python SDK. commit 415546eda7916f6f43b5d101ef5c15aa7948dac9 Author: Ahmet Altay <al...@google.com> Date: 2017-02-25T01:39:06Z This closes #2016 commit 2736c4ae901ad428f8e05fb7a80ad72f0318735e Author: Ismaël MejÃa <ieme...@gmail.com> Date: 2017-02-24T15:46:17Z [BEAM-1554] Add verification of Coder structural equality after enc/dec This also improves the class documentation. commit 437ba250525f5b611a3e3c31bee782aea24a16ce Author: Kenneth Knowles <k...@google.com> Date: 2017-02-25T03:23:13Z This closes #2098: Add verification of Coder structural equality after encoding/decoding commit 32cd4836d2441c10fa71df4344c6b546c2c0ca3a Author: Ahmet Altay <al...@google.com> Date: 2017-02-25T04:19:49Z Disable newly added tests, that is failing is post-commits. commit 38a01e9cf8d959ace3baa1593f8dc17d4f12954d Author: Ahmet Altay <al...@google.com> Date: 2017-02-26T02:30:50Z This closes #2107 commit acc10130982168ad5c4f29b2af3be67e53f7c967 Author: Stas Levin <stasle...@gmail.com> Date: 2017-02-23T22:02:14Z Updated TestPipeline's javadoc. commit 4bba380cb011e38bd78c1ec899a6672a6b02fd05 Author: Stas Levin <stasle...@apache.org> Date: 2017-02-26T15:28:26Z This closes #2089 commit b74f8fa30aea130f990c152a1c91a509c3b6d6c9 Author: Aviem Zur <aviem...@gmail.com> Date: 2017-02-23T05:09:55Z Remove periods from step name in SparkBeamMetric to avoid inconsistent hierarchies in Graphite metric sink commit 3082178b3a35143573caa2b3d9afbf2babb5b2b5 Author: Stas Levin <stasle...@apache.org> Date: 2017-02-27T12:16:55Z This closes #2083 commit 0ae2a3851cbfe1ec2f2c7237954b18c9951c76a3 Author: JingsongLi <lzljs3620...@aliyun.com> Date: 2017-02-27T10:08:34Z [BEAM-156
[GitHub] beam pull request #1429: [BEAM-970] add side-input support to gearpump-runne...
Github user manuzhang closed the pull request at: https://github.com/apache/beam/pull/1429 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2150: [BEAM-79] Add SideInput support for GearpumpRunner
GitHub user manuzhang opened a pull request: https://github.com/apache/beam/pull/2150 [BEAM-79] Add SideInput support for GearpumpRunner Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/manuzhang/incubator-beam gearpump-runner-fix Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2150.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2150 commit 08b18bf326a7ac1bae485fbdf73aeab0de46c716 Author: manuzhang <owenzhang1...@gmail.com> Date: 2017-02-14T04:33:31Z [BEAM-79] Add SideInput support for GearpumpRunner --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #1828: [BEAM-79] Fix PostCommit test confs for Gearpump ru...
GitHub user manuzhang opened a pull request: https://github.com/apache/beam/pull/1828 [BEAM-79] Fix PostCommit test confs for Gearpump runner Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/manuzhang/incubator-beam gearpump-runner-post-commit Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/1828.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1828 commit 4fd216b6b9f7427a7df118a60202752ebeb1836d Author: manuzhang <owenzhang1...@gmail.com> Date: 2017-01-24T05:16:07Z [BEAM-79] Fix PostCommit test confs for Gearpump runner --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
