GitHub user manuzhang opened a pull request: https://github.com/apache/beam/pull/3388
[BEAM-79] Sync gearpump-runner branch with master Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-<Jira issue #>] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `<Jira issue #>` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/manuzhang/beam sync-master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3388.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3388 ---- commit 8362bdb9cd35cc02ed179b3a64fd72f1264a99be Author: JingsongLi <lzljs3620...@aliyun.com> Date: 2017-06-07T17:31:34Z [BEAM-2423] Abstract StateInternalsTest for the different state internals commit f9d51aa5cc718c44c82ec4773dbd16a57e7d394e Author: Aljoscha Krettek <aljoscha.kret...@gmail.com> Date: 2017-06-12T20:35:35Z This closes #3313 commit 809f17876d847002ba76979cb3362451fa01c110 Author: Charles Chen <c...@google.com> Date: 2017-06-12T21:17:50Z Reverse removal of NativeWrite evaluator in Python DirectRunner commit 86e04893a14d9b6aa29b78e25dc064879b712fb3 Author: Ahmet Altay <al...@google.com> Date: 2017-06-12T22:43:44Z This closes #3346 commit f0f98c70b0b01c43ce1b093c2035b20bd90ba907 Author: Thomas Groh <tg...@google.com> Date: 2017-06-12T18:07:47Z Cleanup Combine Tests with Context Split out the "shared" bit of all the accumulators, so they show up as an explicit component of the final result string. Update timestamped creation logic. commit ce3dd4583294fcbb65121f5fecfc49a8b85634ad Author: Thomas Groh <tg...@google.com> Date: 2017-06-12T22:48:38Z This closes #3345 commit 1ac18b2eb1371422e60d50a8c3f37b3b24d59611 Author: Thomas Groh <tg...@google.com> Date: 2017-06-12T23:55:59Z Check for Deferral on Non-additional inputs Because Side Inputs are represented within the expanded inputs, the check that the transform is a Combine with Side Inputs would never be hit. This ensures that we do not consider additional inputs during the check to defer evaluation of the node. commit 32d55323cd5a88f604ad9c763ab265de845df08c Author: Kenneth Knowles <k...@google.com> Date: 2017-06-13T03:20:40Z This closes #3347: Check for Deferral on Non-additional inputs commit 6859f80400bb16bbca34ed282e7a5e8a1328f955 Author: Ismaël MejÃa <ieme...@apache.org> Date: 2017-06-05T21:20:27Z [BEAM-2412] Update HBaseIO to use HBase client 1.2.6 commit fe3d55403334bf235952fff33e6a90c0f4445a0d Author: Jean-Baptiste Onofré <jbono...@apache.org> Date: 2017-06-13T05:59:30Z [BEAM-2412] This closes #3327 commit 4c36508733a69fafce0f7dfb86c71eee5eb6bc84 Author: JingsongLi <lzljs3620...@aliyun.com> Date: 2017-06-07T06:34:25Z Use CoderTypeSerializer and remove unuse code in FlinkStateInternals commit 10b166b355a03daeae78dd1e71016fc72805939d Author: JingsongLi <lzljs3620...@aliyun.com> Date: 2017-06-07T06:40:30Z [BEAM-1483] Support SetState in Flink runner and fix MapState to be consistent with InMemoryStateInternals. commit 4d18606378f43c7b0d3ac05d45ca6e0570e49eef Author: JingsongLi <lzljs3620...@aliyun.com> Date: 2017-06-13T02:15:33Z Add set and map readable test to StateInternalsTest commit 7126fdc6ee5671e99a2dede3f25ba616aa0e8fa4 Author: Aljoscha Krettek <aljoscha.kret...@gmail.com> Date: 2017-06-13T09:35:43Z This closes #3309 commit e71eb66ae319bdf0cdad1fe9b54662962c8e8f16 Author: Robert Bradshaw <rober...@google.com> Date: 2017-06-09T23:44:55Z Actually test the fn_api_runner. The test suite was not being run due to a typo. Fix breakage due to changes in the code in the meantime. commit bd83612f246c1654b80df19890a25f8de48a063a Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-06-13T15:56:36Z Closes #3339 commit b547b5a1eee61e293ff2f8ccfac57f308867328c Author: Maria Garcia Herrero <mari...@google.com> Date: 2017-06-10T06:34:59Z Make unique test names for value-provider arguments commit e906fe9c3a1e12f01d84e05fbb872d47b28d2fe9 Author: Ahmet Altay <al...@google.com> Date: 2017-06-13T16:34:16Z This closes #3341 commit ee728f1b2f617dac8e5cd729cacf1a46911021e0 Author: Vikas Kedigehalli <vika...@google.com> Date: 2017-06-13T06:11:22Z Fix WindowValueCoder for large timestamps commit c33e9b446ef071de02db77883517b308a205c9ff Author: Ahmet Altay <al...@google.com> Date: 2017-06-13T17:11:33Z This closes #3350 commit 33662b92f1a9936f594223c9aee1a7233f59a569 Author: chamik...@google.com <chamik...@google.com> Date: 2017-06-08T21:56:24Z Adds ability to dynamically replace PTransforms during runtime. To this end, adds two interfaces, PTransformMatcher and PTransformOverride. Currently only supports replacements where input and output types are an exact match (we have to address complexities due to type hints before supporting replacements with different types). This will be used by SplittableDoFn where matching ParDo transforms will be dynamically replaced by SplittableParDo. commit 7d0f24a0d91fb1f45027f794d6ba00082f2332ce Author: chamik...@google.com <chamik...@google.com> Date: 2017-06-13T18:47:28Z This closes #3333 commit 3a48e47c73520580021c21a037ab412a761d0eeb Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-06-12T18:51:39Z Improves message when transitively serializing PipelineOptions commit 996e35c1de8310a265c890cf8ca8e92416b3d16b Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-06-13T20:04:28Z This closes #3344: Improves message when transitively serializing PipelineOptions commit 9115af488ceb907de121313ffa096d58a0ccc1e1 Author: Mairbek Khadikov <mair...@google.com> Date: 2017-06-07T23:27:01Z SpannerIO: Introduced a MutationGroup. Allows to group together mutation in a logical bundle that is submitted in the same transaction. commit 646caf255510c735488b88ecc69edd9b9bde4081 Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-06-13T20:28:47Z This closes #3319: [BEAM-1542] SpannerIO: Introduced a MutationGroup commit 7e4e51fda633d208c4cc5e88182e5db16156f2cb Author: Charles Chen <c...@google.com> Date: 2017-06-13T22:03:15Z Move Runner API protos to portability/api commit eebff9089e441599b70528b4897c2db1535626c3 Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-06-13T23:25:38Z Closes #3354 commit 8a850af3304a48618739dc23e286800dc0c4641a Author: Thomas Groh <tg...@google.com> Date: 2017-06-13T19:50:58Z Do not produce Unprocessed Inputs if all inputs were Processed This stops the WatermarkManager "Pending Bundles" from growing without bound. commit dd9abc397c67b47f95414bfd5debb08b7b6a6eaf Author: Thomas Groh <tg...@google.com> Date: 2017-06-14T17:21:02Z This closes #3353 ---- --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---