[ https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16015131#comment-16015131 ]
ASF GitHub Bot commented on BEAM-79: ------------------------------------ GitHub user manuzhang opened a pull request: https://github.com/apache/beam/pull/3172 [BEAM-79] Sync gearpump-runner with master Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-<Jira issue #>] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `<Jira issue #>` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/manuzhang/beam sync-master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3172.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3172 ---- commit 44c9385ddbcfbdf904c5d054754dca3ddde88ec2 Author: Dan Halperin <dhalp...@google.com> Date: 2017-05-06T00:16:34Z [BEAM-2190] pom.xml: do a better job of dependency management Even if Beam appears to have the correct dependencies, we cannot guarantee that modules that depend on us transitively get the right dependencies. For example, even though grpc-protobuf-lite has protobuf-lite excluded, and the Maven Enforcer banned-dependencies check passes... if a user happens to get a transitive dependency on grpc-all first, they may pull in grpc-protobuf from that other source without the exclusion. Thus we need to exclude protobuf-lite from grpc-all as well. While we're here, also add guava-jdk5 to the set of banned dependencies, though (as above) we cannot currently properly identify the places it might be transitively exposed in a users' pom.xml. commit 2b0e699b8f6a0aac26ac57c536e8d45cee1c3d2e Author: Dan Halperin <dhalp...@google.com> Date: 2017-05-08T18:35:56Z This closes #2934 commit 5bac40efcc7bcd9d53183579ec6089c23aeda782 Author: Dan Halperin <dhalp...@google.com> Date: 2017-05-06T02:06:03Z [BEAM-2212] FileBasedSource: improve message when logging. ValueProvider should not be printed, rather the string instead. commit 9423babd8f827e843723c218441e9a91aaa7b361 Author: Dan Halperin <dhalp...@google.com> Date: 2017-05-08T16:59:16Z [BEAM-2212] FileBasedSource: refactor to remove uses of fileOrPatternSpec.get() Makes it less likely to have errors from printing ValueProviders instead of runtime values commit e1791c3f83a007013625b43f9976e7ffa64e8a49 Author: Dan Halperin <dhalp...@google.com> Date: 2017-05-08T18:48:42Z This closes #2956 commit 13e51c9cd114eec73c47b71f46214dd92ee81048 Author: Dan Halperin <dhalp...@google.com> Date: 2017-05-06T02:49:29Z Convert Executor Services to use Daemon Threads This will cause the DirectRunner to automatically shut down when the worker threads are shut down. commit 63c6bea3bbdb52964ac54073326d4ad99ba7432c Author: Thomas Groh <tg...@google.com> Date: 2017-05-08T19:52:09Z This closes #2954 commit 50a4c56de1d3aa0e51e8b2b08b71715be4c2ef80 Author: Sourabh Bajaj <sourabhba...@google.com> Date: 2017-05-07T20:55:20Z [BEAM-2206] Move pipelineOptions into options modules commit 604c2f59e2a928a95106814b0b55b8097dc39faa Author: Ahmet Altay <al...@google.com> Date: 2017-05-08T19:56:15Z This closes #2925 commit d3fc7d18a4a3c394d32d85ef34c08a85712a8b18 Author: Reuven Lax <re...@google.com> Date: 2017-05-08T16:06:55Z TableDescription is allowed to be null. commit c481396edc7a3e906cd79dcc2a356baf9c93895e Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-05-08T20:41:01Z This closes #2953 commit 1fd15bd7441e1be72feb3f9535ffe536d730a531 Author: Reuven Lax <re...@google.com> Date: 2017-05-07T18:50:08Z Shortens PubsubIO names for reading/writing messages. commit d1914ed0c3195e1de2f707e2bb7bc57597b834fe Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-05-08T20:45:25Z This closes #2950 commit b7635aadc2532b5c3bdc38f070865091ebf901c1 Author: Ahmet Altay <al...@google.com> Date: 2017-05-08T20:42:02Z Remove apex README.md. This information is already in the website in quickstart and apex runner pages. https://github.com/apache/beam-site/pull/232 moves small bits of missing content. commit 962e8796d68f9a81b0ae3d1b30d7a4bcadbdaf60 Author: Ahmet Altay <al...@google.com> Date: 2017-05-08T21:28:36Z This closes #2962 commit fba4c010ce90db8b5cdac489d47126ca3ee8db65 Author: Ahmet Altay <al...@google.com> Date: 2017-05-08T22:00:33Z Remove readme.md files that are covered in web site commit 152c5bcc7f6fb53565ed132653384f4244542b59 Author: Ahmet Altay <al...@google.com> Date: 2017-05-08T22:04:46Z This closes #2966 commit 78c29238aa471138fb55681d8700b095a3606e0d Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-05-08T21:54:17Z [BEAM-2224] Fix temp file management on Windows. commit f315cc9ff649ccee9c9aff18f4af30cfbd58e5e5 Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-05-08T23:49:05Z Closes #2965 commit fd1dda09fc4db48cfbc21afd6310320533fffe78 Author: Reuven Lax <re...@google.com> Date: 2017-04-29T14:33:54Z [BEAM-2154] More scalable dynamic BigQueryIO.Write If too many tables are generated in a bundle, spill and group the results before writing files. Generating hundreds or thousands of file write buffers in a single bundle was causing workers to crash with out of memory. commit 6a2586a4ee279f66fd4eae816e93fa8e397acf6b Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-05-09T01:02:14Z This closes #2883 commit 367fcb28d544934797d25cb34d54136b2d7d6e99 Author: Vikas Kedigehalli <vika...@google.com> Date: 2017-05-06T02:24:51Z Remove google api BackOff usage from sdks/core commit 87578a6a69f15bca502d9b4c15999aa383e539a5 Author: Vikas Kedigehalli <vika...@google.com> Date: 2017-05-06T02:33:45Z Use guava Base64 encoding instead of google api client commit 88e044fda0a7d0ea1372f075a4c7a4bd54af47da Author: Luke Cwik <lc...@google.com> Date: 2017-05-09T02:20:01Z [BEAM-1871] Remove Google API client dependency from sdks/core This closes #2937 commit 16f355f7e481ebf029c0edb878742f6fea57b6cd Author: Dan Halperin <dhalp...@google.com> Date: 2017-05-08T22:37:29Z [BEAM-2211] DataflowRunner: remove validation of file read/write paths Now that users can implement and register custom FileSystems, we can no longer really effectively validate filesystems they can read or write files from. They can even register file:// to point to some HDFS path, e.g., commit 23731fe7ab12ea5d3ebb7b3f16c788b83d0caf0b Author: Dan Halperin <dhalp...@google.com> Date: 2017-05-09T03:13:38Z This closes #2968 commit 27e9a060ed593b2b53b88481591f14b1a274c61b Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-05-05T23:20:37Z Remove explicit used of nested contexts. find . -type f -name '*.java' | xargs sed -i '' 's/\([.]..code[(].*\), *context.nested..[)]/\1)/' find . -type f -name '*.java' | xargs sed -i '' 's/\([.]..code[(].*\), *nestedContext[)]/\1)/' find . -type f -name '*.java' | xargs sed -i '' 's/\([.]..code[(].*\), *Context.NESTED[)]/\1)/' find . -type f -name '*.java' | xargs sed -i '' 's/\([.]..code[(].*\), *[^ ]*.Context.NESTED[)]/\1)/' Added back explicit context in CoGbkResult.java due to compile error. commit 996dce37b76b103f104328b7caa65f73a1bcb15a Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-05-05T23:36:47Z Remove contexts from coders where they'll never be used. commit b7f3341ed4fc073a834a55773bcb4a2c7f821c52 Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-05-06T00:24:02Z automated context removal or redirection commit 45e09b2df7a97cfbbe0d8013c16cfdd7a55afbde Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-05-06T00:27:13Z get it compiling ---- > Gearpump runner > --------------- > > Key: BEAM-79 > URL: https://issues.apache.org/jira/browse/BEAM-79 > Project: Beam > Issue Type: New Feature > Components: runner-gearpump > Reporter: Tyler Akidau > Assignee: Manu Zhang > > Intel is submitting Gearpump (http://www.gearpump.io) to ASF > (https://wiki.apache.org/incubator/GearpumpProposal). Appears to be a mix of > low-level primitives a la MillWheel, with some higher level primitives like > non-merging windowing mixed in. Seems like it would make a nice Beam runner. -- This message was sent by Atlassian JIRA (v6.3.15#6346)