[beam] 01/01: Merge pull request #5535: Use Docker in the ReferenceRunner
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 4511734b9cae49eca785b051e5a57c5bacd87952 Merge: e33da0d d6fc2f2 Author: Thomas Groh AuthorDate: Fri Jun 1 18:40:06 2018 -0700 Merge pull request #5535: Use Docker in the ReferenceRunner [BEAM-2899][BEAM-2885] .../runners/core/construction/Environments.java| 5 +- .../runners/direct/portable/QuiescenceDriver.java | 1 + .../runners/direct/portable/ReferenceRunner.java | 100 -- .../portable/RemoteStageEvaluatorFactory.java | 2 +- .../portable/TransformEvaluatorRegistry.java | 1 - .../UnsupportedArtifactRetrievalService.java | 43 ++ .../portable/job/ReferenceRunnerJobService.java| 48 +-- .../direct/portable/ReferenceRunnerTest.java | 12 +- .../portable/RemoteStageEvaluatorFactoryTest.java | 4 +- .../UnsupportedArtifactRetrievalServiceTest.java | 153 + ...SingleEnvironmentInstanceJobBundleFactory.java} | 31 +++-- .../fnexecution/control/RemoteExecutionTest.java | 111 --- ...leEnvironmentInstanceJobBundleFactoryTest.java} | 12 +- .../src/main/resources/beam/findbugs-filter.xml| 7 + 14 files changed, 426 insertions(+), 104 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (e33da0d -> 4511734)
This is an automated email from the ASF dual-hosted git repository.
tgroh pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.
from e33da0d Merge pull request #5530: [SQL] BeamSqlEnv refactor
add 6b4eed8 Use the Current Users Container as the Environment
add 4949cc9 Enable Docker in the PortableDirectRunner
add 76fb544 Move DirectJobBundleFactory to fn-execution
add d0a02f0 Cleanups to the PortableDirectRunner
add f7476a8 Cleanups and Wiring for the ReferenceRunnerJobService
add d6fc2f2 Use an UnsupportedArtifactRetrievalService
new 4511734 Merge pull request #5535: Use Docker in the ReferenceRunner
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../runners/core/construction/Environments.java| 5 +-
.../runners/direct/portable/QuiescenceDriver.java | 1 +
.../runners/direct/portable/ReferenceRunner.java | 100 --
.../portable/RemoteStageEvaluatorFactory.java | 2 +-
.../portable/TransformEvaluatorRegistry.java | 1 -
...va => UnsupportedArtifactRetrievalService.java} | 25 +++-
.../portable/job/ReferenceRunnerJobService.java| 48 +--
.../direct/portable/ReferenceRunnerTest.java | 12 +-
.../portable/RemoteStageEvaluatorFactoryTest.java | 4 +-
.../UnsupportedArtifactRetrievalServiceTest.java | 153 +
...SingleEnvironmentInstanceJobBundleFactory.java} | 31 +++--
.../fnexecution/control/RemoteExecutionTest.java | 111 ---
...leEnvironmentInstanceJobBundleFactoryTest.java} | 12 +-
.../src/main/resources/beam/findbugs-filter.xml| 7 +
14 files changed, 407 insertions(+), 105 deletions(-)
copy
runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/artifact/{package-info.java
=> UnsupportedArtifactRetrievalService.java} (56%)
create mode 100644
runners/direct-java/src/test/java/org/apache/beam/runners/direct/portable/artifact/UnsupportedArtifactRetrievalServiceTest.java
rename
runners/{direct-java/src/main/java/org/apache/beam/runners/direct/portable/DirectJobBundleFactory.java
=>
java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/control/SingleEnvironmentInstanceJobBundleFactory.java}
(86%)
rename
runners/{direct-java/src/test/java/org/apache/beam/runners/direct/portable/DirectJobBundleFactoryTest.java
=>
java-fn-execution/src/test/java/org/apache/beam/runners/fnexecution/control/SingleEnvironmentInstanceJobBundleFactoryTest.java}
(95%)
--
To stop receiving notification emails like this one, please contact
tg...@apache.org.
[beam] 01/01: Merge pull request #5468: Add a test to show Flatten Execution
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit ab01229848487570ed7fb0b7418db9e3b92a733d Merge: 1414dd5 4a347e1 Author: Thomas Groh AuthorDate: Thu May 31 11:07:34 2018 -0700 Merge pull request #5468: Add a test to show Flatten Execution .../portable/RemoteStageEvaluatorFactoryTest.java | 65 ++ 1 file changed, 65 insertions(+) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (1414dd5 -> ab01229)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 1414dd5 Merge pull request #4748: Avoid fully qualified class names, when there is an existing (static)… add 4a347e1 Add a test to show Flatten Execution new ab01229 Merge pull request #5468: Add a test to show Flatten Execution The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../portable/RemoteStageEvaluatorFactoryTest.java | 65 ++ 1 file changed, 65 insertions(+) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (0e6166e -> 856d842)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 0e6166e enable failOnWarning for build-tools add f5b3555 Add convenience aliases for nullary type constructors add 34cfbf6 Add MAP field support to Schema.Builder add 30268a5 Add static factories for array, row, map types add c6055fe Schema fields are non-null by default add cf15487 TestUtils.RowsBuilder to use FieldType instead of TypeName add adf4edc Clean up CalciteUtils conversions a bit add 856d842 Merge pull request #5490: Refactor schemas and fields, simplify a bit No new revisions were added by this update. Summary of changes: .../java/org/apache/beam/sdk/schemas/Schema.java | 131 - .../org/apache/beam/sdk/coders/RowCoderTest.java | 36 +++--- .../org/apache/beam/sdk/schemas/SchemaTest.java| 20 ++-- .../apache/beam/sdk/transforms/JsonToRowTest.java | 8 +- .../beam/sdk/util/RowJsonDeserializerTest.java | 55 - .../java/org/apache/beam/sdk/values/RowTest.java | 20 ++-- .../sdk/values/reflect/InferredRowCoderTest.java | 6 +- .../beam/sdk/extensions/sql/RowSqlTypes.java | 2 +- .../extensions/sql/impl/utils/CalciteUtils.java| 89 +++--- .../extensions/sql/InferredRowCoderSqlTest.java| 18 +-- .../beam/sdk/extensions/sql/JsonToRowSqlTest.java | 9 +- .../apache/beam/sdk/extensions/sql/TestUtils.java | 4 +- .../sdk/extensions/sql/impl/JdbcDriverTest.java| 6 +- .../sql/impl/rel/BeamEnumerableConverterTest.java | 7 +- .../sql/impl/rel/BeamIntersectRelTest.java | 26 ++-- .../impl/rel/BeamJoinRelBoundedVsBoundedTest.java | 62 +- .../rel/BeamJoinRelUnboundedVsBoundedTest.java | 45 --- .../rel/BeamJoinRelUnboundedVsUnboundedTest.java | 42 +++ .../extensions/sql/impl/rel/BeamMinusRelTest.java | 26 ++-- .../sql/impl/rel/BeamSetOperatorRelBaseTest.java | 16 +-- .../extensions/sql/impl/rel/BeamSortRelTest.java | 78 ++-- .../extensions/sql/impl/rel/BeamUnionRelTest.java | 20 ++-- .../extensions/sql/impl/rel/BeamValuesRelTest.java | 22 ++-- .../sdk/io/gcp/bigquery/BigQueryUtilsTest.java | 22 ++-- 24 files changed, 408 insertions(+), 362 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5477: Add a Portable TransformEvaluatorRegistry
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 707fb29ffd9f6aaad8b736792ff1b476f20de28b Merge: d2394ca 80f8a23 Author: Thomas Groh AuthorDate: Fri May 25 18:17:45 2018 -0700 Merge pull request #5477: Add a Portable TransformEvaluatorRegistry .../direct/portable/FlattenEvaluatorFactory.java | 2 +- .../GroupAlsoByWindowEvaluatorFactory.java | 2 +- .../portable/GroupByKeyOnlyEvaluatorFactory.java | 4 +-- .../direct/portable/ImpulseEvaluatorFactory.java | 2 +- .../portable/TransformEvaluatorRegistry.java | 33 ++ .../portable/FlattenEvaluatorFactoryTest.java | 2 +- .../GroupByKeyOnlyEvaluatorFactoryTest.java| 2 +- .../portable/ImpulseEvaluatorFactoryTest.java | 2 +- 8 files changed, 41 insertions(+), 8 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (d2394ca -> 707fb29)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from d2394ca Merge pull request #5452: [BEAM-4371] Namespace shares the same name between Jenkins Performance test jobs add f571cd5 Add a Portable TransformEvaluatorRegistry add 80f8a23 Consistent Parameter Order in Evaluator Factories new 707fb29 Merge pull request #5477: Add a Portable TransformEvaluatorRegistry The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../direct/portable/FlattenEvaluatorFactory.java | 2 +- .../GroupAlsoByWindowEvaluatorFactory.java | 2 +- .../portable/GroupByKeyOnlyEvaluatorFactory.java | 4 +-- .../direct/portable/ImpulseEvaluatorFactory.java | 2 +- .../portable/TransformEvaluatorRegistry.java | 33 ++ .../portable/FlattenEvaluatorFactoryTest.java | 2 +- .../GroupByKeyOnlyEvaluatorFactoryTest.java| 2 +- .../portable/ImpulseEvaluatorFactoryTest.java | 2 +- 8 files changed, 41 insertions(+), 8 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (3173f01 -> b0b7e3b)
This is an automated email from the ASF dual-hosted git repository.
tgroh pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.
from 3173f01 Merge pull request #5469: Update Docker Command
add b5deb31 Move the Local Artifact Services to the DirectRunner
add 01f7ce1 Migrate the ReferenceRunnerJobServer to runners-direct
add 316caeb Fix License Headers
new b0b7e3b Merge pull request #5370: Move the ReferenceRunner Job
Service, Artifact Services to the DirectRunner module
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
runners/direct-java/build.gradle | 1 +
.../artifact}/LocalArtifactStagingLocation.java| 4 +-
.../LocalFileSystemArtifactRetrievalService.java | 2 +-
.../LocalFileSystemArtifactStagerService.java | 14 +-
.../direct/portable/artifact}/package-info.java| 6 +-
.../runners/direct/portable}/job/PreparingJob.java | 7 +-
.../portable}/job/ReferenceRunnerJobServer.java| 39 +++---
.../portable}/job/ReferenceRunnerJobService.java | 13 +-
.../runners/direct/portable}/job/package-info.java | 2 +-
.../runners/direct/DirectRunnerApiSurfaceTest.java | 4 +-
.../LocalArtifactStagingLocationTest.java | 2 +-
...ocalFileSystemArtifactRetrievalServiceTest.java | 4 +-
.../LocalFileSystemArtifactStagerServiceTest.java | 4 +-
.../job/ReferenceRunnerJobServiceTest.java | 3 +-
runners/local-artifact-service-java/build.gradle | 44 ---
runners/local-artifact-service-java/pom.xml| 137 ---
.../apache/beam/artifact/local/package-info.java | 22
runners/reference/job-server/build.gradle | 38 --
runners/reference/job-server/pom.xml | 145 -
runners/reference/pom.xml | 1 -
sdks/java/javadoc/build.gradle | 2 -
settings.gradle| 2 -
22 files changed, 58 insertions(+), 438 deletions(-)
rename
runners/{local-artifact-service-java/src/main/java/org/apache/beam/artifact/local
=>
direct-java/src/main/java/org/apache/beam/runners/direct/portable/artifact}/LocalArtifactStagingLocation.java
(97%)
rename
runners/{local-artifact-service-java/src/main/java/org/apache/beam/artifact/local
=>
direct-java/src/main/java/org/apache/beam/runners/direct/portable/artifact}/LocalFileSystemArtifactRetrievalService.java
(98%)
rename
runners/{local-artifact-service-java/src/main/java/org/apache/beam/artifact/local
=>
direct-java/src/main/java/org/apache/beam/runners/direct/portable/artifact}/LocalFileSystemArtifactStagerService.java
(95%)
copy
runners/{apex/src/main/java/org/apache/beam/runners/apex/translation/utils =>
direct-java/src/main/java/org/apache/beam/runners/direct/portable/artifact}/package-info.java
(86%)
rename
runners/{reference/job-server/src/main/java/org/apache/beam/runners/reference
=>
direct-java/src/main/java/org/apache/beam/runners/direct/portable}/job/PreparingJob.java
(92%)
rename
runners/{reference/job-server/src/main/java/org/apache/beam/runners/reference
=>
direct-java/src/main/java/org/apache/beam/runners/direct/portable}/job/ReferenceRunnerJobServer.java
(72%)
rename
runners/{reference/job-server/src/main/java/org/apache/beam/runners/reference
=>
direct-java/src/main/java/org/apache/beam/runners/direct/portable}/job/ReferenceRunnerJobService.java
(92%)
rename
runners/{reference/job-server/src/main/java/org/apache/beam/runners/reference
=>
direct-java/src/main/java/org/apache/beam/runners/direct/portable}/job/package-info.java
(94%)
rename
runners/{local-artifact-service-java/src/test/java/org/apache/beam/artifact/local
=>
direct-java/src/test/java/org/apache/beam/runners/direct/portable/artifact}/LocalArtifactStagingLocationTest.java
(98%)
rename
runners/{local-artifact-service-java/src/test/java/org/apache/beam/artifact/local
=>
direct-java/src/test/java/org/apache/beam/runners/direct/portable/artifact}/LocalFileSystemArtifactRetrievalServiceTest.java
(98%)
rename
runners/{local-artifact-service-java/src/test/java/org/apache/beam/artifact/local
=>
direct-java/src/test/java/org/apache/beam/runners/direct/portable/artifact}/LocalFileSystemArtifactStagerServiceTest.java
(98%)
rename
runners/{reference/job-server/src/test/java/org/apache/beam/runners/reference
=>
direct-java/src/test/java/org/apache/beam/runners/direct/portable}/job/ReferenceRunnerJobServiceTest.java
(99%)
delete mode 100644 runners/local-artifact-service-java/build.gradle
delete mode 100644 runners/local-artifact-service-java/pom.xml
delete mode 100644
runners/local-artifact-service-java/src/ma
[beam] 01/01: Merge pull request #5370: Move the ReferenceRunner Job Service, Artifact Services to the DirectRunner module
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit b0b7e3bf4941f874d360923ffd1d03e38befc589 Merge: 3173f01 316caeb Author: Thomas Groh AuthorDate: Thu May 24 16:20:08 2018 -0700 Merge pull request #5370: Move the ReferenceRunner Job Service, Artifact Services to the DirectRunner module [BEAM-2885] runners/direct-java/build.gradle | 1 + .../artifact}/LocalArtifactStagingLocation.java| 4 +- .../LocalFileSystemArtifactRetrievalService.java | 2 +- .../LocalFileSystemArtifactStagerService.java | 14 +- .../direct/portable/artifact}/package-info.java| 6 +- .../runners/direct/portable}/job/PreparingJob.java | 7 +- .../portable}/job/ReferenceRunnerJobServer.java| 39 +++--- .../portable}/job/ReferenceRunnerJobService.java | 13 +- .../runners/direct/portable}/job/package-info.java | 2 +- .../runners/direct/DirectRunnerApiSurfaceTest.java | 4 +- .../LocalArtifactStagingLocationTest.java | 2 +- ...ocalFileSystemArtifactRetrievalServiceTest.java | 4 +- .../LocalFileSystemArtifactStagerServiceTest.java | 4 +- .../job/ReferenceRunnerJobServiceTest.java | 3 +- runners/local-artifact-service-java/build.gradle | 44 --- runners/local-artifact-service-java/pom.xml| 137 --- runners/reference/job-server/build.gradle | 38 -- runners/reference/job-server/pom.xml | 145 - runners/reference/pom.xml | 1 - sdks/java/javadoc/build.gradle | 2 - settings.gradle| 2 - 21 files changed, 58 insertions(+), 416 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5460: Update GroupAlsoByWindowEvaluatorFactory
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 542d8828138ee6d97556181228b64ad2b1a364ff Merge: 3ec7b09 a25fc8d Author: Thomas Groh AuthorDate: Thu May 24 14:38:13 2018 -0700 Merge pull request #5460: Update GroupAlsoByWindowEvaluatorFactory .../GroupAlsoByWindowEvaluatorFactory.java | 149 - .../direct/portable/StateAndTimerProvider.java | 32 + 2 files changed, 120 insertions(+), 61 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (3ec7b09 -> 542d882)
This is an automated email from the ASF dual-hosted git repository.
tgroh pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.
from 3ec7b09 [BEAM-4392] Fix :beam-runners-java-fn-execution failure
add a25fc8d Update GroupAlsoByWindowEvaluatorFactory
new 542d882 Merge pull request #5460: Update
GroupAlsoByWindowEvaluatorFactory
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../GroupAlsoByWindowEvaluatorFactory.java | 149 -
...utorFactory.java => StateAndTimerProvider.java} | 18 +--
2 files changed, 97 insertions(+), 70 deletions(-)
copy
runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/{TransformExecutorFactory.java
=> StateAndTimerProvider.java} (66%)
--
To stop receiving notification emails like this one, please contact
tg...@apache.org.
[beam] branch master updated (3156b60 -> e0cf774)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 3156b60 Merge pull request #5461: [BEAM-4393] Appending lacked LICENSE of shaded dependencies add e54f627 Add an InProcessEnvironmentFactory add e869448 Update InProcessManagedChannelFactory add 9385d32 Add a RemoteStageEvaluatorFactory add e515e22 Move InProcessEnvironmentFactory to java-fn-execution new e0cf774 Merge pull request #5349: Add a Remote Stage Evaluator The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: runners/direct-java/build.gradle | 1 + runners/direct-java/pom.xml| 6 + .../portable/RemoteStageEvaluatorFactory.java | 89 +++ .../portable/RemoteStageEvaluatorFactoryTest.java | 172 + runners/java-fn-execution/build.gradle | 1 + runners/java-fn-execution/pom.xml | 12 +- .../environment/InProcessEnvironmentFactory.java | 123 +++ .../runners/fnexecution/InProcessSdkHarness.java | 50 +++--- .../fnexecution/control/RemoteExecutionTest.java | 2 +- .../fnexecution/control/SdkHarnessClientTest.java | 4 +- .../beam/runners/reference/PortableRunnerTest.java | 4 +- .../fn/test/InProcessManagedChannelFactory.java| 5 + 12 files changed, 427 insertions(+), 42 deletions(-) create mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/RemoteStageEvaluatorFactory.java create mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/portable/RemoteStageEvaluatorFactoryTest.java create mode 100644 runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/environment/InProcessEnvironmentFactory.java -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5349: Add a Remote Stage Evaluator
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit e0cf774ac150300d92c0c62b9c5e070eb210419f Merge: 3156b60 e515e22 Author: Thomas Groh AuthorDate: Thu May 24 09:56:56 2018 -0700 Merge pull request #5349: Add a Remote Stage Evaluator [BEAM-3326] runners/direct-java/build.gradle | 1 + runners/direct-java/pom.xml| 6 + .../portable/RemoteStageEvaluatorFactory.java | 89 +++ .../portable/RemoteStageEvaluatorFactoryTest.java | 172 + runners/java-fn-execution/build.gradle | 1 + runners/java-fn-execution/pom.xml | 12 +- .../environment/InProcessEnvironmentFactory.java | 123 +++ .../runners/fnexecution/InProcessSdkHarness.java | 50 +++--- .../fnexecution/control/RemoteExecutionTest.java | 2 +- .../fnexecution/control/SdkHarnessClientTest.java | 4 +- .../beam/runners/reference/PortableRunnerTest.java | 4 +- .../fn/test/InProcessManagedChannelFactory.java| 5 + 12 files changed, 427 insertions(+), 42 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5459: Use WindowOnlyContext in ReduceFnContextFactory
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit e8ca1e17851f36f955f2dd497db935f2c14b1114 Merge: e31f27f 773583d Author: Thomas Groh AuthorDate: Wed May 23 16:36:03 2018 -0700 Merge pull request #5459: Use WindowOnlyContext in ReduceFnContextFactory .../main/java/org/apache/beam/runners/core/ReduceFnContextFactory.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (e31f27f -> e8ca1e1)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from e31f27f Merge pull request #5422 from timrobertson100/BEAM-4347 add 773583d Use WindowOnlyContext in ReduceFnContextFactory new e8ca1e1 Merge pull request #5459: Use WindowOnlyContext in ReduceFnContextFactory The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../main/java/org/apache/beam/runners/core/ReduceFnContextFactory.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (1739e8d -> 8baf51d)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 1739e8d Merge pull request #5298: Add a Default RemoteEnvironment Implementation add e20d81f Add a Window-only State Context new 8baf51d Merge pull request #5427: Add a Window-only State Context The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../org/apache/beam/sdk/state/StateContexts.java | 28 +++ .../apache/beam/sdk/state/StateContextsTest.java | 95 ++ 2 files changed, 123 insertions(+) create mode 100644 sdks/java/core/src/test/avro/org/apache/beam/sdk/state/StateContextsTest.java -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5427: Add a Window-only State Context
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 8baf51da1eb7b72c27200d9b0e27a60e7dde23da Merge: 1739e8d e20d81f Author: Thomas Groh AuthorDate: Tue May 22 10:58:15 2018 -0700 Merge pull request #5427: Add a Window-only State Context .../org/apache/beam/sdk/state/StateContexts.java | 28 +++ .../apache/beam/sdk/state/StateContextsTest.java | 95 ++ 2 files changed, 123 insertions(+) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (4670344 -> 1739e8d)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 4670344 Merge pull request #5413: [BEAM-4341] Enforce ErrorProne analysis in google-cloud-platform IO add 0afd25b Add a Default RemoteEnvironment Implementation new 1739e8d Merge pull request #5298: Add a Default RemoteEnvironment Implementation The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../fnexecution/environment/RemoteEnvironment.java | 32 ++ .../environment/RemoteEnvironmentTest.java | 71 ++ 2 files changed, 103 insertions(+) create mode 100644 runners/java-fn-execution/src/test/java/org/apache/beam/runners/fnexecution/environment/RemoteEnvironmentTest.java -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5298: Add a Default RemoteEnvironment Implementation
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 1739e8ddc85de6ff1ab4f42785b89f00cca0fbcf Merge: 4670344 0afd25b Author: Thomas Groh AuthorDate: Tue May 22 10:57:50 2018 -0700 Merge pull request #5298: Add a Default RemoteEnvironment Implementation .../fnexecution/environment/RemoteEnvironment.java | 32 ++ .../environment/RemoteEnvironmentTest.java | 71 ++ 2 files changed, 103 insertions(+) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (5051030 -> 60e498b)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 5051030 Merge pull request #5444: Revert BEAM-3256 add archetype testing to gradle build PreCommits add f19a3bf Add a BundleFactoryOutputReceiverFactory add 7a85153 Add a DirectJobBundleFactory add 468e8e7 Fix Copyright Headers in sdks/java/fn-execution new 60e498b Merge pull request #5348: Add a Direct Job Bundle Factory The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: runners/direct-java/build.gradle | 1 + runners/direct-java/pom.xml| 5 + .../BundleFactoryOutputRecieverFactory.java| 69 ++ .../direct/portable/DirectJobBundleFactory.java| 178 ++ .../BundleFactoryOutputRecieverFactoryTest.java| 259 + .../portable/DirectJobBundleFactoryTest.java | 178 ++ .../java/org/apache/beam/sdk/fn/IdGenerator.java | 31 ++- .../java/org/apache/beam/sdk/fn/package-info.java | 30 ++- .../org/apache/beam/sdk/fn/IdGeneratorsTest.java | 30 ++- 9 files changed, 729 insertions(+), 52 deletions(-) create mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/BundleFactoryOutputRecieverFactory.java create mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/DirectJobBundleFactory.java create mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/portable/BundleFactoryOutputRecieverFactoryTest.java create mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/portable/DirectJobBundleFactoryTest.java -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5348: Add a Direct Job Bundle Factory
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 60e498bfa7e21986cf32407995229b41680cdf32 Merge: 5051030 468e8e7 Author: Thomas Groh AuthorDate: Tue May 22 09:49:51 2018 -0700 Merge pull request #5348: Add a Direct Job Bundle Factory [BEAM-3326] runners/direct-java/build.gradle | 1 + runners/direct-java/pom.xml| 5 + .../BundleFactoryOutputRecieverFactory.java| 69 ++ .../direct/portable/DirectJobBundleFactory.java| 178 ++ .../BundleFactoryOutputRecieverFactoryTest.java| 259 + .../portable/DirectJobBundleFactoryTest.java | 178 ++ .../java/org/apache/beam/sdk/fn/IdGenerator.java | 31 ++- .../java/org/apache/beam/sdk/fn/package-info.java | 30 ++- .../org/apache/beam/sdk/fn/IdGeneratorsTest.java | 30 ++- 9 files changed, 729 insertions(+), 52 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (67f6cf2 -> 5051030)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 67f6cf2 Merge pull request #5440: [BEAM-4375] Replaces usage of JUnit runners that do not work well with Gradle add 3f89214 rollback BEAM-3256 add archetype testing to gradle build PreCommits new 5051030 Merge pull request #5444: Revert BEAM-3256 add archetype testing to gradle build PreCommits The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .test-infra/jenkins/job_PreCommit_Java_GradleBuild.groovy | 2 -- build.gradle | 2 -- sdks/java/maven-archetypes/examples/build.gradle | 9 - sdks/java/maven-archetypes/starter/build.gradle | 9 - 4 files changed, 22 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5444: Revert BEAM-3256 add archetype testing to gradle build PreCommits
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 505103010e8a8e7a8211827aac14803e953b3587 Merge: 67f6cf2 3f89214 Author: Thomas Groh AuthorDate: Tue May 22 09:45:18 2018 -0700 Merge pull request #5444: Revert BEAM-3256 add archetype testing to gradle build PreCommits .test-infra/jenkins/job_PreCommit_Java_GradleBuild.groovy | 2 -- build.gradle | 2 -- sdks/java/maven-archetypes/examples/build.gradle | 9 - sdks/java/maven-archetypes/starter/build.gradle | 9 - 4 files changed, 22 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (30d8176 -> c21ce55)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 30d8176 Merge pull request #5386: Add a new DockerEnvironmentFactory Constructor add 5fe2eb5 Update worker_id Documentation new c21ce55 Merge pull request #5426: Update worker_id Documentation The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: model/fn-execution/src/main/proto/beam_provision_api.proto | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5426: Update worker_id Documentation
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit c21ce55cd3ca6983edc660e84e424db253bb0c33 Merge: 30d8176 5fe2eb5 Author: Thomas Groh AuthorDate: Fri May 18 12:25:25 2018 -0700 Merge pull request #5426: Update worker_id Documentation model/fn-execution/src/main/proto/beam_provision_api.proto | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (fc7ddf2 -> 30d8176)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from fc7ddf2 Merge pull request #4793: Stop adding holds behind the watermark for merging late windows add 386f52c Add a new DockerEnvironmentFactory Constructor new 30d8176 Merge pull request #5386: Add a new DockerEnvironmentFactory Constructor The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../fnexecution/environment/DockerCommand.java | 6 .../environment/DockerEnvironmentFactory.java | 32 ++ .../fnexecution/environment/DockerCommandTest.java | 10 ++- .../environment/DockerEnvironmentFactoryTest.java | 10 +++ 4 files changed, 39 insertions(+), 19 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5386: Add a new DockerEnvironmentFactory Constructor
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 30d8176a4190c6401cfbc2b43e8c9d930f99ee99 Merge: fc7ddf2 386f52c Author: Thomas Groh AuthorDate: Fri May 18 10:37:50 2018 -0700 Merge pull request #5386: Add a new DockerEnvironmentFactory Constructor [BEAM-4258] .../fnexecution/environment/DockerCommand.java | 6 .../environment/DockerEnvironmentFactory.java | 32 ++ .../fnexecution/environment/DockerCommandTest.java | 10 ++- .../environment/DockerEnvironmentFactoryTest.java | 10 +++ 4 files changed, 39 insertions(+), 19 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #4793: Stop adding holds behind the watermark for merging late windows
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit fc7ddf2d828e4bb3709f4d0dfa81e92806f02966 Merge: c3a80b2 9452aca Author: Thomas Groh AuthorDate: Fri May 18 10:35:46 2018 -0700 Merge pull request #4793: Stop adding holds behind the watermark for merging late windows [BEAM-3776] Fix issue with merging late windows where a watermark hold could be added behind the input watermark. pom.xml| 7 + runners/core-java/build.gradle | 2 +- runners/core-java/pom.xml | 7 +- .../org/apache/beam/runners/core/StateMerging.java | 95 - .../apache/beam/runners/core/WatermarkHold.java| 74 ++- .../beam/runners/core/ReduceFnRunnerTest.java | 222 - .../apache/beam/runners/core/ReduceFnTester.java | 8 +- .../beam/runners/core/StateInternalsTest.java | 42 .../FlinkBroadcastStateInternalsTest.java | 8 - .../streaming/FlinkKeyGroupStateInternalsTest.java | 8 - .../streaming/FlinkSplitStateInternalsTest.java| 8 - .../test/java/org/apache/beam/sdk/io/ReadTest.java | 5 + 12 files changed, 313 insertions(+), 173 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (c3a80b2 -> fc7ddf2)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from c3a80b2 Merge pull request #5424: [BEAM-4352] Enforce ErrorProne analysis in Redis IO add 9452aca [BEAM-3776] Fix issue with merging late windows where a watermark hold could be added behind the input watermark. Add additional tests exercising merging windows and lateness. In particular, testMergingLateWatermarkHolds previously failed due to the issue. new fc7ddf2 Merge pull request #4793: Stop adding holds behind the watermark for merging late windows The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: pom.xml| 7 + runners/core-java/build.gradle | 2 +- runners/core-java/pom.xml | 7 +- .../org/apache/beam/runners/core/StateMerging.java | 95 - .../apache/beam/runners/core/WatermarkHold.java| 74 ++- .../beam/runners/core/ReduceFnRunnerTest.java | 222 - .../apache/beam/runners/core/ReduceFnTester.java | 8 +- .../beam/runners/core/StateInternalsTest.java | 42 .../FlinkBroadcastStateInternalsTest.java | 8 - .../streaming/FlinkKeyGroupStateInternalsTest.java | 8 - .../streaming/FlinkSplitStateInternalsTest.java| 8 - .../test/java/org/apache/beam/sdk/io/ReadTest.java | 5 + 12 files changed, 313 insertions(+), 173 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (9ba58ee -> 7e42515)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 9ba58ee Merge pull request #5290: [BEAM-3983] Restore BigQuery SQL Support with copied enums add 4e50f02 Add an IdGenerator class to java/fn-execution new 7e42515 Merge pull request #5394: Add an IdGenerator class to java/fn-execution The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../fnexecution/control/SdkHarnessClient.java | 34 +++ .../java/org/apache/beam/sdk/fn/IdGenerator.java | 30 ++ .../java/org/apache/beam/sdk/fn/IdGenerators.java | 40 + .../java/org/apache/beam/sdk/fn/package-info.java | 23 .../org/apache/beam/sdk/fn/IdGeneratorsTest.java | 65 ++ .../java/org/apache/beam/fn/harness/FnHarness.java | 11 ++-- .../org/apache/beam/fn/harness/IdGenerator.java| 33 --- .../harness/state/BeamFnStateGrpcClientCache.java | 16 +++--- .../apache/beam/fn/harness/IdGeneratorTest.java| 40 - .../state/BeamFnStateGrpcClientCacheTest.java | 13 +++-- 10 files changed, 189 insertions(+), 116 deletions(-) create mode 100644 sdks/java/fn-execution/src/main/java/org/apache/beam/sdk/fn/IdGenerator.java create mode 100644 sdks/java/fn-execution/src/main/java/org/apache/beam/sdk/fn/IdGenerators.java create mode 100644 sdks/java/fn-execution/src/main/java/org/apache/beam/sdk/fn/package-info.java create mode 100644 sdks/java/fn-execution/src/test/java/org/apache/beam/sdk/fn/IdGeneratorsTest.java delete mode 100644 sdks/java/harness/src/main/java/org/apache/beam/fn/harness/IdGenerator.java delete mode 100644 sdks/java/harness/src/test/java/org/apache/beam/fn/harness/IdGeneratorTest.java -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5394: Add an IdGenerator class to java/fn-execution
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 7e42515b777f63b623739d807396c1b5a0f7ec18 Merge: 9ba58ee 4e50f02 Author: Thomas Groh AuthorDate: Thu May 17 09:03:40 2018 -0700 Merge pull request #5394: Add an IdGenerator class to java/fn-execution .../fnexecution/control/SdkHarnessClient.java | 34 +++ .../java/org/apache/beam/sdk/fn/IdGenerator.java | 30 ++ .../java/org/apache/beam/sdk/fn/IdGenerators.java | 40 + .../java/org/apache/beam/sdk/fn/package-info.java | 23 .../org/apache/beam/sdk/fn/IdGeneratorsTest.java | 65 ++ .../java/org/apache/beam/fn/harness/FnHarness.java | 11 ++-- .../org/apache/beam/fn/harness/IdGenerator.java| 33 --- .../harness/state/BeamFnStateGrpcClientCache.java | 16 +++--- .../apache/beam/fn/harness/IdGeneratorTest.java| 40 - .../state/BeamFnStateGrpcClientCacheTest.java | 13 +++-- 10 files changed, 189 insertions(+), 116 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (1e69890 -> f1c9490)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 1e69890 [BEAM-4236, BEAM-2927] Make Python SDK side inputs work with non well known coders and also work with Dataflow add 7ba93e5 [BEAM-4277] Adding unshaded jars used by Dataflow to poms. add f1c9490 Merge pull request #5340: Adding unshaded jars used by Dataflow to poms No new revisions were added by this update. Summary of changes: runners/core-construction-java/pom.xml | 17 + runners/direct-java/pom.xml| 17 + runners/local-java/pom.xml | 22 ++ sdks/java/extensions/protobuf/pom.xml | 18 ++ 4 files changed, 74 insertions(+) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (2927993 -> 05fa730)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 2927993 Merge pull request #5362 from timrobertson100/fix-elasticsearch-errorprone add b56321d Log when Handler Delegation Throws add 0c81cc9 squash! Revert "Log when Handler Delegation Throws" new 05fa730 Merge pull request #5377: Roll forward handler delegation The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../fn/harness/control/BeamFnControlClient.java| 85 ++ .../harness/control/BeamFnControlClientTest.java | 171 - 2 files changed, 191 insertions(+), 65 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5377: Roll forward handler delegation
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 05fa730267c0f3cd4bcd6e56e371fc7eb3d27131 Merge: 2927993 0c81cc9 Author: Thomas Groh AuthorDate: Wed May 16 09:50:54 2018 -0700 Merge pull request #5377: Roll forward handler delegation .../fn/harness/control/BeamFnControlClient.java| 85 ++ .../harness/control/BeamFnControlClientTest.java | 171 - 2 files changed, 191 insertions(+), 65 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5376: Revert "Log when Handler Delegation Throws"
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 72b6c8f6d0923da641b5344eb6a1fa8c260d596c Merge: 7f24583 6593686 Author: Thomas Groh AuthorDate: Tue May 15 15:41:59 2018 -0700 Merge pull request #5376: Revert "Log when Handler Delegation Throws" .../fn/harness/control/BeamFnControlClient.java| 85 -- .../harness/control/BeamFnControlClientTest.java | 171 + 2 files changed, 65 insertions(+), 191 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (7f24583 -> 72b6c8f)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 7f24583 Merge pull request #5330: Log when Handler Delegation Throws add 6593686 Revert "Log when Handler Delegation Throws" new 72b6c8f Merge pull request #5376: Revert "Log when Handler Delegation Throws" The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../fn/harness/control/BeamFnControlClient.java| 85 -- .../harness/control/BeamFnControlClientTest.java | 171 + 2 files changed, 65 insertions(+), 191 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (15b788d -> 7f24583)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 15b788d Merge pull request #5271: Migrate DirectRunner Evaluators to use Portable Graph Components add 77b4670 Log when Handler Delegation Throws new 7f24583 Merge pull request #5330: Log when Handler Delegation Throws The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../fn/harness/control/BeamFnControlClient.java| 85 ++ .../harness/control/BeamFnControlClientTest.java | 171 - 2 files changed, 191 insertions(+), 65 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5330: Log when Handler Delegation Throws
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 7f24583ba50a0b75b13c47c96889c897215f672e Merge: 15b788d 77b4670 Author: Thomas Groh AuthorDate: Tue May 15 14:37:32 2018 -0700 Merge pull request #5330: Log when Handler Delegation Throws .../fn/harness/control/BeamFnControlClient.java| 85 ++ .../harness/control/BeamFnControlClientTest.java | 171 - 2 files changed, 191 insertions(+), 65 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (eb979a0 -> 15b788d)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from eb979a0 Merge pull request #5347: [BEAM-4201][SQL] Add integration test for PubsubIO JSON -> SQL add 09f313a Migrate ImpulseEvaluatorFactory add a34f9b0 Migrate FlattenEvaluatorFactoryTest add be7a539 Migrate GroupByKeyOnly to the RR add 8007954 Use ModelCoders in LengthPrefixUnknownCoders new 15b788d Merge pull request #5271: Migrate DirectRunner Evaluators to use Portable Graph Components The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../runners/core/construction/ModelCoders.java | 9 +- runners/direct-java/build.gradle | 1 + runners/direct-java/pom.xml| 5 + .../direct/portable/FlattenEvaluatorFactory.java | 37 +++-- .../portable/GroupByKeyOnlyEvaluatorFactory.java | 110 - .../direct/portable/ImpulseEvaluatorFactory.java | 44 +++--- .../direct/portable/RootProviderRegistry.java | 6 +- .../portable/FlattenEvaluatorFactoryTest.java | 64 .../GroupByKeyOnlyEvaluatorFactoryTest.java| 170 + .../portable/ImpulseEvaluatorFactoryTest.java | 30 ++-- .../wire/LengthPrefixUnknownCoders.java| 18 +-- 11 files changed, 297 insertions(+), 197 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5271: Migrate DirectRunner Evaluators to use Portable Graph Components
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 15b788d52ac6717868023ef071714073b62aa62f Merge: eb979a0 8007954 Author: Thomas Groh AuthorDate: Tue May 15 13:51:19 2018 -0700 Merge pull request #5271: Migrate DirectRunner Evaluators to use Portable Graph Components [BEAM-4073] .../runners/core/construction/ModelCoders.java | 9 +- runners/direct-java/build.gradle | 1 + runners/direct-java/pom.xml| 5 + .../direct/portable/FlattenEvaluatorFactory.java | 37 +++-- .../portable/GroupByKeyOnlyEvaluatorFactory.java | 110 - .../direct/portable/ImpulseEvaluatorFactory.java | 44 +++--- .../direct/portable/RootProviderRegistry.java | 6 +- .../portable/FlattenEvaluatorFactoryTest.java | 64 .../GroupByKeyOnlyEvaluatorFactoryTest.java| 170 + .../portable/ImpulseEvaluatorFactoryTest.java | 30 ++-- .../wire/LengthPrefixUnknownCoders.java| 18 +-- 11 files changed, 297 insertions(+), 197 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch tgroh-patch-1 deleted (was dda97b8)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch tgroh-patch-1 in repository https://gitbox.apache.org/repos/asf/beam.git. was dda97b8 Use QueryablePipeline#forPipeline More This change permanently discards the following revisions: discard dda97b8 Use QueryablePipeline#forPipeline More -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5312: Add a Pipeline Constructor to QueryablePipeline
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 3a52bb108f3f2937c42d81d1e9b8b055c8047e1a Merge: 92ce97d bf52bc5 Author: Thomas Groh AuthorDate: Fri May 11 16:34:11 2018 -0700 Merge pull request #5312: Add a Pipeline Constructor to QueryablePipeline .../beam/runners/core/construction/graph/QueryablePipeline.java | 8 1 file changed, 8 insertions(+) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (92ce97d -> 3a52bb1)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 92ce97d Merge pull request #5346: Suppress more flaky tests. add bf52bc5 Add a Pipeline Constructor to QueryablePipeline new 3a52bb1 Merge pull request #5312: Add a Pipeline Constructor to QueryablePipeline The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../beam/runners/core/construction/graph/QueryablePipeline.java | 8 1 file changed, 8 insertions(+) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (c712231 -> 666d492)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from c712231 Merge pull request #4517: [BEAM-3246] Bigtable: Merge splits if they exceed 15K add fd26777 Add Generic Types to StageBundleFactory add bd9c63c Remove `Components` from ProcessBundleDescriptors new 666d492 Merge pull request #5295: Cleanup in StageBundleFactory, ProcessBundleDescriptors The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../apache/beam/runners/fnexecution/control/JobBundleFactory.java | 2 +- .../beam/runners/fnexecution/control/ProcessBundleDescriptors.java | 6 +++--- .../apache/beam/runners/fnexecution/control/StageBundleFactory.java | 4 ++-- 3 files changed, 6 insertions(+), 6 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5295: Cleanup in StageBundleFactory, ProcessBundleDescriptors
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 666d492d69c40267e2d531ace2059b44231b004b Merge: c712231 bd9c63c Author: Thomas Groh AuthorDate: Fri May 11 13:34:14 2018 -0700 Merge pull request #5295: Cleanup in StageBundleFactory, ProcessBundleDescriptors .../apache/beam/runners/fnexecution/control/JobBundleFactory.java | 2 +- .../beam/runners/fnexecution/control/ProcessBundleDescriptors.java | 6 +++--- .../apache/beam/runners/fnexecution/control/StageBundleFactory.java | 4 ++-- 3 files changed, 6 insertions(+), 6 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5300: Suppress a test in ExecutorServiceParallelExecutorTest
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 4c1e2feabbf48714ff78cda965186c11ab4a4434 Merge: d9215f1 d4eb5e8 Author: Thomas Groh AuthorDate: Tue May 8 09:46:26 2018 -0700 Merge pull request #5300: Suppress a test in ExecutorServiceParallelExecutorTest [BEAM-4088] .../ExecutorServiceParallelExecutorTest.java | 70 +++--- 1 file changed, 35 insertions(+), 35 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (d9215f1 -> 4c1e2fe)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from d9215f1 Merge pull request #5227: [BEAM-4172] Move textio.FileSystem and its registry to a separate package add 7eb1dd7 Suppress Thread Leak Test add d4eb5e8 Rename ExecutorServiceParlallelExecutorTest#test new 4c1e2fe Merge pull request #5300: Suppress a test in ExecutorServiceParallelExecutorTest The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../ExecutorServiceParallelExecutorTest.java | 70 +++--- 1 file changed, 35 insertions(+), 35 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (bf94e36 -> 86fdb74)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from bf94e36 Merge pull request #5173: [BEAM-3773][SQL] Add EnumerableConverter for JDBC support add 1af8537 Add an Unsupported StateRequestHandler add 86fdb74 Merge pull request #5269: Add an Unsupported StateRequestHandler No new revisions were added by this update. Summary of changes: .../beam/runners/fnexecution/state/StateRequestHandler.java | 13 ++--- 1 file changed, 10 insertions(+), 3 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (184b4a6 -> 4971e7e)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 184b4a6 Merge pull request #5172: Add ProtoOverrides add b0cafc9 Add additional methods to QueryablePipeline add 7a948d1 Expand the ExecutableGraph interface new 4971e7e Merge pull request #5228: Expand ExecutableGraph, and add a Portable Representation The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../core/construction/graph/QueryablePipeline.java | 22 +++ .../apache/beam/runners/direct/DirectGraph.java| 4 +- .../beam/runners/direct/ExecutableGraph.java | 2 +- .../apache/beam/runners/direct/PortableGraph.java | 69 .../beam/runners/direct/PortableGraphTest.java | 178 + 5 files changed, 272 insertions(+), 3 deletions(-) create mode 100644 runners/direct-java/src/main/java/org/apache/beam/runners/direct/PortableGraph.java create mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/PortableGraphTest.java -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5228: Expand ExecutableGraph, and add a Portable Representation
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 4971e7ec48ad85d002a1c7596cb436dc8c8e88ea Merge: 184b4a6 7a948d1 Author: Thomas Groh AuthorDate: Thu May 3 15:20:07 2018 -0700 Merge pull request #5228: Expand ExecutableGraph, and add a Portable Representation [BEAM-4073] .../core/construction/graph/QueryablePipeline.java | 22 +++ .../apache/beam/runners/direct/DirectGraph.java| 4 +- .../beam/runners/direct/ExecutableGraph.java | 2 +- .../apache/beam/runners/direct/PortableGraph.java | 69 .../beam/runners/direct/PortableGraphTest.java | 178 + 5 files changed, 272 insertions(+), 3 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5172: Add ProtoOverrides
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 184b4a600e49873c015f1c72cd04e65196210fb1 Merge: b6dadff 02fbd1f Author: Thomas Groh AuthorDate: Thu May 3 14:59:11 2018 -0700 Merge pull request #5172: Add ProtoOverrides [BEAM-4125] .../core/construction/graph/ProtoOverrides.java| 102 + .../construction/graph/ProtoOverridesTest.java | 255 + 2 files changed, 357 insertions(+) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (b6dadff -> 184b4a6)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from b6dadff Merge pull request #5272: Add labels flag to Go SDK Dataflow Runner. add 02fbd1f Add ProtoOverrides new 184b4a6 Merge pull request #5172: Add ProtoOverrides The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../core/construction/graph/ProtoOverrides.java| 102 + .../construction/graph/ProtoOverridesTest.java | 255 + 2 files changed, 357 insertions(+) create mode 100644 runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/graph/ProtoOverrides.java create mode 100644 runners/core-construction-java/src/test/java/org/apache/beam/runners/core/construction/graph/ProtoOverridesTest.java -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5272: Add labels flag to Go SDK Dataflow Runner.
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit b6dadffb3c35648839909790de3a070fb6c34127 Merge: 5cb3969 b390ba3 Author: Thomas Groh AuthorDate: Thu May 3 14:57:00 2018 -0700 Merge pull request #5272: Add labels flag to Go SDK Dataflow Runner. [BEAM-4031] sdks/go/pkg/beam/runners/dataflow/dataflow.go | 8 1 file changed, 8 insertions(+) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (5cb3969 -> b6dadff)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 5cb3969 Merge pull request #5226: [BEAM-3972] Translate portable batch pipelines by proto add ea9ade2 Add labels to Go SDK Dataflow Runner. add b390ba3 Add verbose error message if unmarshaling fails for label flag. new b6dadff Merge pull request #5272: Add labels flag to Go SDK Dataflow Runner. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: sdks/go/pkg/beam/runners/dataflow/dataflow.go | 8 1 file changed, 8 insertions(+) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5234: Add KV and Window Coder Utilities to ModelCoders
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit e0567246354defdef3abc0b41feb8d846fa647b0 Merge: 04b8b44 3a33483 Author: Thomas Groh AuthorDate: Wed May 2 18:57:06 2018 -0700 Merge pull request #5234: Add KV and Window Coder Utilities to ModelCoders .../pipeline/src/main/proto/beam_runner_api.proto | 10 ++ .../core/construction/ModelCoderRegistrar.java | 5 + .../runners/core/construction/ModelCoders.java | 52 + .../runners/core/construction/ModelCodersTest.java | 120 + .../control/ProcessBundleDescriptors.java | 22 ++-- 5 files changed, 193 insertions(+), 16 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (04b8b44 -> e056724)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 04b8b44 Merge pull request #5263: Remove StateRequestHandler from JobBundleFactory add fffbdaa Add KV and Window Coder Utilities to ModelCoders add 3a33483 Use ModelCoders in ProcessBundleDescriptors new e056724 Merge pull request #5234: Add KV and Window Coder Utilities to ModelCoders The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../pipeline/src/main/proto/beam_runner_api.proto | 10 ++ .../core/construction/ModelCoderRegistrar.java | 5 + .../runners/core/construction/ModelCoders.java | 52 + .../runners/core/construction/ModelCodersTest.java | 120 + .../control/ProcessBundleDescriptors.java | 22 ++-- 5 files changed, 193 insertions(+), 16 deletions(-) create mode 100644 runners/core-construction-java/src/test/java/org/apache/beam/runners/core/construction/ModelCodersTest.java -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (6ac67f7 -> 04b8b44)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 6ac67f7 Merge pull request #5259: SpannerIO fixes add 5464996 Remove StateRequestHandler from JobBundleFactory new 04b8b44 Merge pull request #5263: Remove StateRequestHandler from JobBundleFactory The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../org/apache/beam/runners/fnexecution/control/JobBundleFactory.java | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5263: Remove StateRequestHandler from JobBundleFactory
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 04b8b44d96b457501df7b6f30280f267a4825a3b Merge: 6ac67f7 5464996 Author: Thomas Groh AuthorDate: Wed May 2 18:51:30 2018 -0700 Merge pull request #5263: Remove StateRequestHandler from JobBundleFactory .../org/apache/beam/runners/fnexecution/control/JobBundleFactory.java | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5152: Harness Manager Interfaces
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 48f9cce76cf87ffe7073c3ad9c34a9f2f6de4b76 Merge: d19c366 c11c837 Author: Thomas Groh AuthorDate: Wed May 2 12:12:08 2018 -0700 Merge pull request #5152: Harness Manager Interfaces [BEAM-3327] .../fnexecution/artifact/ArtifactSource.java | 41 + .../fnexecution/control/JobBundleFactory.java | 33 + .../fnexecution/control/OutputReceiverFactory.java | 31 .../runners/fnexecution/control/RemoteBundle.java | 43 ++ .../fnexecution/control/RemoteOutputReceiver.java | 38 +++ .../fnexecution/control/SdkHarnessClient.java | 19 +- .../fnexecution/control/StageBundleFactory.java| 38 +++ .../runners/fnexecution/provisioning/JobInfo.java | 37 +++ .../fnexecution/control/RemoteExecutionTest.java | 1 - .../fnexecution/control/SdkHarnessClientTest.java | 1 - 10 files changed, 263 insertions(+), 19 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (d19c366 -> 48f9cce)
This is an automated email from the ASF dual-hosted git repository.
tgroh pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.
from d19c366 Merge pull request #5236
add 74d6867 Create ArtifactSource Interface
add 880d50e Create RemoteBundle interface
add 353be65 Create JobInfo for job declaration by operators.
add c11c837 Create BundleFactory chain.
new 48f9cce Merge pull request #5152: Harness Manager Interfaces
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../{package-info.java => ArtifactSource.java} | 24 +++---
.../{package-info.java => JobBundleFactory.java} | 16 +---
...ackage-info.java => OutputReceiverFactory.java} | 14 ---
.../runners/fnexecution/control/RemoteBundle.java} | 29 +++---
.../RemoteOutputReceiver.java} | 17 +++--
.../fnexecution/control/SdkHarnessClient.java | 19 ++
.../{package-info.java => StageBundleFactory.java} | 21 +---
.../runners/fnexecution/provisioning/JobInfo.java | 22
.../fnexecution/control/RemoteExecutionTest.java | 1 -
.../fnexecution/control/SdkHarnessClientTest.java | 1 -
10 files changed, 99 insertions(+), 65 deletions(-)
copy
runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/artifact/{package-info.java
=> ArtifactSource.java} (59%)
copy
runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/control/{package-info.java
=> JobBundleFactory.java} (62%)
copy
runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/control/{package-info.java
=> OutputReceiverFactory.java} (71%)
copy
runners/{core-java/src/main/java/org/apache/beam/runners/core/KeyedWorkItem.java
=>
java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/control/RemoteBundle.java}
(52%)
copy
runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/{data/RemoteInputDestination.java
=> control/RemoteOutputReceiver.java} (63%)
copy
runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/control/{package-info.java
=> StageBundleFactory.java} (53%)
copy
sdks/java/core/src/main/java/org/apache/beam/sdk/util/DoFnAndMainOutput.java =>
runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/provisioning/JobInfo.java
(60%)
--
To stop receiving notification emails like this one, please contact
tg...@apache.org.
[beam] branch master updated (971f1f1 -> d19c366)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 971f1f1 Fix DynamicDestinations documentation add 1d32930 Remove unused Test Utilities, Test new d19c366 Merge pull request #5236 The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../beam/runners/direct/StructuralKeyTest.java | 90 -- .../runners/direct/portable/StructuralKeyTest.java | 89 - 2 files changed, 179 deletions(-) delete mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/StructuralKeyTest.java delete mode 100644 runners/direct-java/src/test/java/org/apache/beam/runners/direct/portable/StructuralKeyTest.java -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5236
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit d19c3665a41978b6bdf3330dacc4ff4e8404fd79 Merge: 971f1f1 1d32930 Author: Thomas Groh AuthorDate: Wed May 2 11:14:06 2018 -0700 Merge pull request #5236 Remove unused Test .../beam/runners/direct/StructuralKeyTest.java | 90 -- .../runners/direct/portable/StructuralKeyTest.java | 89 - 2 files changed, 179 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (a0036d5 -> 643e3da5)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from a0036d5 [BEAM-3042] Refactor of TransformIOCounters (performance, inheritance). (#5075) add f6d7c21 Findbugs cleanup; no functional changes new 643e3da5 Merge pull request #5186: Findbugs cleanup; no functional changes The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../fnexecution/control/SdkHarnessClientTest.java | 2 +- .../apache/beam/sdk/coders/DoubleCoderTest.java| 4 ++-- .../sdk/options/ProxyInvocationHandlerTest.java| 22 +++--- .../sdk/transforms/display/DisplayDataTest.java| 6 +++--- 4 files changed, 17 insertions(+), 17 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5186: Findbugs cleanup; no functional changes
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 643e3da528b7147c1e56023ac4868ad254c6cdaf Merge: a0036d5 f6d7c21 Author: Thomas Groh AuthorDate: Tue May 1 13:41:26 2018 -0700 Merge pull request #5186: Findbugs cleanup; no functional changes .../fnexecution/control/SdkHarnessClientTest.java | 2 +- .../apache/beam/sdk/coders/DoubleCoderTest.java| 4 ++-- .../sdk/options/ProxyInvocationHandlerTest.java| 22 +++--- .../sdk/transforms/display/DisplayDataTest.java| 6 +++--- 4 files changed, 17 insertions(+), 17 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5237: Mark some ReduceFnRunner arguments Nullable
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit d5b0316060c29581291a8e12058bb4ca2677f92b Merge: 96fb765 59a4a99 Author: Thomas Groh AuthorDate: Tue May 1 08:56:33 2018 -0700 Merge pull request #5237: Mark some ReduceFnRunner arguments Nullable .../apache/beam/runners/core/ReduceFnContextFactory.java | 16 +++- .../org/apache/beam/runners/core/ReduceFnRunner.java | 15 +++ 2 files changed, 18 insertions(+), 13 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (96fb765 -> d5b0316)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 96fb765 [BEAM-4131] Include SDK into Python SDK harness container. add 59a4a99 Mark some ReduceFnRunner arguments Nullable new d5b0316 Merge pull request #5237: Mark some ReduceFnRunner arguments Nullable The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../apache/beam/runners/core/ReduceFnContextFactory.java | 16 +++- .../org/apache/beam/runners/core/ReduceFnRunner.java | 15 +++ 2 files changed, 18 insertions(+), 13 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (77cbcd2 -> 7129343)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 77cbcd2 Removing some unused local variables (#4684) add 85adafa Move examples to godoc examples part 1. new 7129343 Merge pull request #5201: Move text examples to Go Doc examples The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: sdks/go/pkg/beam/coder.go| 6 +- sdks/go/pkg/beam/create.go | 9 +-- sdks/go/pkg/beam/doc.go | 53 +- sdks/go/pkg/beam/doc_test.go | 166 +++ sdks/go/pkg/beam/encoding.go | 6 ++ sdks/go/pkg/beam/flatten.go | 7 -- sdks/go/pkg/beam/forward.go | 48 ++--- sdks/go/pkg/beam/gbk.go | 8 --- sdks/go/pkg/beam/impulse.go | 9 +-- sdks/go/pkg/beam/option.go | 7 +- sdks/go/pkg/beam/pardo.go| 4 +- sdks/go/pkg/beam/util.go | 6 -- 12 files changed, 223 insertions(+), 106 deletions(-) create mode 100644 sdks/go/pkg/beam/doc_test.go -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5201: Move text examples to Go Doc examples
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 712934389c57bf5eaa101dd0101af6cf6bcc37a8 Merge: 77cbcd2 85adafa Author: Thomas Groh AuthorDate: Thu Apr 26 11:30:29 2018 -0700 Merge pull request #5201: Move text examples to Go Doc examples [BEAM-3826] sdks/go/pkg/beam/coder.go| 6 +- sdks/go/pkg/beam/create.go | 9 +-- sdks/go/pkg/beam/doc.go | 53 +- sdks/go/pkg/beam/doc_test.go | 166 +++ sdks/go/pkg/beam/encoding.go | 6 ++ sdks/go/pkg/beam/flatten.go | 7 -- sdks/go/pkg/beam/forward.go | 48 ++--- sdks/go/pkg/beam/gbk.go | 8 --- sdks/go/pkg/beam/impulse.go | 9 +-- sdks/go/pkg/beam/option.go | 7 +- sdks/go/pkg/beam/pardo.go| 4 +- sdks/go/pkg/beam/util.go | 6 -- 12 files changed, 223 insertions(+), 106 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5148: Cleanups in GroupByKeyOnlyEvaluatorFactory
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 6bb6425dec9ba764d8ba70135e47dc71ecfa22dd Merge: cae963a 5a70b7f Author: Thomas Groh AuthorDate: Tue Apr 24 15:59:53 2018 -0700 Merge pull request #5148: Cleanups in GroupByKeyOnlyEvaluatorFactory .../direct/GroupByKeyOnlyEvaluatorFactory.java | 55 +++--- 1 file changed, 7 insertions(+), 48 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (cae963a -> 6bb6425)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from cae963a Merge pull request #5181: Use Generic types in Bundle add 5a70b7f Cleanups in GroupByKeyOnlyEvaluatorFactory new 6bb6425 Merge pull request #5148: Cleanups in GroupByKeyOnlyEvaluatorFactory The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../direct/GroupByKeyOnlyEvaluatorFactory.java | 55 +++--- 1 file changed, 7 insertions(+), 48 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (2bcc606 -> cae963a)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 2bcc606 Merge pull request #4947: [BEAM-3983] Add utils for converting to BigQuery types add 86d052e Add a CollectionT to Bundle add 4adf454 Use Bundle in WatermarkManager new cae963a Merge pull request #5181: Use Generic types in Bundle The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../beam/runners/direct/BundleProcessor.java | 3 +- .../beam/runners/direct/CommittedBundle.java | 16 +++-- .../beam/runners/direct/EvaluationContext.java | 5 +- .../direct/ExecutorServiceParallelExecutor.java| 4 +- .../beam/runners/direct/QuiescenceDriver.java | 9 ++- .../beam/runners/direct/WatermarkManager.java | 80 +++--- .../beam/runners/direct/WatermarkManagerTest.java | 7 +- .../java/org/apache/beam/runners/local/Bundle.java | 27 +++- 8 files changed, 93 insertions(+), 58 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5181: Use Generic types in Bundle
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit cae963a810224ba6ce6c9623ee70fbaa57de9c11 Merge: 2bcc606 4adf454 Author: Thomas Groh AuthorDate: Tue Apr 24 15:59:30 2018 -0700 Merge pull request #5181: Use Generic types in Bundle [BEAM-4153] .../beam/runners/direct/BundleProcessor.java | 3 +- .../beam/runners/direct/CommittedBundle.java | 16 +++-- .../beam/runners/direct/EvaluationContext.java | 5 +- .../direct/ExecutorServiceParallelExecutor.java| 4 +- .../beam/runners/direct/QuiescenceDriver.java | 9 ++- .../beam/runners/direct/WatermarkManager.java | 80 +++--- .../beam/runners/direct/WatermarkManagerTest.java | 7 +- .../java/org/apache/beam/runners/local/Bundle.java | 27 +++- 8 files changed, 93 insertions(+), 58 deletions(-) diff --cc runners/direct-java/src/test/java/org/apache/beam/runners/direct/WatermarkManagerTest.java index 5c4fdf9,d105efe..b8aef85 --- a/runners/direct-java/src/test/java/org/apache/beam/runners/direct/WatermarkManagerTest.java +++ b/runners/direct-java/src/test/java/org/apache/beam/runners/direct/WatermarkManagerTest.java @@@ -67,8 -65,10 +67,7 @@@ import org.apache.beam.sdk.util.Windowe import org.apache.beam.sdk.values.KV; import org.apache.beam.sdk.values.PCollection; import org.apache.beam.sdk.values.PCollectionList; - import org.apache.beam.sdk.values.PValue; import org.apache.beam.sdk.values.TimestampedValue; -import org.hamcrest.BaseMatcher; -import org.hamcrest.Description; -import org.hamcrest.Matcher; import org.hamcrest.Matchers; import org.joda.time.Duration; import org.joda.time.Instant; -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (756812f -> d3cd611)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 756812f Merge pull request #5189: Basic Docker environment factory add 7948014 Stop implementing EvaluatorFactory in Registry new d3cd611 Merge pull request #5129: Stop implementing EvaluatorFactory in Registry The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../org/apache/beam/runners/direct/DirectTransformExecutor.java | 8 .../apache/beam/runners/direct/TransformEvaluatorRegistry.java| 4 +--- 2 files changed, 5 insertions(+), 7 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5129: Stop implementing EvaluatorFactory in Registry
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit d3cd6116517cce04cf5c35bbd2d05a494dd8fed3 Merge: 756812f 7948014 Author: Thomas Groh AuthorDate: Tue Apr 24 10:25:03 2018 -0700 Merge pull request #5129: Stop implementing EvaluatorFactory in Registry .../org/apache/beam/runners/direct/DirectTransformExecutor.java | 8 .../apache/beam/runners/direct/TransformEvaluatorRegistry.java| 4 +--- 2 files changed, 5 insertions(+), 7 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (feb1027 -> 756812f)
This is an automated email from the ASF dual-hosted git repository.
tgroh pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.
from feb1027 Merge pull request #5202: [BEAM-4066] Fix Nexmark fails when
running with PubSub
add 6686b91 [BEAM-4149] Set worker id to "" if it is not set in the
request header
add 0aaf8ec [BEAM-3327] Refactor ControlClientPool to allow client
multiplexing
add ff81f7a [BEAM-3327] Basic Docker environment factory
add 756812f Merge pull request #5189: Basic Docker environment factory
No new revisions were added by this update.
Summary of changes:
runners/java-fn-execution/build.gradle | 15 ++
runners/java-fn-execution/pom.xml | 47 +++
.../GrpcContextHeaderAccessorProvider.java | 9 +-
.../fnexecution/control/ControlClientPool.java | 62 -
.../control/FnApiControlClientPoolService.java | 60 +---
.../control/InstructionRequestHandler.java | 3 +-
.../fnexecution/control/MapControlClientPool.java | 78 +++
.../control/QueueControlClientPool.java| 62 -
.../fnexecution/environment/DockerCommand.java | 142 +++
.../environment/DockerContainerEnvironment.java| 86
.../environment/DockerEnvironmentFactory.java | 154 +
.../environment/EnvironmentFactory.java| 14 +-
.../environment/testing/NeedsDocker.java | 9 +-
.../environment/testing}/package-info.java | 6 +-
.../runners/fnexecution/InProcessSdkHarness.java | 27 +++-
.../control/FnApiControlClientPoolServiceTest.java | 12 +-
.../fnexecution/control/RemoteExecutionTest.java | 7 +-
.../fnexecution/control/SdkHarnessClientTest.java | 5 +-
.../fnexecution/environment/DockerCommandTest.java | 84 +++
.../environment/DockerEnvironmentFactoryTest.java | 122
20 files changed, 873 insertions(+), 131 deletions(-)
create mode 100644
runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/control/MapControlClientPool.java
delete mode 100644
runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/control/QueueControlClientPool.java
create mode 100644
runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/environment/DockerCommand.java
create mode 100644
runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/environment/DockerContainerEnvironment.java
create mode 100644
runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/environment/DockerEnvironmentFactory.java
copy
sdks/java/core/src/main/java/org/apache/beam/sdk/testing/UsesParDoLifecycle.java
=>
runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/environment/testing/NeedsDocker.java
(83%)
copy
runners/{apex/src/main/java/org/apache/beam/runners/apex/translation/operators
=>
java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/environment/testing}/package-info.java
(86%)
create mode 100644
runners/java-fn-execution/src/test/java/org/apache/beam/runners/fnexecution/environment/DockerCommandTest.java
create mode 100644
runners/java-fn-execution/src/test/java/org/apache/beam/runners/fnexecution/environment/DockerEnvironmentFactoryTest.java
--
To stop receiving notification emails like this one, please contact
tg...@apache.org.
[beam] branch master updated (229e69d -> 0f2ba71)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 229e69d Merge pull request #5185: [BEAM-4086]: KafkaIO tests: Avoid busy loop in MockConsumer.poll(), reduce flakes. add d9cd312 Use Existing Matchers in WatermarkManagerTest add 0f2ba71 Merge pull request #5195: Use Existing Matchers in WatermarkManagerTest No new revisions were added by this update. Summary of changes: .../beam/runners/direct/WatermarkManagerTest.java | 189 + 1 file changed, 85 insertions(+), 104 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5119: Removing PrimitiveCombineGroupedValues override w/ FnAPI
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit fdb051e2ffebdf023e21216d297466a53b425d43 Merge: 247a62f 084fbbc Author: Thomas Groh AuthorDate: Mon Apr 23 11:25:53 2018 -0700 Merge pull request #5119: Removing PrimitiveCombineGroupedValues override w/ FnAPI [BEAM-3513] .../apache/beam/runners/dataflow/DataflowRunner.java| 17 +++-- 1 file changed, 11 insertions(+), 6 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (247a62f -> fdb051e)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 247a62f Merge pull request #5112 from thelastpickle/BEAM-4049-improve-cassandra-io add 084fbbc [BEAM-3513] Removing PrimitiveCombineGroupedValues override w/ FnAPI. new fdb051e Merge pull request #5119: Removing PrimitiveCombineGroupedValues override w/ FnAPI The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../apache/beam/runners/dataflow/DataflowRunner.java| 17 +++-- 1 file changed, 11 insertions(+), 6 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5177: Stop taking the whole result in WatermarkManager
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 3eacf8b8d7301d1bd4db17874a31caf3b9949526 Merge: 0e87ba1 f0c5d4c Author: Thomas Groh AuthorDate: Thu Apr 19 20:56:17 2018 -0700 Merge pull request #5177: Stop taking the whole result in WatermarkManager [BEAM-4135] .../beam/runners/direct/EvaluationContext.java | 13 +- .../beam/runners/direct/QuiescenceDriver.java | 4 +- .../beam/runners/direct/WatermarkManager.java | 148 --- .../beam/runners/direct/WatermarkManagerTest.java | 481 ++--- 4 files changed, 312 insertions(+), 334 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (0e87ba1 -> 3eacf8b)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 0e87ba1 Merge pull request #5190: Add pubsub to Go maven build add 291745c Remove Unused WatermarkManager Method add f0c5d4c Stop taking the whole result in WatermarkManager new 3eacf8b Merge pull request #5177: Stop taking the whole result in WatermarkManager The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../beam/runners/direct/EvaluationContext.java | 13 +- .../beam/runners/direct/QuiescenceDriver.java | 4 +- .../beam/runners/direct/WatermarkManager.java | 148 --- .../beam/runners/direct/WatermarkManagerTest.java | 481 ++--- 4 files changed, 312 insertions(+), 334 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5169: Add a Test for Length Prefix between Coders
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 050f1795a9921c3427eb02e34aa8b8a696573901 Merge: f5afe53 6c433c5 Author: Thomas Groh AuthorDate: Thu Apr 19 12:01:54 2018 -0700 Merge pull request #5169: Add a Test for Length Prefix between Coders .../beam/sdk/coders/LengthPrefixCoderTest.java | 27 ++ 1 file changed, 27 insertions(+) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (f5afe53 -> 050f179)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from f5afe53 Merge pull request #4761: Remove SdkHarnessClientControlService add 6c433c5 Add a Test for Length Prefix between Coders new 050f179 Merge pull request #5169: Add a Test for Length Prefix between Coders The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../beam/sdk/coders/LengthPrefixCoderTest.java | 27 ++ 1 file changed, 27 insertions(+) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (3cdecd0 -> 7300eb2)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 3cdecd0 Merge pull request #5055 from coheigea/equalsIgnoreCase add 07ae520 BEAM-3339 Mobile gaming automation for Java nightly snapshot new 7300eb2 Merge pull request #4788: Add Mobile gaming automation for Java nightly snapshot on core runners The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../job_beam_PostRelease_NightlySnapshot.groovy| 4 +- build_rules.gradle | 43 -- .../examples/complete/game/injector/Injector.java | 2 + release/build.gradle | 4 +- .../src/main/groovy/MobileGamingCommands.groovy| 144 + release/src/main/groovy/QuickstartArchetype.groovy | 16 ++- release/src/main/groovy/TestScripts.groovy | 77 --- .../main/groovy/mobilegaming-java-dataflow.groovy | 111 .../main/groovy/mobilegaming-java-direct.groovy| 105 +++ .../src/main/groovy/quickstart-java-apex.groovy| 6 +- .../main/groovy/quickstart-java-dataflow.groovy| 6 +- .../src/main/groovy/quickstart-java-direct.groovy | 6 +- .../main/groovy/quickstart-java-flinklocal.groovy | 6 +- .../src/main/groovy/quickstart-java-spark.groovy | 6 +- runners/apex/build.gradle | 2 +- runners/direct-java/build.gradle | 15 ++- runners/flink/build.gradle | 2 +- runners/google-cloud-dataflow-java/build.gradle| 24 +++- runners/spark/build.gradle | 2 +- 19 files changed, 513 insertions(+), 68 deletions(-) create mode 100644 release/src/main/groovy/MobileGamingCommands.groovy create mode 100644 release/src/main/groovy/mobilegaming-java-dataflow.groovy create mode 100644 release/src/main/groovy/mobilegaming-java-direct.groovy -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #4788: Add Mobile gaming automation for Java nightly snapshot on core runners
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 7300eb2bfdd412767e60173c0a0e0d76ba7d067d Merge: 3cdecd0 07ae520 Author: Thomas Groh AuthorDate: Wed Apr 18 18:04:30 2018 -0700 Merge pull request #4788: Add Mobile gaming automation for Java nightly snapshot on core runners [BEAM-3339] .../job_beam_PostRelease_NightlySnapshot.groovy| 4 +- build_rules.gradle | 43 -- .../examples/complete/game/injector/Injector.java | 2 + release/build.gradle | 4 +- .../src/main/groovy/MobileGamingCommands.groovy| 144 + release/src/main/groovy/QuickstartArchetype.groovy | 16 ++- release/src/main/groovy/TestScripts.groovy | 77 --- .../main/groovy/mobilegaming-java-dataflow.groovy | 111 .../main/groovy/mobilegaming-java-direct.groovy| 105 +++ .../src/main/groovy/quickstart-java-apex.groovy| 6 +- .../main/groovy/quickstart-java-dataflow.groovy| 6 +- .../src/main/groovy/quickstart-java-direct.groovy | 6 +- .../main/groovy/quickstart-java-flinklocal.groovy | 6 +- .../src/main/groovy/quickstart-java-spark.groovy | 6 +- runners/apex/build.gradle | 2 +- runners/direct-java/build.gradle | 15 ++- runners/flink/build.gradle | 2 +- runners/google-cloud-dataflow-java/build.gradle| 24 +++- runners/spark/build.gradle | 2 +- 19 files changed, 513 insertions(+), 68 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5126: Gracefully deserialize empty options structs
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit b8b4510f4fdc287638cc065a906cae881476bf84 Merge: e567675 86f375e Author: Thomas Groh AuthorDate: Wed Apr 18 11:02:35 2018 -0700 Merge pull request #5126: Gracefully deserialize empty options structs [BEAM-4069] .../PipelineOptionsTranslationTest.java | 21 + .../beam/sdk/options/ProxyInvocationHandler.java| 16 ++-- 2 files changed, 31 insertions(+), 6 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (e567675 -> b8b4510)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from e567675 Merge pull request #5163 Fix python presubmit add 86f375e [BEAM-4069] Gracefully deserialize empty options structs new b8b4510 Merge pull request #5126: Gracefully deserialize empty options structs The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../PipelineOptionsTranslationTest.java | 21 + .../beam/sdk/options/ProxyInvocationHandler.java| 16 ++-- 2 files changed, 31 insertions(+), 6 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5144: Rename EnvironmentManager to EnvironmentFactory
This is an automated email from the ASF dual-hosted git repository.
tgroh pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
commit a6e6f2dc91fed81dd3cdd04bd70c96f3ee6b1473
Merge: b9fb8f6 c5a2c6a
Author: Thomas Groh
AuthorDate: Tue Apr 17 09:03:01 2018 -0700
Merge pull request #5144: Rename EnvironmentManager to EnvironmentFactory
[BEAM-3327]
.../environment/{EnvironmentManager.java => EnvironmentFactory.java} | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
--
To stop receiving notification emails like this one, please contact
tg...@apache.org.
[beam] branch master updated (b9fb8f6 -> a6e6f2d)
This is an automated email from the ASF dual-hosted git repository.
tgroh pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.
from b9fb8f6 Merge pull request #5040 from apilloud/ddl
add c5a2c6a Rename EnvironmentManager to EnvironmentFactory
new a6e6f2d Merge pull request #5144: Rename EnvironmentManager to
EnvironmentFactory
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../environment/{EnvironmentManager.java => EnvironmentFactory.java} | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
rename
runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/environment/{EnvironmentManager.java
=> EnvironmentFactory.java} (97%)
--
To stop receiving notification emails like this one, please contact
tg...@apache.org.
[beam] 01/01: Merge pull request #5118: Identify side inputs by transform id and local name
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit a7819fec7cfb74e96979dcb5055567dcba827cfa Merge: 5f75e14 a1f3f2b Author: Thomas Groh AuthorDate: Mon Apr 16 11:36:52 2018 -0700 Merge pull request #5118: Identify side inputs by transform id and local name [BEAM-4056] .../pipeline/src/main/proto/beam_runner_api.proto | 18 +-- .../core/construction/graph/ExecutableStage.java | 22 +--- .../core/construction/graph/GreedyStageFuser.java | 2 +- .../graph/ImmutableExecutableStage.java| 6 +-- .../core/construction/graph/QueryablePipeline.java | 17 +++--- .../construction/graph/SideInputReference.java | 61 ++ .../construction/graph/ExecutableStageMatcher.java | 36 +++-- .../construction/graph/ExecutableStageTest.java| 7 ++- .../graph/GreedyPipelineFuserTest.java | 6 +++ .../construction/graph/GreedyStageFuserTest.java | 10 ++-- .../graph/ImmutableExecutableStageTest.java| 7 ++- .../construction/graph/QueryablePipelineTest.java | 24 ++--- 12 files changed, 177 insertions(+), 39 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (5f75e14 -> a7819fe)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 5f75e14 Merge pull request #5133 from jasonkuster/patch-8 add 19c83e6 [BEAM-4056] Identify side inputs by transform id and local name add a1f3f2b Add side input assertions to ExecutableStageMatcher new a7819fe Merge pull request #5118: Identify side inputs by transform id and local name The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../pipeline/src/main/proto/beam_runner_api.proto | 18 +-- .../core/construction/graph/ExecutableStage.java | 22 +--- .../core/construction/graph/GreedyStageFuser.java | 2 +- .../graph/ImmutableExecutableStage.java| 6 +-- .../core/construction/graph/QueryablePipeline.java | 17 +++--- .../construction/graph/SideInputReference.java | 61 ++ .../construction/graph/ExecutableStageMatcher.java | 36 +++-- .../construction/graph/ExecutableStageTest.java| 7 ++- .../graph/GreedyPipelineFuserTest.java | 6 +++ .../construction/graph/GreedyStageFuserTest.java | 10 ++-- .../graph/ImmutableExecutableStageTest.java| 7 ++- .../construction/graph/QueryablePipelineTest.java | 24 ++--- 12 files changed, 177 insertions(+), 39 deletions(-) create mode 100644 runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/graph/SideInputReference.java -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (96615a1 -> 5f75e14)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 96615a1 Merge pull request #5085: Update Dataflow Development Container Version add 1073baa Add region to dataflowOptions struct. new 5f75e14 Merge pull request #5133 from jasonkuster/patch-8 The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: sdks/go/pkg/beam/runners/dataflow/dataflow.go | 1 + 1 file changed, 1 insertion(+) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5133 from jasonkuster/patch-8
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 5f75e14b007af6ca676c73dab0b9025008ad3816 Merge: 96615a1 1073baa Author: Thomas Groh AuthorDate: Mon Apr 16 11:35:53 2018 -0700 Merge pull request #5133 from jasonkuster/patch-8 Add region to dataflowOptions struct. sdks/go/pkg/beam/runners/dataflow/dataflow.go | 1 + 1 file changed, 1 insertion(+) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5085: Update Dataflow Development Container Version
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 96615a1c5dfc2e67c2d408cc4f589952036988e8 Merge: 0ccdd54 748f9e5 Author: Thomas Groh AuthorDate: Mon Apr 16 09:16:12 2018 -0700 Merge pull request #5085: Update Dataflow Development Container Version runners/google-cloud-dataflow-java/build.gradle | 2 +- runners/google-cloud-dataflow-java/pom.xml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (0ccdd54 -> 96615a1)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 0ccdd54 Merge pull request #5125: Use Explicit PipelineOptions in Native Evaluators add 748f9e5 Update Dataflow Development Container Version new 96615a1 Merge pull request #5085: Update Dataflow Development Container Version The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: runners/google-cloud-dataflow-java/build.gradle | 2 +- runners/google-cloud-dataflow-java/pom.xml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5127: Add region flag to dataflow.go
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit c3bb6be6785c0bb502307cd9e13534f3602d8ed6 Merge: 7b30ca6 ece34f7 Author: Thomas Groh AuthorDate: Fri Apr 13 16:02:25 2018 -0700 Merge pull request #5127: Add region flag to dataflow.go sdks/go/pkg/beam/runners/dataflow/dataflow.go | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (7b30ca6 -> c3bb6be)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 7b30ca6 Merge pull request #5008: Use typed client pool sinks and sources add 9837af2 Add region argument to dataflow.go add ece34f7 Region isn't on proto; create and get instead. new c3bb6be Merge pull request #5127: Add region flag to dataflow.go The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: sdks/go/pkg/beam/runners/dataflow/dataflow.go | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (a716332 -> 7b30ca6)
This is an automated email from the ASF dual-hosted git repository.
tgroh pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.
from a716332 Merge pull request #5123: Rename `defaultRegistry` to
`javaSdkNativeRegistry`
add 87361e0 [BEAM-3994] Use typed client pool sinks and sources
add b2e8517 [BEAM-3966] Move functional utilities into shared module
add 9a70eef Use general functional interfaces in ControlClientPool
add d1c80ef Rename createLinked() to createBuffered() in
QueueControlClientPool
new 7b30ca6 Merge pull request #5008: Use typed client pool sinks and
sources
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
...nRequestHandler.java => ControlClientPool.java} | 17 +++---
.../control/FnApiControlClientPoolService.java | 14 +++--
.../control/QueueControlClientPool.java| 62 ++
.../control/SdkHarnessClientControlService.java| 13 ++---
.../runners/fnexecution/InProcessSdkHarness.java | 13 ++---
.../control/FnApiControlClientPoolServiceTest.java | 11 ++--
.../fnexecution/control/RemoteExecutionTest.java | 8 ++-
.../fn/function}/CloseableThrowingConsumer.java| 3 +-
.../beam/sdk/fn/function}/ThrowingBiConsumer.java | 2 +-
.../beam/sdk/fn/function}/ThrowingBiFunction.java | 2 +-
.../beam/sdk/fn/function}/ThrowingConsumer.java| 3 +-
.../beam/sdk/fn/function}/ThrowingFunction.java| 2 +-
.../beam/sdk/fn/function}/ThrowingRunnable.java| 2 +-
.../apache/beam/sdk/fn/function}/package-info.java | 6 +--
.../beam/fn/harness/BeamFnDataReadRunner.java | 2 +-
.../beam/fn/harness/BeamFnDataWriteRunner.java | 2 +-
.../beam/fn/harness/BoundedSourceRunner.java | 2 +-
.../org/apache/beam/fn/harness/FlattenRunner.java | 2 +-
.../apache/beam/fn/harness/FnApiDoFnRunner.java| 2 +-
.../java/org/apache/beam/fn/harness/FnHarness.java | 2 +-
.../org/apache/beam/fn/harness/MapFnRunner.java| 4 +-
.../beam/fn/harness/PTransformRunnerFactory.java | 2 +-
.../beam/fn/harness/WindowMappingFnRunner.java | 2 +-
.../fn/harness/control/BeamFnControlClient.java| 2 +-
.../fn/harness/control/ProcessBundleHandler.java | 2 +-
.../apache/beam/fn/harness/fn/package-info.java| 22
.../beam/fn/harness/BeamFnDataReadRunnerTest.java | 2 +-
.../beam/fn/harness/BeamFnDataWriteRunnerTest.java | 2 +-
.../beam/fn/harness/BoundedSourceRunnerTest.java | 2 +-
.../beam/fn/harness/FnApiDoFnRunnerTest.java | 2 +-
.../apache/beam/fn/harness/MapFnRunnerTest.java| 4 +-
.../beam/fn/harness/WindowMappingFnRunnerTest.java | 2 +-
.../harness/control/BeamFnControlClientTest.java | 2 +-
.../harness/control/ProcessBundleHandlerTest.java | 4 +-
34 files changed, 134 insertions(+), 90 deletions(-)
copy
runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/control/{InstructionRequestHandler.java
=> ControlClientPool.java} (68%)
create mode 100644
runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/control/QueueControlClientPool.java
rename sdks/java/{harness/src/main/java/org/apache/beam/fn/harness/fn =>
fn-execution/src/main/java/org/apache/beam/sdk/fn/function}/CloseableThrowingConsumer.java
(95%)
rename sdks/java/{harness/src/main/java/org/apache/beam/fn/harness/fn =>
fn-execution/src/main/java/org/apache/beam/sdk/fn/function}/ThrowingBiConsumer.java
(96%)
rename sdks/java/{harness/src/main/java/org/apache/beam/fn/harness/fn =>
fn-execution/src/main/java/org/apache/beam/sdk/fn/function}/ThrowingBiFunction.java
(96%)
rename sdks/java/{harness/src/main/java/org/apache/beam/fn/harness/fn =>
fn-execution/src/main/java/org/apache/beam/sdk/fn/function}/ThrowingConsumer.java
(96%)
rename sdks/java/{harness/src/main/java/org/apache/beam/fn/harness/fn =>
fn-execution/src/main/java/org/apache/beam/sdk/fn/function}/ThrowingFunction.java
(96%)
rename sdks/java/{harness/src/main/java/org/apache/beam/fn/harness/fn =>
fn-execution/src/main/java/org/apache/beam/sdk/fn/function}/ThrowingRunnable.java
(96%)
copy {runners/spark/src/main/java/org/apache/beam/runners/spark/io =>
sdks/java/fn-execution/src/main/java/org/apache/beam/sdk/fn/function}/package-info.java
(90%)
delete mode 100644
sdks/java/harness/src/main/java/org/apache/beam/fn/harness/fn/package-info.java
--
To stop receiving notification emails like this one, please contact
tg...@apache.org.
[beam] 01/01: Merge pull request #5008: Use typed client pool sinks and sources
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 7b30ca64b3ef312610f2b78008ab82f62697fd30 Merge: a716332 d1c80ef Author: Thomas Groh AuthorDate: Fri Apr 13 15:28:35 2018 -0700 Merge pull request #5008: Use typed client pool sinks and sources [BEAM-3994] .../fnexecution/control/ControlClientPool.java | 23 .../control/FnApiControlClientPoolService.java | 14 +++-- .../control/QueueControlClientPool.java| 62 ++ .../control/SdkHarnessClientControlService.java| 13 ++--- .../runners/fnexecution/InProcessSdkHarness.java | 13 ++--- .../control/FnApiControlClientPoolServiceTest.java | 11 ++-- .../fnexecution/control/RemoteExecutionTest.java | 8 ++- .../fn/function}/CloseableThrowingConsumer.java| 3 +- .../beam/sdk/fn/function}/ThrowingBiConsumer.java | 2 +- .../beam/sdk/fn/function}/ThrowingBiFunction.java | 2 +- .../beam/sdk/fn/function}/ThrowingConsumer.java| 3 +- .../beam/sdk/fn/function}/ThrowingFunction.java| 2 +- .../beam/sdk/fn/function}/ThrowingRunnable.java| 2 +- .../apache/beam/sdk/fn/function}/package-info.java | 6 +-- .../beam/fn/harness/BeamFnDataReadRunner.java | 2 +- .../beam/fn/harness/BeamFnDataWriteRunner.java | 2 +- .../beam/fn/harness/BoundedSourceRunner.java | 2 +- .../org/apache/beam/fn/harness/FlattenRunner.java | 2 +- .../apache/beam/fn/harness/FnApiDoFnRunner.java| 2 +- .../java/org/apache/beam/fn/harness/FnHarness.java | 2 +- .../org/apache/beam/fn/harness/MapFnRunner.java| 4 +- .../beam/fn/harness/PTransformRunnerFactory.java | 2 +- .../beam/fn/harness/WindowMappingFnRunner.java | 2 +- .../fn/harness/control/BeamFnControlClient.java| 2 +- .../fn/harness/control/ProcessBundleHandler.java | 2 +- .../beam/fn/harness/BeamFnDataReadRunnerTest.java | 2 +- .../beam/fn/harness/BeamFnDataWriteRunnerTest.java | 2 +- .../beam/fn/harness/BoundedSourceRunnerTest.java | 2 +- .../beam/fn/harness/FnApiDoFnRunnerTest.java | 2 +- .../apache/beam/fn/harness/MapFnRunnerTest.java| 4 +- .../beam/fn/harness/WindowMappingFnRunnerTest.java | 2 +- .../harness/control/BeamFnControlClientTest.java | 2 +- .../harness/control/ProcessBundleHandlerTest.java | 4 +- 33 files changed, 134 insertions(+), 74 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] 01/01: Merge pull request #5123: Rename `defaultRegistry` to `javaSdkNativeRegistry`
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit a716332a2b2babc7f50236c50762b0d9a938f395 Merge: 6107c31 908effc Author: Thomas Groh AuthorDate: Fri Apr 13 14:50:24 2018 -0700 Merge pull request #5123: Rename `defaultRegistry` to `javaSdkNativeRegistry` .../main/java/org/apache/beam/runners/direct/DirectRunner.java | 2 +- .../apache/beam/runners/direct/TransformEvaluatorRegistry.java | 9 - 2 files changed, 9 insertions(+), 2 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
[beam] branch master updated (6107c31 -> a716332)
This is an automated email from the ASF dual-hosted git repository. tgroh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 6107c31 Merge pull request #4995: Fix GreedyPipelineFuser test add 908effc Rename `defaultRegistry` to `javaSdkNativeRegistry` new a716332 Merge pull request #5123: Rename `defaultRegistry` to `javaSdkNativeRegistry` The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../main/java/org/apache/beam/runners/direct/DirectRunner.java | 2 +- .../apache/beam/runners/direct/TransformEvaluatorRegistry.java | 9 - 2 files changed, 9 insertions(+), 2 deletions(-) -- To stop receiving notification emails like this one, please contact tg...@apache.org.
