[GitHub] incubator-beam pull request #652: Added cy_combiners.py to the list of Cytho...

[mdvorsky]

GitHub user mdvorsky opened a pull request:

https://github.com/apache/incubator-beam/pull/652

Added cy_combiners.py to the list of Cythonized Python SDK files.

R: @robertwb 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mdvorsky/incubator-beam cythonize

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/652.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #652


commit 1a8ef4fc3facca2a2545f6a20572118e8ba3d45a
Author: Marian Dvorsky 
Date:   2016-07-14T00:16:58Z

Added cy_combiners.py to the list of Cythonized Python SDK files.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (BEAM-434) Limit the number of output files a beam-examples execution writes

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15376035#comment-15376035
 ] 

ASF GitHub Bot commented on BEAM-434:
-

GitHub user tgroh opened a pull request:

https://github.com/apache/incubator-beam/pull/651

[BEAM-434] Control the number of output shards in the Direct Runner

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---

Add a Write Override Factory that limits the number of shards is
unspecified. This ensures that we will not write an output file per-key
due to bundling.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tgroh/incubator-beam write_sharding

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/651.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #651


commit dfecd170d67c90ae20a4baed3986bd1c7116e79e
Author: Thomas Groh 
Date:   2016-07-13T21:26:10Z

Control the number of output shards in the Direct Runner

Add a Write Override Factory that limits the number of shards is
unspecified. This ensures that we will not write an output file per-key
due to bundling.




> Limit the number of output files a beam-examples execution writes
> -
>
> Key: BEAM-434
> URL: https://issues.apache.org/jira/browse/BEAM-434
> Project: Beam
>  Issue Type: Bug
>  Components: examples-java
>Reporter: Amit Sela
>Assignee: Amit Sela
>Priority: Minor
>
> When using `TextIO.Write.to("/path/to/output")` without any restrictions on 
> the number of shards, it might generate many output files (depending on your 
> input), for WordCount for example, you'll get as many output files as unique 
> words in your input.
> Since I think examples are expected to execute in a friendly manner to "see" 
> what it does and not optimize for performance in some way, I suggest to use 
> `withoutSharding()` when writing the example output to an output file.
> Examples I could find that behave this way:
> org.apache.beam.examples.WordCount
> org.apache.beam.examples.complete.TfIdf
> org.apache.beam.examples.cookbook.DeDupExample



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-beam pull request #651: [BEAM-434] Control the number of output sh...

[tgroh]

GitHub user tgroh opened a pull request:

https://github.com/apache/incubator-beam/pull/651

[BEAM-434] Control the number of output shards in the Direct Runner

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---

Add a Write Override Factory that limits the number of shards is
unspecified. This ensures that we will not write an output file per-key
due to bundling.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tgroh/incubator-beam write_sharding

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/651.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #651


commit dfecd170d67c90ae20a4baed3986bd1c7116e79e
Author: Thomas Groh 
Date:   2016-07-13T21:26:10Z

Control the number of output shards in the Direct Runner

Add a Write Override Factory that limits the number of shards is
unspecified. This ensures that we will not write an output file per-key
due to bundling.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-beam pull request #650: Update some of the example tests to use as...

[aaltay]

GitHub user aaltay opened a pull request:

https://github.com/apache/incubator-beam/pull/650

Update some of the example tests to use assert_that

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/aaltay/incubator-beam example

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/650.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #650


commit e1545038cca71dcba46bf44da1e2261e0db7e1b5
Author: Ahmet Altay 
Date:   2016-07-13T23:28:46Z

Update some of the example tests to use assert_that




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Jenkins build is back to normal : beam_PostCommit_RunnableOnService_GoogleCloudDataflow #751

[Apache Jenkins Server]

See 

Jenkins build is back to normal : beam_PostCommit_MavenVerify #880

[Apache Jenkins Server]

See 

Jenkins build is back to normal : beam_PostCommit_MavenVerify » Apache Beam :: SDKs :: Java :: Core #880

[Apache Jenkins Server]

See 

[2/2] incubator-beam git commit: [BEAM-446] Fix checkstyle error: remove unused import

[lcwik]

[BEAM-446] Fix checkstyle error: remove unused import

This closes #647


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/f0119b23
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/f0119b23
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/f0119b23

Branch: refs/heads/master
Commit: f0119b23ad903e081e6ff503903094e504d68119
Parents: 2596d13 d608efb
Author: Luke Cwik 
Authored: Wed Jul 13 15:06:49 2016 -0700
Committer: Luke Cwik 
Committed: Wed Jul 13 15:06:49 2016 -0700

--
 .../src/test/java/org/apache/beam/sdk/util/IOChannelUtilsTest.java  | 1 -
 1 file changed, 1 deletion(-)
--

[GitHub] incubator-beam pull request #647: Fix checkstyle error: remove unused import

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/647


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[1/2] incubator-beam git commit: Fix checkstyle error: remove unused import

[lcwik]

Repository: incubator-beam
Updated Branches:
  refs/heads/master 2596d136d -> f0119b23a


Fix checkstyle error: remove unused import


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/d608efb7
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/d608efb7
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/d608efb7

Branch: refs/heads/master
Commit: d608efb77408f970e84f922ef41723920514
Parents: 2596d13
Author: Scott Wegner 
Authored: Wed Jul 13 13:32:40 2016 -0700
Committer: Scott Wegner 
Committed: Wed Jul 13 13:32:40 2016 -0700

--
 .../src/test/java/org/apache/beam/sdk/util/IOChannelUtilsTest.java  | 1 -
 1 file changed, 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/d608efb7/sdks/java/core/src/test/java/org/apache/beam/sdk/util/IOChannelUtilsTest.java
--
diff --git 
a/sdks/java/core/src/test/java/org/apache/beam/sdk/util/IOChannelUtilsTest.java 
b/sdks/java/core/src/test/java/org/apache/beam/sdk/util/IOChannelUtilsTest.java
index 8db2f75..12312d1 100644
--- 
a/sdks/java/core/src/test/java/org/apache/beam/sdk/util/IOChannelUtilsTest.java
+++ 
b/sdks/java/core/src/test/java/org/apache/beam/sdk/util/IOChannelUtilsTest.java
@@ -30,7 +30,6 @@ import org.junit.runner.RunWith;
 import org.junit.runners.JUnit4;
 
 import java.io.File;
-import java.io.IOException;
 import java.nio.charset.StandardCharsets;
 
 /**

[GitHub] incubator-beam pull request #649: Fix build -- checkstyle error

[dhalperi]

Github user dhalperi closed the pull request at:

https://github.com/apache/incubator-beam/pull/649


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-beam pull request #649: Fix build -- checkstyle error

[dhalperi]

GitHub user dhalperi opened a pull request:

https://github.com/apache/incubator-beam/pull/649

Fix build -- checkstyle error

R: @lukecwik @markflyhigh

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dhalperi/incubator-beam patch-1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/649.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #649


commit 98b939ea25c04a9965657fcf09334ee15c7ddce4
Author: Daniel Halperin 
Date:   2016-07-13T21:59:14Z

Fix build -- checkstyle error




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-beam pull request #648: [BEAM-445] Switch to netty-tcnative uber-j...

[swegner]

GitHub user swegner opened a pull request:

https://github.com/apache/incubator-beam/pull/648

[BEAM-445] Switch to netty-tcnative uber-jar dependency

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---
Currently, we are using the os-maven-plugin extension to compute the 
platform-specific version of the netty-tcnative dependency to use in gcp-io. 
Unfortunately, netty-tcnative is a transitive dependency and all pom's which 
depend on gcp-io (directly or transitively) also need to declare the 
os-maven-plugin dependency.

This PR removes os-maven-plugin and it's dynamic resolution. We instead 
depend on the "uber jar" version of netty-tcnative, which includes all 
platform-native versions.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/swegner/incubator-beam os-maven-plugin

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/648.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #648


commit 55ed28eb4d8305ca1968d9b012b22779814d0a44
Author: Scott Wegner 
Date:   2016-07-12T23:20:10Z

Remove netty-tcnative dynamic os dependency




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Build failed in Jenkins: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #750

[Apache Jenkins Server]

See 


Changes:

[lcwik] [BEAM-446] Improve IOChannelUtils.resolve to accept multiple paths

[lcwik] remove old method and fix javadoc

[lcwik] fix javadoc issue

--
[...truncated 7004 lines...]
[INFO] Excluding org.tukaani:xz:jar:1.5 from the shaded jar.
[INFO] Excluding com.google.auto.service:auto-service:jar:1.0-rc2 from the 
shaded jar.
[INFO] Excluding com.google.auto:auto-common:jar:0.3 from the shaded jar.
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing 

 with 

[INFO] Replacing original test artifact with shaded test artifact.
[INFO] Replacing 

 with 

[INFO] Dependency-reduced POM written at: 

[INFO] Dependency-reduced POM written at: 

[INFO] 
[INFO] --- maven-shade-plugin:2.4.1:shade (bundle-rest-without-repackaging) @ 
beam-sdks-java-core ---
[INFO] Including io.grpc:grpc-all:jar:0.13.1 in the shaded jar.
[INFO] Including io.grpc:grpc-auth:jar:0.13.1 in the shaded jar.
[INFO] Including io.grpc:grpc-netty:jar:0.13.1 in the shaded jar.
[INFO] Including io.netty:netty-codec-http2:jar:4.1.0.CR1 in the shaded jar.
[INFO] Including io.netty:netty-codec-http:jar:4.1.0.CR1 in the shaded jar.
[INFO] Including io.grpc:grpc-protobuf:jar:0.13.1 in the shaded jar.
[INFO] Including io.grpc:grpc-core:jar:0.13.1 in the shaded jar.
[INFO] Including io.grpc:grpc-okhttp:jar:0.13.1 in the shaded jar.
[INFO] Including com.squareup.okio:okio:jar:1.6.0 in the shaded jar.
[INFO] Including com.squareup.okhttp:okhttp:jar:2.5.0 in the shaded jar.
[INFO] Including io.grpc:grpc-protobuf-nano:jar:0.13.1 in the shaded jar.
[INFO] Including com.google.protobuf.nano:protobuf-javanano:jar:3.0.0-alpha-5 
in the shaded jar.
[INFO] Including io.grpc:grpc-stub:jar:0.13.1 in the shaded jar.
[INFO] Including com.google.auth:google-auth-library-oauth2-http:jar:0.4.0 in 
the shaded jar.
[INFO] Including com.google.auth:google-auth-library-credentials:jar:0.4.0 in 
the shaded jar.
[INFO] Including io.netty:netty-handler:jar:4.1.0.CR1 in the shaded jar.
[INFO] Including io.netty:netty-buffer:jar:4.1.0.CR1 in the shaded jar.
[INFO] Including io.netty:netty-common:jar:4.1.0.CR1 in the shaded jar.
[INFO] Including io.netty:netty-transport:jar:4.1.0.CR1 in the shaded jar.
[INFO] Including io.netty:netty-resolver:jar:4.1.0.CR1 in the shaded jar.
[INFO] Including io.netty:netty-codec:jar:4.1.0.CR1 in the shaded jar.
[INFO] Including com.google.api.grpc:grpc-pubsub-v1:jar:0.0.2 in the shaded jar.
[INFO] Including com.google.api.grpc:grpc-core-proto:jar:0.0.3 in the shaded 
jar.
[INFO] Including com.google.api-client:google-api-client:jar:1.22.0 in the 
shaded jar.
[INFO] Including 
com.google.apis:google-api-services-bigquery:jar:v2-rev295-1.22.0 in the shaded 
jar.
[INFO] Including com.google.apis:google-api-services-pubsub:jar:v1-rev10-1.22.0 
in the shaded jar.
[INFO] Including 
com.google.apis:google-api-services-storage:jar:v1-rev71-1.22.0 in the shaded 
jar.
[INFO] Including com.google.http-client:google-http-client:jar:1.22.0 in the 
shaded jar.
[INFO] Including org.apache.httpcomponents:httpclient:jar:4.0.1 in the shaded 
jar.
[INFO] Including org.apache.httpcomponents:httpcore:jar:4.0.1 in the shaded jar.
[INFO] Including commons-logging:commons-logging:jar:1.1.1 in the shaded jar.
[INFO] Including commons-codec:commons-codec:jar:1.3 in the shaded jar.
[INFO] Including com.google.http-client:google-http-client-jackson:jar:1.22.0 
in the shaded jar.
[INFO] Including com.google.http-client:google-http-client-jackson2:jar:1.22.0 
in the shaded jar.
[INFO] Including com.google.http-client:google-http-client-protobuf:jar:1.22.0 
in the shaded jar.
[INFO] Including com.google.oauth-client:google-oauth-client-java6:jar:1.22.0 
in the shaded jar.
[INFO] Including com.google.oauth-client:google-oauth-client:jar:1.22.0 in the 
shaded jar.
[INFO] Including com.google.cloud.bigdataoss:gcsio:jar:1.4.5 in the shaded jar.
[INFO] Including 

Jenkins build became unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #749

[Apache Jenkins Server]

See 

Build failed in Jenkins: beam_PostCommit_MavenVerify #879

[Apache Jenkins Server]

See 

--
[...truncated 3217 lines...]
[WARNING]   - javax.annotation.MatchesPattern
[WARNING]   - 25 more...
[WARNING] annotations-3.0.1.jar, jcip-annotations-1.0.jar define 4 overlapping 
classes: 
[WARNING]   - net.jcip.annotations.GuardedBy
[WARNING]   - net.jcip.annotations.NotThreadSafe
[WARNING]   - net.jcip.annotations.ThreadSafe
[WARNING]   - net.jcip.annotations.Immutable
[WARNING] grpc-all-0.13.1.jar, grpc-protobuf-0.13.1.jar define 4 overlapping 
classes: 
[WARNING]   - io.grpc.protobuf.ProtoUtils$2
[WARNING]   - io.grpc.protobuf.ProtoUtils
[WARNING]   - io.grpc.protobuf.ProtoUtils$1
[WARNING]   - io.grpc.protobuf.ProtoInputStream
[WARNING] grpc-all-0.13.1.jar, grpc-stub-0.13.1.jar define 30 overlapping 
classes: 
[WARNING]   - io.grpc.stub.ServerCalls$UnaryRequestMethod
[WARNING]   - io.grpc.stub.StreamObserver
[WARNING]   - io.grpc.stub.ClientCalls$CallToStreamObserverAdapter
[WARNING]   - io.grpc.stub.ServerCalls$EmptyServerCallListener
[WARNING]   - io.grpc.stub.MetadataUtils$2$1
[WARNING]   - io.grpc.stub.MetadataUtils$1$1
[WARNING]   - io.grpc.stub.MetadataUtils$2$1$1
[WARNING]   - io.grpc.stub.MetadataUtils
[WARNING]   - io.grpc.stub.ServerCalls$1
[WARNING]   - io.grpc.stub.ServerCalls$ClientStreamingMethod
[WARNING]   - 20 more...
[WARNING] grpc-core-0.13.1.jar, grpc-all-0.13.1.jar define 254 overlapping 
classes: 
[WARNING]   - io.grpc.ClientInterceptors$1
[WARNING]   - io.grpc.internal.DelayedClientTransport$1
[WARNING]   - io.grpc.internal.AbstractReferenceCounted
[WARNING]   - io.grpc.inprocess.InProcessTransport
[WARNING]   - io.grpc.internal.ServerImpl
[WARNING]   - io.grpc.internal.MessageFramer$OutputStreamAdapter
[WARNING]   - io.grpc.internal.ReadableBuffers$BufferInputStream
[WARNING]   - io.grpc.internal.SerializingExecutor$TaskRunner
[WARNING]   - io.grpc.internal.SharedResourceHolder$ScheduledExecutorFactory
[WARNING]   - io.grpc.internal.ContextRunnable
[WARNING]   - 244 more...
[WARNING] grpc-all-0.13.1.jar, grpc-netty-0.13.1.jar define 78 overlapping 
classes: 
[WARNING]   - io.grpc.netty.AbstractNettyHandler
[WARNING]   - io.grpc.netty.NettyClientTransport
[WARNING]   - io.grpc.netty.NettyClientStream$1
[WARNING]   - io.grpc.netty.SendResponseHeadersCommand
[WARNING]   - io.grpc.netty.NettyServer$2
[WARNING]   - io.grpc.netty.NettyClientHandler$4
[WARNING]   - io.grpc.netty.NettyClientTransport$3
[WARNING]   - io.grpc.netty.NettyClientHandler$FrameListener
[WARNING]   - io.grpc.netty.ProtocolNegotiators$1
[WARNING]   - io.grpc.netty.JettyTlsUtil
[WARNING]   - 68 more...
[WARNING] grpc-all-0.13.1.jar, grpc-auth-0.13.1.jar define 2 overlapping 
classes: 
[WARNING]   - io.grpc.auth.ClientAuthInterceptor$1
[WARNING]   - io.grpc.auth.ClientAuthInterceptor
[WARNING] maven-shade-plugin has detected that some class files are
[WARNING] present in two or more JARs. When this happens, only one
[WARNING] single version of the class is copied to the uber jar.
[WARNING] Usually this is not harmful and you can skip these warnings,
[WARNING] otherwise try to manually exclude artifacts based on
[WARNING] mvn dependency:tree -Ddetail=true and the above output.
[WARNING] See http://docs.codehaus.org/display/MAVENUSER/Shade+Plugin
[INFO] Replacing 

 with 

[INFO] 
[INFO] --- maven-dependency-plugin:2.10:analyze-only (default) @ 
beam-sdks-java-core ---
[INFO] No dependency problems found
[INFO] 
[INFO] --- apache-rat-plugin:0.11:check (default) @ beam-sdks-java-core ---
[INFO] 51 implicit excludes (use -debug for more details).
[INFO] Exclude: **/target/**/*
[INFO] Exclude: **/dependency-reduced-pom.xml
[INFO] Exclude: .github/**/*
[INFO] Exclude: **/*.iml
[INFO] Exclude: **/package-list
[INFO] Exclude: **/user.avsc
[INFO] Exclude: **/test/resources/**/*.txt
[INFO] Exclude: **/test/**/.placeholder
[INFO] Exclude: .repository/**/*
[INFO] 604 resources included (use -debug for more details)
[INFO] Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0 
approved: 604 licence.
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.17:check (default) @ beam-sdks-java-core 
---
[INFO] Downloading: 
https://repo.maven.apache.org/maven2/com/puppycrawl/tools/checkstyle/6.17/checkstyle-6.17.pom
[INFO] Downloaded: 
https://repo.maven.apache.org/maven2/com/puppycrawl/tools/checkstyle/6.17/checkstyle-6.17.pom
 (70 KB at 1647.9 KB/sec)
[INFO] Downloading: 
https://repo.maven.apache.org/maven2/antlr/antlr/2.7.7/antlr-2.7.7.pom
[INFO] Downloaded: 
https://repo.maven.apache.org/maven2/antlr/antlr/2.7.7/antlr-2.7.7.pom (632 B 
at 16.2 KB/sec)
[INFO] Downloading: 

Build failed in Jenkins: beam_PostCommit_MavenVerify » Apache Beam :: SDKs :: Java :: Core #879

[Apache Jenkins Server]

See 


--
[...truncated 1236 lines...]
[INFO] Excluding com.google.oauth-client:google-oauth-client-java6:jar:1.22.0 
from the shaded jar.
[INFO] Excluding com.google.oauth-client:google-oauth-client:jar:1.22.0 from 
the shaded jar.
[INFO] Excluding com.google.cloud.bigdataoss:gcsio:jar:1.4.5 from the shaded 
jar.
[INFO] Excluding com.google.api-client:google-api-client-java6:jar:1.22.0 from 
the shaded jar.
[INFO] Excluding com.google.api-client:google-api-client-jackson2:jar:1.22.0 
from the shaded jar.
[INFO] Excluding com.google.cloud.bigdataoss:util:jar:1.4.5 from the shaded jar.
[INFO] Including com.google.guava:guava:jar:19.0 in the shaded jar.
[INFO] Excluding com.google.protobuf:protobuf-java:jar:3.0.0-beta-1 from the 
shaded jar.
[INFO] Excluding com.google.code.findbugs:annotations:jar:3.0.1 from the shaded 
jar.
[INFO] Excluding net.jcip:jcip-annotations:jar:1.0 from the shaded jar.
[INFO] Excluding com.google.code.findbugs:jsr305:jar:3.0.1 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-core:jar:2.7.2 from the 
shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-annotations:jar:2.7.2 from 
the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-databind:jar:2.7.2 from the 
shaded jar.
[INFO] Excluding org.slf4j:slf4j-api:jar:1.7.14 from the shaded jar.
[INFO] Excluding org.apache.avro:avro:jar:1.7.7 from the shaded jar.
[INFO] Excluding org.codehaus.jackson:jackson-core-asl:jar:1.9.13 from the 
shaded jar.
[INFO] Excluding org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13 from the 
shaded jar.
[INFO] Excluding com.thoughtworks.paranamer:paranamer:jar:2.3 from the shaded 
jar.
[INFO] Excluding org.xerial.snappy:snappy-java:jar:1.1.2.1 from the shaded jar.
[INFO] Excluding org.apache.commons:commons-compress:jar:1.9 from the shaded 
jar.
[INFO] Excluding joda-time:joda-time:jar:2.4 from the shaded jar.
[INFO] Excluding org.codehaus.woodstox:stax2-api:jar:3.1.4 from the shaded jar.
[INFO] Excluding org.codehaus.woodstox:woodstox-core-asl:jar:4.4.1 from the 
shaded jar.
[INFO] Excluding org.tukaani:xz:jar:1.5 from the shaded jar.
[INFO] Excluding com.google.auto.service:auto-service:jar:1.0-rc2 from the 
shaded jar.
[INFO] Excluding com.google.auto:auto-common:jar:0.3 from the shaded jar.
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing 

 with 

[INFO] Replacing original test artifact with shaded test artifact.
[INFO] Replacing 

 with 

[INFO] Dependency-reduced POM written at: 

[INFO] Dependency-reduced POM written at: 

[INFO] 
[INFO] --- maven-shade-plugin:2.4.1:shade (bundle-rest-without-repackaging) @ 
beam-sdks-java-core ---
[INFO] Including io.grpc:grpc-all:jar:0.13.1 in the shaded jar.
[INFO] Including io.grpc:grpc-auth:jar:0.13.1 in the shaded jar.
[INFO] Including io.grpc:grpc-netty:jar:0.13.1 in the shaded jar.
[INFO] Including io.netty:netty-codec-http2:jar:4.1.0.CR1 in the shaded jar.
[INFO] Including io.netty:netty-codec-http:jar:4.1.0.CR1 in the shaded jar.
[INFO] Including io.grpc:grpc-protobuf:jar:0.13.1 in the shaded jar.
[INFO] Including io.grpc:grpc-core:jar:0.13.1 in the shaded jar.
[INFO] Including io.grpc:grpc-okhttp:jar:0.13.1 in the shaded jar.
[INFO] Including com.squareup.okio:okio:jar:1.6.0 in the shaded jar.
[INFO] Including com.squareup.okhttp:okhttp:jar:2.5.0 in the shaded jar.
[INFO] Including io.grpc:grpc-protobuf-nano:jar:0.13.1 in the shaded jar.
[INFO] Including com.google.protobuf.nano:protobuf-javanano:jar:3.0.0-alpha-5 
in the shaded jar.
[INFO] Including io.grpc:grpc-stub:jar:0.13.1 in the shaded jar.
[INFO] Including com.google.auth:google-auth-library-oauth2-http:jar:0.4.0 in 
the shaded jar.
[INFO] Including com.google.auth:google-auth-library-credentials:jar:0.4.0 in 
the shaded jar.
[INFO] Including io.netty:netty-handler:jar:4.1.0.CR1 in the shaded jar.
[INFO] Including io.netty:netty-buffer:jar:4.1.0.CR1 in the shaded jar.
[INFO] Including 

Build failed in Jenkins: beam_PostCommit_MavenVerify #878

[Apache Jenkins Server]

See 

Changes:

[lcwik] [BEAM-446] Improve IOChannelUtils.resolve to accept multiple paths

[lcwik] remove old method and fix javadoc

[lcwik] fix javadoc issue

--
[...truncated 865 lines...]
[INFO] Including org.codehaus.jackson:jackson-core-asl:jar:1.9.13 in the shaded 
jar.
[INFO] Including org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13 in the 
shaded jar.
[INFO] Including com.thoughtworks.paranamer:paranamer:jar:2.3 in the shaded jar.
[INFO] Including org.xerial.snappy:snappy-java:jar:1.1.2.1 in the shaded jar.
[INFO] Including org.apache.commons:commons-compress:jar:1.9 in the shaded jar.
[INFO] Including joda-time:joda-time:jar:2.4 in the shaded jar.
[INFO] Including org.codehaus.woodstox:stax2-api:jar:3.1.4 in the shaded jar.
[INFO] Including org.codehaus.woodstox:woodstox-core-asl:jar:4.4.1 in the 
shaded jar.
[INFO] Including org.tukaani:xz:jar:1.5 in the shaded jar.
[INFO] Including com.google.auto.service:auto-service:jar:1.0-rc2 in the shaded 
jar.
[INFO] Including com.google.auto:auto-common:jar:0.3 in the shaded jar.
[WARNING] annotations-3.0.1.jar, jsr305-3.0.1.jar define 35 overlapping 
classes: 
[WARNING]   - javax.annotation.RegEx
[WARNING]   - javax.annotation.concurrent.Immutable
[WARNING]   - javax.annotation.meta.TypeQualifierDefault
[WARNING]   - javax.annotation.meta.TypeQualifier
[WARNING]   - javax.annotation.Syntax
[WARNING]   - javax.annotation.Nonnull
[WARNING]   - javax.annotation.CheckReturnValue
[WARNING]   - javax.annotation.CheckForNull
[WARNING]   - javax.annotation.meta.TypeQualifierNickname
[WARNING]   - javax.annotation.MatchesPattern
[WARNING]   - 25 more...
[WARNING] grpc-all-0.13.1.jar, grpc-netty-0.13.1.jar define 78 overlapping 
classes: 
[WARNING]   - io.grpc.netty.AbstractNettyHandler
[WARNING]   - io.grpc.netty.NettyClientTransport
[WARNING]   - io.grpc.netty.NettyClientStream$1
[WARNING]   - io.grpc.netty.SendResponseHeadersCommand
[WARNING]   - io.grpc.netty.NettyServer$2
[WARNING]   - io.grpc.netty.NettyClientHandler$4
[WARNING]   - io.grpc.netty.NettyClientTransport$3
[WARNING]   - io.grpc.netty.NettyClientHandler$FrameListener
[WARNING]   - io.grpc.netty.ProtocolNegotiators$1
[WARNING]   - io.grpc.netty.JettyTlsUtil
[WARNING]   - 68 more...
[WARNING] grpc-all-0.13.1.jar, grpc-okhttp-0.13.1.jar define 75 overlapping 
classes: 
[WARNING]   - io.grpc.okhttp.OkHttpSettingsUtil
[WARNING]   - io.grpc.okhttp.NegotiationType
[WARNING]   - io.grpc.okhttp.AsyncFrameWriter$12
[WARNING]   - io.grpc.okhttp.OkHttpTlsUpgrader
[WARNING]   - io.grpc.okhttp.AsyncFrameWriter$WriteRunnable
[WARNING]   - io.grpc.okhttp.Utils
[WARNING]   - io.grpc.okhttp.OkHttpProtocolNegotiator$AndroidNegotiator
[WARNING]   - io.grpc.okhttp.OkHttpChannelBuilder$2
[WARNING]   - io.grpc.okhttp.internal.framed.Huffman$Node
[WARNING]   - io.grpc.okhttp.AsyncFrameWriter$7
[WARNING]   - 65 more...
[WARNING] grpc-all-0.13.1.jar, grpc-stub-0.13.1.jar define 30 overlapping 
classes: 
[WARNING]   - io.grpc.stub.ServerCalls$UnaryRequestMethod
[WARNING]   - io.grpc.stub.StreamObserver
[WARNING]   - io.grpc.stub.ClientCalls$CallToStreamObserverAdapter
[WARNING]   - io.grpc.stub.ServerCalls$EmptyServerCallListener
[WARNING]   - io.grpc.stub.MetadataUtils$2$1
[WARNING]   - io.grpc.stub.MetadataUtils$1$1
[WARNING]   - io.grpc.stub.MetadataUtils$2$1$1
[WARNING]   - io.grpc.stub.MetadataUtils
[WARNING]   - io.grpc.stub.ServerCalls$1
[WARNING]   - io.grpc.stub.ServerCalls$ClientStreamingMethod
[WARNING]   - 20 more...
[WARNING] grpc-all-0.13.1.jar, grpc-protobuf-0.13.1.jar define 4 overlapping 
classes: 
[WARNING]   - io.grpc.protobuf.ProtoUtils$2
[WARNING]   - io.grpc.protobuf.ProtoUtils
[WARNING]   - io.grpc.protobuf.ProtoUtils$1
[WARNING]   - io.grpc.protobuf.ProtoInputStream
[WARNING] grpc-all-0.13.1.jar, grpc-protobuf-nano-0.13.1.jar define 4 
overlapping classes: 
[WARNING]   - io.grpc.protobuf.nano.NanoProtoInputStream
[WARNING]   - io.grpc.protobuf.nano.NanoUtils
[WARNING]   - io.grpc.protobuf.nano.NanoUtils$1
[WARNING]   - io.grpc.protobuf.nano.MessageNanoFactory
[WARNING] jcip-annotations-1.0.jar, annotations-3.0.1.jar define 4 overlapping 
classes: 
[WARNING]   - net.jcip.annotations.GuardedBy
[WARNING]   - net.jcip.annotations.NotThreadSafe
[WARNING]   - net.jcip.annotations.ThreadSafe
[WARNING]   - net.jcip.annotations.Immutable
[WARNING] grpc-all-0.13.1.jar, grpc-core-0.13.1.jar define 254 overlapping 
classes: 
[WARNING]   - io.grpc.ClientInterceptors$1
[WARNING]   - io.grpc.internal.DelayedClientTransport$1
[WARNING]   - io.grpc.internal.AbstractReferenceCounted
[WARNING]   - io.grpc.inprocess.InProcessTransport
[WARNING]   - io.grpc.internal.ServerImpl
[WARNING]   - io.grpc.internal.MessageFramer$OutputStreamAdapter
[WARNING]   - io.grpc.internal.ReadableBuffers$BufferInputStream
[WARNING]   - io.grpc.internal.SerializingExecutor$TaskRunner
[WARNING]   - 

Build failed in Jenkins: beam_PostCommit_MavenVerify » Apache Beam :: SDKs :: Java :: Core #878

[Apache Jenkins Server]

See 


--
[...truncated 502 lines...]
[INFO] 
1 warning
[WARNING] Javadoc Warnings
[WARNING] 
:122:
 warning - Tag @link: can't find named in org.apache.beam.sdk.io.AvroIO.Write
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-shade-plugin:2.4.1:shade (bundle-and-repackage) @ 
beam-sdks-java-core ---
[INFO] Excluding io.grpc:grpc-all:jar:0.13.1 from the shaded jar.
[INFO] Excluding io.grpc:grpc-auth:jar:0.13.1 from the shaded jar.
[INFO] Excluding io.grpc:grpc-netty:jar:0.13.1 from the shaded jar.
[INFO] Excluding io.netty:netty-codec-http2:jar:4.1.0.CR1 from the shaded jar.
[INFO] Excluding io.netty:netty-codec-http:jar:4.1.0.CR1 from the shaded jar.
[INFO] Excluding io.grpc:grpc-protobuf:jar:0.13.1 from the shaded jar.
[INFO] Excluding io.grpc:grpc-core:jar:0.13.1 from the shaded jar.
[INFO] Excluding io.grpc:grpc-okhttp:jar:0.13.1 from the shaded jar.
[INFO] Excluding com.squareup.okio:okio:jar:1.6.0 from the shaded jar.
[INFO] Excluding com.squareup.okhttp:okhttp:jar:2.5.0 from the shaded jar.
[INFO] Excluding io.grpc:grpc-protobuf-nano:jar:0.13.1 from the shaded jar.
[INFO] Excluding com.google.protobuf.nano:protobuf-javanano:jar:3.0.0-alpha-5 
from the shaded jar.
[INFO] Excluding io.grpc:grpc-stub:jar:0.13.1 from the shaded jar.
[INFO] Excluding com.google.auth:google-auth-library-oauth2-http:jar:0.4.0 from 
the shaded jar.
[INFO] Excluding com.google.auth:google-auth-library-credentials:jar:0.4.0 from 
the shaded jar.
[INFO] Excluding io.netty:netty-handler:jar:4.1.0.CR1 from the shaded jar.
[INFO] Excluding io.netty:netty-buffer:jar:4.1.0.CR1 from the shaded jar.
[INFO] Excluding io.netty:netty-common:jar:4.1.0.CR1 from the shaded jar.
[INFO] Excluding io.netty:netty-transport:jar:4.1.0.CR1 from the shaded jar.
[INFO] Excluding io.netty:netty-resolver:jar:4.1.0.CR1 from the shaded jar.
[INFO] Excluding io.netty:netty-codec:jar:4.1.0.CR1 from the shaded jar.
[INFO] Excluding com.google.api.grpc:grpc-pubsub-v1:jar:0.0.2 from the shaded 
jar.
[INFO] Excluding com.google.api.grpc:grpc-core-proto:jar:0.0.3 from the shaded 
jar.
[INFO] Excluding com.google.api-client:google-api-client:jar:1.22.0 from the 
shaded jar.
[INFO] Excluding 
com.google.apis:google-api-services-bigquery:jar:v2-rev295-1.22.0 from the 
shaded jar.
[INFO] Excluding com.google.apis:google-api-services-pubsub:jar:v1-rev10-1.22.0 
from the shaded jar.
[INFO] Excluding 
com.google.apis:google-api-services-storage:jar:v1-rev71-1.22.0 from the shaded 
jar.
[INFO] Excluding com.google.http-client:google-http-client:jar:1.22.0 from the 
shaded jar.
[INFO] Excluding org.apache.httpcomponents:httpclient:jar:4.0.1 from the shaded 
jar.
[INFO] Excluding org.apache.httpcomponents:httpcore:jar:4.0.1 from the shaded 
jar.
[INFO] Excluding commons-logging:commons-logging:jar:1.1.1 from the shaded jar.
[INFO] Excluding commons-codec:commons-codec:jar:1.3 from the shaded jar.
[INFO] Excluding com.google.http-client:google-http-client-jackson:jar:1.22.0 
from the shaded jar.
[INFO] Excluding com.google.http-client:google-http-client-jackson2:jar:1.22.0 
from the shaded jar.
[INFO] Excluding com.google.http-client:google-http-client-protobuf:jar:1.22.0 
from the shaded jar.
[INFO] Excluding com.google.oauth-client:google-oauth-client-java6:jar:1.22.0 
from the shaded jar.
[INFO] Excluding com.google.oauth-client:google-oauth-client:jar:1.22.0 from 
the shaded jar.
[INFO] Excluding com.google.cloud.bigdataoss:gcsio:jar:1.4.5 from the shaded 
jar.
[INFO] Excluding com.google.api-client:google-api-client-java6:jar:1.22.0 from 
the shaded jar.
[INFO] Excluding com.google.api-client:google-api-client-jackson2:jar:1.22.0 
from the shaded jar.
[INFO] Excluding com.google.cloud.bigdataoss:util:jar:1.4.5 from the shaded jar.
[INFO] Including com.google.guava:guava:jar:19.0 in the shaded jar.
[INFO] Excluding com.google.protobuf:protobuf-java:jar:3.0.0-beta-1 from the 
shaded jar.
[INFO] Excluding com.google.code.findbugs:annotations:jar:3.0.1 from the shaded 
jar.
[INFO] Excluding net.jcip:jcip-annotations:jar:1.0 from the shaded jar.
[INFO] Excluding com.google.code.findbugs:jsr305:jar:3.0.1 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-core:jar:2.7.2 from the 
shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-annotations:jar:2.7.2 from 
the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-databind:jar:2.7.2 from the 
shaded jar.
[INFO] Excluding org.slf4j:slf4j-api:jar:1.7.14 from the shaded jar.
[INFO] Excluding org.apache.avro:avro:jar:1.7.7 from the shaded 

[jira] [Commented] (BEAM-446) Improve IOChannelUtils.resolve() to accept multiple paths at once

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15375632#comment-15375632
 ] 

ASF GitHub Bot commented on BEAM-446:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/646


> Improve IOChannelUtils.resolve() to accept multiple paths at once
> -
>
> Key: BEAM-446
> URL: https://issues.apache.org/jira/browse/BEAM-446
> Project: Beam
>  Issue Type: Improvement
>Reporter: Mark Liu
>Assignee: Mark Liu
>Priority: Minor
> Fix For: 0.2.0-incubating
>
>
> Currently, IOChannelUtils.resolve() method can only resolve one path against 
> base path. 
> It's useful to have another method with arguments that includes one base path 
> and multiple others. The return string will be a directory that start with 
> base path and append rests which are separated by file separator.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (BEAM-446) Improve IOChannelUtils.resolve() to accept multiple paths at once

[Luke Cwik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Luke Cwik resolved BEAM-446.

   Resolution: Fixed
Fix Version/s: 0.2.0-incubating

> Improve IOChannelUtils.resolve() to accept multiple paths at once
> -
>
> Key: BEAM-446
> URL: https://issues.apache.org/jira/browse/BEAM-446
> Project: Beam
>  Issue Type: Improvement
>Reporter: Mark Liu
>Assignee: Mark Liu
>Priority: Minor
> Fix For: 0.2.0-incubating
>
>
> Currently, IOChannelUtils.resolve() method can only resolve one path against 
> base path. 
> It's useful to have another method with arguments that includes one base path 
> and multiple others. The return string will be a directory that start with 
> base path and append rests which are separated by file separator.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[4/4] incubator-beam git commit: [BEAM-446] Improve IOChannelUtils.resolve to accept multiple paths

[lcwik]

[BEAM-446] Improve IOChannelUtils.resolve to accept multiple paths

This closes #646


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/2596d136
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/2596d136
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/2596d136

Branch: refs/heads/master
Commit: 2596d136da61e112b587758b51f0f97ee8025c8a
Parents: 8f5ce28 1acfe33
Author: Luke Cwik 
Authored: Wed Jul 13 12:42:25 2016 -0700
Committer: Luke Cwik 
Committed: Wed Jul 13 12:42:25 2016 -0700

--
 .../apache/beam/sdk/util/IOChannelUtils.java| 25 +++-
 .../beam/sdk/util/IOChannelUtilsTest.java   | 11 -
 2 files changed, 24 insertions(+), 12 deletions(-)
--

[1/4] incubator-beam git commit: fix javadoc issue

[lcwik]

Repository: incubator-beam
Updated Branches:
  refs/heads/master 8f5ce28dc -> 2596d136d


fix javadoc issue


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/1acfe33c
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/1acfe33c
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/1acfe33c

Branch: refs/heads/master
Commit: 1acfe33c95b0c63c34421234bb28a241e4ec2d9a
Parents: 7ff15aa
Author: Mark Liu 
Authored: Wed Jul 13 12:08:19 2016 -0700
Committer: Luke Cwik 
Committed: Wed Jul 13 12:42:07 2016 -0700

--
 .../src/main/java/org/apache/beam/sdk/util/IOChannelUtils.java | 2 +-
 .../test/java/org/apache/beam/sdk/util/IOChannelUtilsTest.java | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/1acfe33c/sdks/java/core/src/main/java/org/apache/beam/sdk/util/IOChannelUtils.java
--
diff --git 
a/sdks/java/core/src/main/java/org/apache/beam/sdk/util/IOChannelUtils.java 
b/sdks/java/core/src/main/java/org/apache/beam/sdk/util/IOChannelUtils.java
index 2994bf3..efb0ff6 100644
--- a/sdks/java/core/src/main/java/org/apache/beam/sdk/util/IOChannelUtils.java
+++ b/sdks/java/core/src/main/java/org/apache/beam/sdk/util/IOChannelUtils.java
@@ -190,7 +190,7 @@ public class IOChannelUtils {
   /**
* Resolve multiple {@code others} against the {@code path} sequentially.
*
-   * Empty paths in {@code others} are ignored. If {@code others} contains one 
or more
+   * Empty paths in {@code others} are ignored. If {@code others} contains 
one or more
* absolute paths, then this method returns a path that starts with the last 
absolute path
* in {@code others} joined with the remaining paths. Resolution of paths is 
highly
* implementation dependent and therefore unspecified.

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/1acfe33c/sdks/java/core/src/test/java/org/apache/beam/sdk/util/IOChannelUtilsTest.java
--
diff --git 
a/sdks/java/core/src/test/java/org/apache/beam/sdk/util/IOChannelUtilsTest.java 
b/sdks/java/core/src/test/java/org/apache/beam/sdk/util/IOChannelUtilsTest.java
index eb35cbe..8db2f75 100644
--- 
a/sdks/java/core/src/test/java/org/apache/beam/sdk/util/IOChannelUtilsTest.java
+++ 
b/sdks/java/core/src/test/java/org/apache/beam/sdk/util/IOChannelUtilsTest.java
@@ -95,10 +95,10 @@ public class IOChannelUtilsTest {
   }
 
   @Test
-  public void testResolveMultiplePaths() throws IOException {
+  public void testResolveMultiplePaths() throws Exception {
 String expected =
-
tmpFolder.getRoot().toPath().resolve("aa").resolve("bb").resolve("cc").toString();
+
tmpFolder.getRoot().toPath().resolve("aa").resolve("bb").resolve("cc").toString();
 assertEquals(expected,
-IOChannelUtils.resolve(tmpFolder.getRoot().getPath(), "aa", "bb", 
"cc"));
+IOChannelUtils.resolve(tmpFolder.getRoot().getPath(), "aa", "bb", 
"cc"));
   }
 }

[GitHub] incubator-beam pull request #646: [BEAM-446] Improve IOChannelUtils.resolve ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/646


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[2/4] incubator-beam git commit: remove old method and fix javadoc

[lcwik]

remove old method and fix javadoc


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/7ff15aa9
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/7ff15aa9
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/7ff15aa9

Branch: refs/heads/master
Commit: 7ff15aa90876996cf35a5bd5d21a6249f3e3
Parents: 72c2c4a
Author: Mark Liu 
Authored: Wed Jul 13 12:00:48 2016 -0700
Committer: Luke Cwik 
Committed: Wed Jul 13 12:42:07 2016 -0700

--
 .../apache/beam/sdk/util/IOChannelUtils.java| 28 
 1 file changed, 5 insertions(+), 23 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/7ff15aa9/sdks/java/core/src/main/java/org/apache/beam/sdk/util/IOChannelUtils.java
--
diff --git 
a/sdks/java/core/src/main/java/org/apache/beam/sdk/util/IOChannelUtils.java 
b/sdks/java/core/src/main/java/org/apache/beam/sdk/util/IOChannelUtils.java
index 1042b61..2994bf3 100644
--- a/sdks/java/core/src/main/java/org/apache/beam/sdk/util/IOChannelUtils.java
+++ b/sdks/java/core/src/main/java/org/apache/beam/sdk/util/IOChannelUtils.java
@@ -188,30 +188,12 @@ public class IOChannelUtils {
   }
 
   /**
-   * Resolve the given {@code other} against the {@code path}.
+   * Resolve multiple {@code others} against the {@code path} sequentially.
*
-   * If the {@code other} parameter is an absolute path then this method 
trivially returns
-   * other. If {@code other} is an empty path then this method trivially 
returns the given
-   * {@code path}. Otherwise this method considers the given {@code path} to 
be a directory and
-   * resolves the {@code other} path against this path. In the simplest case, 
the {@code other}
-   * path does not have a root component, in which case this method joins the 
{@code other} path
-   * to the given {@code path} and returns a resulting path that ends with the 
{@code other} path.
-   * Where the {@code other} path has a root component then resolution is 
highly implementation
-   * dependent and therefore unspecified.
-   */
-  public static String resolve(String path, String other) throws IOException {
-return getFactory(path).resolve(path, other);
-  }
-
-  /**
-   * Resolve given multiple {@code others} against the {@code path}.
-   *
-   * If the {@code others} parameter contains empty paths then empty paths 
are ignored. If the
-   * {@code others} contains one or more absolute paths then this method 
returns a resulting path
-   * that starts with the last absolute path in {@code others} joining with 
rest paths.
-   * Otherwise this method considers the given {@code path} to be a base 
directory and resolves
-   * {@code others} paths against this path sequentially. Where the {@code 
others} paths have
-   * root components then resolution is highly implementation dependent and 
therefore unspecified.
+   * Empty paths in {@code others} are ignored. If {@code others} contains one 
or more
+   * absolute paths, then this method returns a path that starts with the last 
absolute path
+   * in {@code others} joined with the remaining paths. Resolution of paths is 
highly
+   * implementation dependent and therefore unspecified.
*/
   public static String resolve(String path, String... others) throws 
IOException {
 IOChannelFactory ioFactory = getFactory(path);

[3/4] incubator-beam git commit: [BEAM-446] Improve IOChannelUtils.resolve to accept multiple paths

[lcwik]

[BEAM-446] Improve IOChannelUtils.resolve to accept multiple paths


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/72c2c4a3
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/72c2c4a3
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/72c2c4a3

Branch: refs/heads/master
Commit: 72c2c4a3dbee08e636da7f8882432d6b35b1ae25
Parents: 8f5ce28
Author: Mark Liu 
Authored: Wed Jul 13 11:08:18 2016 -0700
Committer: Luke Cwik 
Committed: Wed Jul 13 12:42:07 2016 -0700

--
 .../apache/beam/sdk/util/IOChannelUtils.java| 21 
 .../beam/sdk/util/IOChannelUtilsTest.java   | 11 +-
 2 files changed, 31 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/72c2c4a3/sdks/java/core/src/main/java/org/apache/beam/sdk/util/IOChannelUtils.java
--
diff --git 
a/sdks/java/core/src/main/java/org/apache/beam/sdk/util/IOChannelUtils.java 
b/sdks/java/core/src/main/java/org/apache/beam/sdk/util/IOChannelUtils.java
index 05443fb..1042b61 100644
--- a/sdks/java/core/src/main/java/org/apache/beam/sdk/util/IOChannelUtils.java
+++ b/sdks/java/core/src/main/java/org/apache/beam/sdk/util/IOChannelUtils.java
@@ -202,4 +202,25 @@ public class IOChannelUtils {
   public static String resolve(String path, String other) throws IOException {
 return getFactory(path).resolve(path, other);
   }
+
+  /**
+   * Resolve given multiple {@code others} against the {@code path}.
+   *
+   * If the {@code others} parameter contains empty paths then empty paths 
are ignored. If the
+   * {@code others} contains one or more absolute paths then this method 
returns a resulting path
+   * that starts with the last absolute path in {@code others} joining with 
rest paths.
+   * Otherwise this method considers the given {@code path} to be a base 
directory and resolves
+   * {@code others} paths against this path sequentially. Where the {@code 
others} paths have
+   * root components then resolution is highly implementation dependent and 
therefore unspecified.
+   */
+  public static String resolve(String path, String... others) throws 
IOException {
+IOChannelFactory ioFactory = getFactory(path);
+String fullPath = path;
+
+for (String other : others) {
+  fullPath = ioFactory.resolve(fullPath, other);
+}
+
+return fullPath;
+  }
 }

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/72c2c4a3/sdks/java/core/src/test/java/org/apache/beam/sdk/util/IOChannelUtilsTest.java
--
diff --git 
a/sdks/java/core/src/test/java/org/apache/beam/sdk/util/IOChannelUtilsTest.java 
b/sdks/java/core/src/test/java/org/apache/beam/sdk/util/IOChannelUtilsTest.java
index c79f1a1..eb35cbe 100644
--- 
a/sdks/java/core/src/test/java/org/apache/beam/sdk/util/IOChannelUtilsTest.java
+++ 
b/sdks/java/core/src/test/java/org/apache/beam/sdk/util/IOChannelUtilsTest.java
@@ -30,6 +30,7 @@ import org.junit.runner.RunWith;
 import org.junit.runners.JUnit4;
 
 import java.io.File;
+import java.io.IOException;
 import java.nio.charset.StandardCharsets;
 
 /**
@@ -88,8 +89,16 @@ public class IOChannelUtilsTest {
   }
 
   @Test
-  public void testResolve() throws Exception {
+  public void testResolveSinglePath() throws Exception {
 String expected = tmpFolder.getRoot().toPath().resolve("aa").toString();
 assertEquals(expected, 
IOChannelUtils.resolve(tmpFolder.getRoot().toString(), "aa"));
   }
+
+  @Test
+  public void testResolveMultiplePaths() throws IOException {
+String expected =
+
tmpFolder.getRoot().toPath().resolve("aa").resolve("bb").resolve("cc").toString();
+assertEquals(expected,
+IOChannelUtils.resolve(tmpFolder.getRoot().getPath(), "aa", "bb", 
"cc"));
+  }
 }

Jenkins build is back to stable : beam_PostCommit_RunnableOnService_GoogleCloudDataflow #748

[Apache Jenkins Server]

See 

Jenkins build is back to stable : beam_PostCommit_MavenVerify #876

[Apache Jenkins Server]

See 

Jenkins build is back to stable : beam_PostCommit_MavenVerify » Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform #876

[Apache Jenkins Server]

See 

[jira] [Commented] (BEAM-285) Make MinLongFn and MaxLongFn mimic SumLongFn and use BinaryCombineLongFn

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15375485#comment-15375485
 ] 

ASF GitHub Bot commented on BEAM-285:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/552


> Make MinLongFn and MaxLongFn mimic SumLongFn and use BinaryCombineLongFn
> 
>
> Key: BEAM-285
> URL: https://issues.apache.org/jira/browse/BEAM-285
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Reporter: Mark Shields
>Assignee: Pei He
>
> Ditto for the other 'optimized accumulator' combiner functions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-beam pull request #552: [BEAM-285] Avoid boxing and unboxing in tr...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/552


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[1/2] incubator-beam git commit: [BEAM-285] Avoid boxing and unboxing in transforms Min and Max

[dhalperi]

Repository: incubator-beam
Updated Branches:
  refs/heads/master 1a7cd4112 -> 8f5ce28dc


[BEAM-285] Avoid boxing and unboxing in transforms Min and Max


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/21a5b44c
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/21a5b44c
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/21a5b44c

Branch: refs/heads/master
Commit: 21a5b44c3b541ba6c89df5649afe00412df73d10
Parents: 1a7cd41
Author: Pei He 
Authored: Mon Jul 11 20:31:37 2016 -0700
Committer: Dan Halperin 
Committed: Wed Jul 13 11:20:18 2016 -0700

--
 examples/pom.xml|  2 +-
 .../org/apache/beam/sdk/transforms/Max.java | 42 --
 .../org/apache/beam/sdk/transforms/Min.java | 45 ++--
 .../beam/sdk/util/state/StateTagTest.java   |  5 +--
 4 files changed, 66 insertions(+), 28 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/21a5b44c/examples/pom.xml
--
diff --git a/examples/pom.xml b/examples/pom.xml
index dbf89e4..ee12b69 100644
--- a/examples/pom.xml
+++ b/examples/pom.xml
@@ -48,4 +48,4 @@
   
   
 
-
\ No newline at end of file
+

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/21a5b44c/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Max.java
--
diff --git 
a/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Max.java 
b/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Max.java
index c05bd17..52617b6 100644
--- a/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Max.java
+++ b/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Max.java
@@ -218,10 +218,16 @@ public class Max {
* A {@code CombineFn} that computes the maximum of a collection of {@code 
Integer}s, useful as an
* argument to {@link Combine#globally} or {@link Combine#perKey}.
*/
-  public static class MaxIntegerFn extends MaxFn implements
-  CounterProvider {
-public MaxIntegerFn() {
-  super(Integer.MIN_VALUE, new Top.Largest());
+  public static class MaxIntegerFn extends Combine.BinaryCombineIntegerFn
+  implements CounterProvider {
+@Override
+public int apply(int left, int right) {
+  return left >= right ? left : right;
+}
+
+@Override
+public int identity() {
+  return Integer.MIN_VALUE;
 }
 
 @Override
@@ -234,10 +240,16 @@ public class Max {
* A {@code CombineFn} that computes the maximum of a collection of {@code 
Long}s, useful as an
* argument to {@link Combine#globally} or {@link Combine#perKey}.
*/
-  public static class MaxLongFn extends MaxFn implements
-  CounterProvider {
-public MaxLongFn() {
-  super(Long.MIN_VALUE, new Top.Largest());
+  public static class MaxLongFn extends Combine.BinaryCombineLongFn
+  implements CounterProvider {
+@Override
+public long apply(long left, long right) {
+  return left >= right ? left : right;
+}
+
+@Override
+public long identity() {
+  return Long.MIN_VALUE;
 }
 
 @Override
@@ -250,10 +262,16 @@ public class Max {
* A {@code CombineFn} that computes the maximum of a collection of {@code 
Double}s, useful as an
* argument to {@link Combine#globally} or {@link Combine#perKey}.
*/
-  public static class MaxDoubleFn extends MaxFn implements
-  CounterProvider {
-public MaxDoubleFn() {
-  super(Double.NEGATIVE_INFINITY, new Top.Largest());
+  public static class MaxDoubleFn extends Combine.BinaryCombineDoubleFn
+  implements CounterProvider {
+@Override
+public double apply(double left, double right) {
+  return left >= right ? left : right;
+}
+
+@Override
+public double identity() {
+  return Double.NEGATIVE_INFINITY;
 }
 
 @Override

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/21a5b44c/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Min.java
--
diff --git 
a/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Min.java 
b/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Min.java
index 7a6a8a2..3159134 100644
--- a/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Min.java
+++ b/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Min.java
@@ -218,10 +218,17 @@ public class Min {
* A {@code CombineFn} that computes the minimum of a collection of {@code 
Integer}s, useful as an
* argument to {@link Combine#globally} or {@link Combine#perKey}.
*/
-  public static class MinIntegerFn 

[2/2] incubator-beam git commit: Closes #552

[dhalperi]

Closes #552


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/8f5ce28d
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/8f5ce28d
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/8f5ce28d

Branch: refs/heads/master
Commit: 8f5ce28dcb82a9f96185f783aaf5dca332475b78
Parents: 1a7cd41 21a5b44
Author: Dan Halperin 
Authored: Wed Jul 13 11:20:19 2016 -0700
Committer: Dan Halperin 
Committed: Wed Jul 13 11:20:19 2016 -0700

--
 examples/pom.xml|  2 +-
 .../org/apache/beam/sdk/transforms/Max.java | 42 --
 .../org/apache/beam/sdk/transforms/Min.java | 45 ++--
 .../beam/sdk/util/state/StateTagTest.java   |  5 +--
 4 files changed, 66 insertions(+), 28 deletions(-)
--

[jira] [Commented] (BEAM-446) Improve IOChannelUtils.resolve() to accept multiple paths at once

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15375472#comment-15375472
 ] 

ASF GitHub Bot commented on BEAM-446:
-

GitHub user markflyhigh opened a pull request:

https://github.com/apache/incubator-beam/pull/646

[BEAM-446] Improve IOChannelUtils.resolve to accept multiple paths

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [x] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [x] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [x] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/markflyhigh/incubator-beam 
improve-iochannelutils-resolve

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/646.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #646


commit db38493615b47c11ca5831f58c9d24140d5ef829
Author: Mark Liu 
Date:   2016-07-13T18:08:18Z

[BEAM-446] Improve IOChannelUtils.resolve to accept multiple paths




> Improve IOChannelUtils.resolve() to accept multiple paths at once
> -
>
> Key: BEAM-446
> URL: https://issues.apache.org/jira/browse/BEAM-446
> Project: Beam
>  Issue Type: Improvement
>Reporter: Mark Liu
>Assignee: Mark Liu
>Priority: Minor
>
> Currently, IOChannelUtils.resolve() method can only resolve one path against 
> base path. 
> It's useful to have another method with arguments that includes one base path 
> and multiple others. The return string will be a directory that start with 
> base path and append rests which are separated by file separator.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-beam pull request #646: [BEAM-446] Improve IOChannelUtils.resolve ...

[markflyhigh]

GitHub user markflyhigh opened a pull request:

https://github.com/apache/incubator-beam/pull/646

[BEAM-446] Improve IOChannelUtils.resolve to accept multiple paths

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [x] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [x] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [x] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/markflyhigh/incubator-beam 
improve-iochannelutils-resolve

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/646.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #646


commit db38493615b47c11ca5831f58c9d24140d5ef829
Author: Mark Liu 
Date:   2016-07-13T18:08:18Z

[BEAM-446] Improve IOChannelUtils.resolve to accept multiple paths




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-beam pull request #645: Temporarily reverting pickler changes

[aaltay]

GitHub user aaltay opened a pull request:

https://github.com/apache/incubator-beam/pull/645

Temporarily reverting pickler changes

Changes from @4e2d8ab is casing infinite recursion when some tests are 
directly invoked outside the testing framework.

Added one such test to tox tests for testing failure case (directly by
main function).

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/aaltay/incubator-beam pickle

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/645.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #645


commit 997764750a5d5ef1a501643fff8529f913c74ef8
Author: Ahmet Altay 
Date:   2016-07-13T17:18:03Z

Temporarily reverting pickler changes (@4e2d8ab). It is causing infinite
recursion when some tests are directly invoked outside the testing
framework.

Added one such test to tox tests for testing failing case (directly by
main function).




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (BEAM-448) Print Properties on validation failures in PipelineOptionsValidator

[Thomas Groh (JIRA)]

Thomas Groh created BEAM-448:


 Summary: Print Properties on validation failures in 
PipelineOptionsValidator
 Key: BEAM-448
 URL: https://issues.apache.org/jira/browse/BEAM-448
 Project: Beam
  Issue Type: Improvement
  Components: sdk-java-core
Reporter: Thomas Groh
Assignee: Davor Bonaci
Priority: Minor


If Pipeline Validation fails in the pipeline options Validator, the methods 
that failed validation are printed. Instead, the property names (passed on the 
command line) should be printed for consistency with the PipelineOptionsFactory.

Currently methods are printed at 
https://github.com/apache/incubator-beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/options/PipelineOptionsValidator.java#L72

PipelineOptionsReflector currently extracts the property -> method mapping
https://github.com/apache/incubator-beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/options/PipelineOptionsReflector.java#L95



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (BEAM-447) Stop referring to types with Bound/Unbound

[Thomas Groh (JIRA)]

Thomas Groh created BEAM-447:


 Summary: Stop referring to types with Bound/Unbound
 Key: BEAM-447
 URL: https://issues.apache.org/jira/browse/BEAM-447
 Project: Beam
  Issue Type: Improvement
  Components: sdk-java-core
Reporter: Thomas Groh
Assignee: Davor Bonaci


Bounded and Unbounded are used to refer to PCollections, and the overlap is 
confusing.  These classes should be renamed to be both more specific (e.g. 
ParDo.LackingDoFnSingleOutput, ParDo.SingleOutput, Window.AssignWindows) which 
remove the overlap.

examples:
https://github.com/apache/incubator-beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/ParDo.java#L658
https://github.com/apache/incubator-beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/ParDo.java#L868



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (BEAM-446) Improve IOChannelUtils.resolve() to accept multiple paths at once

[Luke Cwik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Luke Cwik updated BEAM-446:
---
Assignee: Mark Liu

> Improve IOChannelUtils.resolve() to accept multiple paths at once
> -
>
> Key: BEAM-446
> URL: https://issues.apache.org/jira/browse/BEAM-446
> Project: Beam
>  Issue Type: Improvement
>Reporter: Mark Liu
>Assignee: Mark Liu
>Priority: Minor
>
> Currently, IOChannelUtils.resolve() method can only resolve one path against 
> base path. 
> It's useful to have another method with arguments that includes one base path 
> and multiple others. The return string will be a directory that start with 
> base path and append rests which are separated by file separator.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build became unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #747

[Apache Jenkins Server]

See 

Jenkins build became unstable: beam_PostCommit_MavenVerify #875

[Apache Jenkins Server]

See 

Jenkins build became unstable: beam_PostCommit_MavenVerify » Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform #875

[Apache Jenkins Server]

See 

[GitHub] incubator-beam pull request #630: Allow custom timestamp/watermark function ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/630


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[1/2] incubator-beam git commit: Allow for custom timestamp/watermark function in FlinkPipelineRunner

[aljoscha]

Repository: incubator-beam
Updated Branches:
  refs/heads/master ee1a3bcfb -> 1a7cd4112


Allow for custom timestamp/watermark function in FlinkPipelineRunner

Added new "of" signature and constructor for UnboundedFlinkSource to
allow event timestamping


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/9000d95d
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/9000d95d
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/9000d95d

Branch: refs/heads/master
Commit: 9000d95d2b34ab45f799aedb140710986ff19452
Parents: ee1a3bc
Author: David Desberg 
Authored: Mon Jul 11 12:24:18 2016 -0700
Committer: Aljoscha Krettek 
Committed: Wed Jul 13 14:15:54 2016 +0200

--
 .../FlinkStreamingTransformTranslators.java | 10 ++---
 .../streaming/io/UnboundedFlinkSource.java  | 23 
 2 files changed, 30 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/9000d95d/runners/flink/runner/src/main/java/org/apache/beam/runners/flink/translation/FlinkStreamingTransformTranslators.java
--
diff --git 
a/runners/flink/runner/src/main/java/org/apache/beam/runners/flink/translation/FlinkStreamingTransformTranslators.java
 
b/runners/flink/runner/src/main/java/org/apache/beam/runners/flink/translation/FlinkStreamingTransformTranslators.java
index 5d04068..fa6b387 100644
--- 
a/runners/flink/runner/src/main/java/org/apache/beam/runners/flink/translation/FlinkStreamingTransformTranslators.java
+++ 
b/runners/flink/runner/src/main/java/org/apache/beam/runners/flink/translation/FlinkStreamingTransformTranslators.java
@@ -70,7 +70,9 @@ import org.apache.flink.streaming.api.datastream.DataStream;
 import org.apache.flink.streaming.api.datastream.DataStreamSink;
 import org.apache.flink.streaming.api.datastream.KeyedStream;
 import org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator;
+import org.apache.flink.streaming.api.functions.AssignerWithPeriodicWatermarks;
 import org.apache.flink.streaming.api.functions.IngestionTimeExtractor;
+import org.apache.flink.streaming.api.watermark.Watermark;
 import org.apache.flink.util.Collector;
 import org.joda.time.Instant;
 import org.slf4j.Logger;
@@ -252,6 +254,8 @@ public class FlinkStreamingTransformTranslators {
   if (transform.getSource().getClass().equals(UnboundedFlinkSource.class)) 
{
 @SuppressWarnings("unchecked")
 UnboundedFlinkSource flinkSourceFunction = 
(UnboundedFlinkSource) transform.getSource();
+final AssignerWithPeriodicWatermarks flinkAssigner = 
flinkSourceFunction.getFlinkTimestampAssigner();
+
 DataStream flinkSource = context.getExecutionEnvironment()
 .addSource(flinkSourceFunction.getFlinkSource());
 
@@ -260,17 +264,17 @@ public class FlinkStreamingTransformTranslators {
   context.getExecutionEnvironment().getConfig()));
 
 source = flinkSource
+.assignTimestampsAndWatermarks(flinkAssigner)
 .flatMap(new FlatMapFunction() {
   @Override
   public void flatMap(T s, Collector collector) 
throws Exception {
 collector.collect(
 WindowedValue.of(
 s,
-Instant.now(),
+new Instant(flinkAssigner.extractTimestamp(s, -1)),
 GlobalWindow.INSTANCE,
 PaneInfo.NO_FIRING));
-  }
-}).assignTimestampsAndWatermarks(new 
IngestionTimeExtractor());
+  }});
   } else {
 try {
   transform.getSource();

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/9000d95d/runners/flink/runner/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/UnboundedFlinkSource.java
--
diff --git 
a/runners/flink/runner/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/UnboundedFlinkSource.java
 
b/runners/flink/runner/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/UnboundedFlinkSource.java
index 94b73ce..716ca30 100644
--- 
a/runners/flink/runner/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/UnboundedFlinkSource.java
+++ 
b/runners/flink/runner/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/UnboundedFlinkSource.java
@@ -23,6 +23,8 @@ import org.apache.beam.sdk.coders.Coder;
 import org.apache.beam.sdk.io.UnboundedSource;
 import org.apache.beam.sdk.options.PipelineOptions;
 

Jenkins build is back to stable : beam_PostCommit_MavenVerify #873

[Apache Jenkins Server]

See 

Jenkins build became unstable: beam_PostCommit_MavenVerify » Apache Beam :: Examples :: Java #872

[Apache Jenkins Server]

See