[jira] (BEAM-1360) Reference to the obsolete python-sdk branch in the documentation.

2017-01-31 Thread Prabeesh K (JIRA)
Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Prabeesh K created an issue 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 Beam /  BEAM-1360 
 
 
 
  Reference to the obsolete python-sdk branch in the documentation.  
 
 
 
 
 
 
 
 
 

Issue Type:
 
  Improvement 
 
 
 

Assignee:
 
 Davor Bonaci 
 
 
 

Components:
 

 project-management 
 
 
 

Created:
 

 31/Jan/17 08:30 
 
 
 

Priority:
 
  Major 
 
 
 

Reporter:
 
 Prabeesh K 
 
 
 
 
 
 
 
 
 
 
Following documents contains reference to obsolete python-sdk 
 

README
 

PYTHON SDK README
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

[jira] (BEAM-1354) TextIO should comply with PTransform style guide

2017-01-31 Thread Aviem Zur (JIRA)
Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Aviem Zur updated an issue 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 Beam /  BEAM-1354 
 
 
 
  TextIO should comply with PTransform style guide  
 
 
 
 
 
 
 
 
 

Change By:
 
 Aviem Zur 
 
 
 
 
 
 
 
 
 
 * TextIO.Read,Write.Bound,Unbound - Bound,Unbound are banned names. Instead, TextIO should make the user specify the type parameters explicitly, and have simply TextIO.Read and TextIO.Write themselves be the transform tyles.* Both should take simply String, and not use Coder as a general-purpose serialization mechanism.* * The Javadoc should be changed to reflect this.*  Should perhaps use AutoValue for parameter builders 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   



[jira] (BEAM-1361) Update build documentation with the new Python SDK

2017-01-31 Thread JIRA
Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Sergio Fernández created an issue 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 Beam /  BEAM-1361 
 
 
 
  Update build documentation with the new Python SDK  
 
 
 
 
 
 
 
 
 

Issue Type:
 
  Task 
 
 
 

Affects Versions:
 

 0.5.0 
 
 
 

Assignee:
 
 Ahmet Altay 
 
 
 

Components:
 

 sdk-py 
 
 
 

Created:
 

 31/Jan/17 12:29 
 
 
 

Priority:
 
  Minor 
 
 
 

Reporter:
 
 Sergio Fernández 
 
 
 
 
 
 
 
 
 
 
Superseding the work on BEAM-378, now that the Python SDK has been merged into master, there are some details of the build process that has to be properly updated. 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

[GitHub] beam pull request #1879: BEAM-1361: update Python build instructinons

2017-01-31 Thread wikier
GitHub user wikier opened a pull request:

https://github.com/apache/beam/pull/1879

BEAM-1361: update Python build instructinons

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [X] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [X] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [X] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [X] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---

As described at 
[BEAM-1361](https://issues.apache.org/jira/browse/BEAM-1361).


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/wikier/beam BEAM-1361_python_build

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/1879.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1879


commit bddc1c3ea671dc2b7b367a89590769f2652f3efc
Author: Sergio Fernández 
Date:   2017-01-31T12:31:42Z

BEAM-1361: updated sdk status at readme

commit 2e5a67a3d1ad5cb1427cc45e7b13c2a306c7
Author: Sergio Fernández 
Date:   2017-01-31T12:35:48Z

BEAM-1361: updated basic system requiremets




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] (BEAM-1361) Update build documentation with the new Python SDK

2017-01-31 Thread ASF GitHub Bot (JIRA)
Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 ASF GitHub Bot commented on  BEAM-1361 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
  Re: Update build documentation with the new Python SDK  
 
 
 
 
 
 
 
 
 
 
GitHub user wikier opened a pull request: 
 https://github.com/apache/beam/pull/1879 
 BEAM-1361: update Python build instructinons 
 Be sure to do all of the following to help us incorporate your contribution quickly and easily: 
 

[X] Make sure the PR title is formatted like: `BEAM- Description of pull request`
 

[X] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes).
 

[X] Replace `` in the title with the actual Jira issue number, if there is one.
 

[X] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt).
 
 
 — 
 As described at BEAM-1361(https://issues.apache.org/jira/browse/BEAM-1361). 
You can merge this pull request into a Git repository by running: 
 $ git pull https://github.com/wikier/beam BEAM-1361_python_build 
Alternatively you can review and apply these changes as the patch at: 
 https://github.com/apache/beam/pull/1879.patch 
To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: 
 This closes #1879 
 
commit bddc1c3ea671dc2b7b367a89590769f2652f3efc Author: Sergio Fernández  Date: 2017-01-31T12:31:42Z 
 BEAM-1361: updated sdk status at readme 
commit 2e5a67a3d1ad5cb1427cc45e7b13c2a306c7 Author: Sergio Fernández  Date: 2017-01-31T12:35:48Z 
 BEAM-1361: updated basic system requiremets 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
  

[GitHub] beam pull request #1880: [BEAM-1360] Remove reference to the obsolete python...

2017-01-31 Thread prabeesh
GitHub user prabeesh opened a pull request:

https://github.com/apache/beam/pull/1880

[BEAM-1360] Remove reference to the obsolete python-sdk branch

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/prabeesh/beam master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/1880.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1880


commit 1846a226c5cadc9b1cf27a60dac2bf0598ccee55
Author: Prabeesh 
Date:   2017-01-31T12:55:03Z

remove reference to the obsolete python-sdk branch




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] (BEAM-1360) Reference to the obsolete python-sdk branch in the documentation.

2017-01-31 Thread ASF GitHub Bot (JIRA)
Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 ASF GitHub Bot commented on  BEAM-1360 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
  Re: Reference to the obsolete python-sdk branch in the documentation.  
 
 
 
 
 
 
 
 
 
 
GitHub user prabeesh opened a pull request: 
 https://github.com/apache/beam/pull/1880 
 BEAM-1360 Remove reference to the obsolete python-sdk branch 
 Be sure to do all of the following to help us incorporate your contribution quickly and easily: 
 

[ ] Make sure the PR title is formatted like: `BEAM- Description of pull request`
 

[ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes).
 

[ ] Replace `` in the title with the actual Jira issue number, if there is one.
 

[ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt).
 
 
 — 
You can merge this pull request into a Git repository by running: 
 $ git pull https://github.com/prabeesh/beam master 
Alternatively you can review and apply these changes as the patch at: 
 https://github.com/apache/beam/pull/1880.patch 
To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: 
 This closes #1880 
 
commit 1846a226c5cadc9b1cf27a60dac2bf0598ccee55 Author: Prabeesh  Date: 2017-01-31T12:55:03Z 
 remove reference to the obsolete python-sdk branch 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
   

[jira] (BEAM-1278) Use checkstyle on everything

2017-01-31 Thread Benson Margulies (JIRA)
Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Benson Margulies commented on  BEAM-1278 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
  Re: Use checkstyle on everything  
 
 
 
 
 
 
 
 
 
 
There is a dilemma here which surfaced in a discussion I had with Davor Bonaci yesterday. 
The PR enables checkstyle everywhere by using a file system path for the rules instead of the classpath. It uses a Maven plugin to acquire the 'project root directory' from the subdirectories. (I don't really know how that plugin manages to do the right thing if you cd into some subdir and run it; does it just chase relativePath until it hits a parent which is somewhere else?) 
In any case, the effect of this is to make it impossible to build a submodule in isolation by unpacking its source jar. We're not happy to break that. 
There is an alternative: use the classpath when we can, and use the file path when we cannot. The top-level pom, and the 'sdks' pom, and the 'sdks/java' pom, would use the file path. The poms that currently use the classpath to enable checkstyle would override the configuration to use the classpath. That forces some duplication of the configuration; given the current directory structure, I don't know a way to avoid repeating this in several jars. 
Well, actually, I do, but it would be rather structurally violent: the top-level pom would need to have two children: buildtools and 'everything else'. 
Before I either go rework the patch to have this checkstyle repetition, or we close this down as a cure worse than a disease, I want to just point back to JB's remark: a common practice is to separately release a maven project containing these materials. We don't need to have a separate git repo; we can just have a directory which has an independent pom with the necessary maven-release-plugin configuration. The claim here is that the checkstyle rules do not change very often, so having to release the buildtools component whenever you change them is not so awful. 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   

[jira] (BEAM-1278) Use checkstyle on everything

2017-01-31 Thread JIRA
Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Jean-Baptiste Onofré commented on  BEAM-1278 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
  Re: Use checkstyle on everything  
 
 
 
 
 
 
 
 
 
 
Yes agree: having the checkstyle is a specific module with its own release cycle makes sense as it doesn't change so often (since the beginning of the Apache Beam, AFAIR, we changed the checkstyle rules 2 times only). 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   



[jira] (BEAM-1346) Drop Late Data in ReduceFnRunner

2017-01-31 Thread Aljoscha Krettek (JIRA)
Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Aljoscha Krettek commented on  BEAM-1346 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
  Re: Drop Late Data in ReduceFnRunner  
 
 
 
 
 
 
 
 
 
 
 + Thomas Weise and Amit Sela 
AFAIK this is not just an issue for the Flink Runner, from a quick glance it looks as if the Apex runner and Spark Runner also don't drop late elements. For the Dataflow Runner I don't know but I assume that there is some internal code that does the right thing. 
If we want to ensure that the behaviour for the supported features is always correct with different Runners then I would consider this a blocker. 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   



[jira] (BEAM-1346) Drop Late Data in ReduceFnRunner

2017-01-31 Thread Amit Sela (JIRA)
Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Amit Sela commented on  BEAM-1346 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
  Re: Drop Late Data in ReduceFnRunner  
 
 
 
 
 
 
 
 
 
 
For the Spark runner, my (endless..) WIP branch doesn't use DoFnRunner, but adopts the DirectRunner approach by dropping late data before processing the elements: https://github.com/amitsela/beam/blob/beam-model-streaming/runners/spark/src/main/java/org/apache/beam/runners/spark/stateful/StateSpecFunctions.java#L228. The implementation itself is a copy of DirectRunner code (with Thomas' blessing) and could be pulled to the runner API utils. 
But this is not on Master yet, so the Spark runner is not affected anyway. 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   



[jira] (BEAM-843) Use New DoFn Directly in Flink Runner

2017-01-31 Thread Aljoscha Krettek (JIRA)
Title: Message Title
 
 
 
 
 
 
 
 
 
 
  
 
 Aljoscha Krettek commented on  BEAM-843 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
  Re: Use New DoFn Directly in Flink Runner  
 
 
 
 
 
 
 
 
 
 
Daniel Halperin FV 0.6 is fine, sorry for the inconvenience. 
 
 
 
 
 
 
 
 
 
 
 
 

 
 Add Comment 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 

 This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d) 
 
 
 
 
  
 
 
 
 
 
 
 
 
   



[jira] (BEAM-1346) Drop Late Data in ReduceFnRunner

2017-01-31 Thread Aljoscha Krettek (JIRA)

[ 
https://issues.apache.org/jira/browse/BEAM-1346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15847109#comment-15847109
 ] 

Aljoscha Krettek commented on BEAM-1346:


[~kenn], yes exactly, that's how I was able to home in on those two commits. 
The changes are still good, we just need another place in to code to do the 
late-data dropping.

> Drop Late Data in ReduceFnRunner
> 
>
> Key: BEAM-1346
> URL: https://issues.apache.org/jira/browse/BEAM-1346
> Project: Beam
>  Issue Type: Bug
>  Components: runner-core
>Affects Versions: 0.5.0
>Reporter: Aljoscha Krettek
>
> I think these two commits recently broke late-data dropping for the Flink 
> Runner (and maybe for other runners as well):
> - https://github.com/apache/beam/commit/2b26ec8
> - https://github.com/apache/beam/commit/8989473
> It boils down to the {{LateDataDroppingDoFnRunner}} not being used anymore  
> because {{DoFnRunners.lateDataDroppingRunner()}} is not called anymore when a 
> {{DoFn}} is a {{ReduceFnExecutor}} (because that interface was removed).
> Maybe we should think about dropping late data in another place, my 
> suggestion is {{ReduceFnRunner}} but that's open for discussion.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] (BEAM-1346) Drop Late Data in ReduceFnRunner

2017-01-31 Thread Aljoscha Krettek (JIRA)

[ 
https://issues.apache.org/jira/browse/BEAM-1346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15847134#comment-15847134
 ] 

Aljoscha Krettek commented on BEAM-1346:


Also, isn't dropping late data before giving it to the {{ReduceFnRunner}} wrong 
when we have a merging {{WindowFn}}? When an element arrives with a window that 
would be late with respect to the watermark but that would end up getting 
merged into a bigger window that is not late (which will therefore still be 
holding back the output watermark) we wouldn't have to drop that element.

> Drop Late Data in ReduceFnRunner
> 
>
> Key: BEAM-1346
> URL: https://issues.apache.org/jira/browse/BEAM-1346
> Project: Beam
>  Issue Type: Bug
>  Components: runner-core
>Affects Versions: 0.5.0
>Reporter: Aljoscha Krettek
>
> I think these two commits recently broke late-data dropping for the Flink 
> Runner (and maybe for other runners as well):
> - https://github.com/apache/beam/commit/2b26ec8
> - https://github.com/apache/beam/commit/8989473
> It boils down to the {{LateDataDroppingDoFnRunner}} not being used anymore  
> because {{DoFnRunners.lateDataDroppingRunner()}} is not called anymore when a 
> {{DoFn}} is a {{ReduceFnExecutor}} (because that interface was removed).
> Maybe we should think about dropping late data in another place, my 
> suggestion is {{ReduceFnRunner}} but that's open for discussion.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[1/2] beam git commit: remove reference to the obsolete python-sdk branch

2017-01-31 Thread dhalperi
Repository: beam
Updated Branches:
  refs/heads/master 7a235f9d9 -> 7da5a2c27


remove reference to the obsolete python-sdk branch


Project: http://git-wip-us.apache.org/repos/asf/beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/1846a226
Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/1846a226
Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/1846a226

Branch: refs/heads/master
Commit: 1846a226c5cadc9b1cf27a60dac2bf0598ccee55
Parents: 7a235f9
Author: Prabeesh 
Authored: Tue Jan 31 16:55:03 2017 +0400
Committer: Prabeesh 
Committed: Tue Jan 31 16:55:03 2017 +0400

--
 README.md | 2 +-
 sdks/python/README.md | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/beam/blob/1846a226/README.md
--
diff --git a/README.md b/README.md
index f7492e4..ddb168a 100644
--- a/README.md
+++ b/README.md
@@ -52,7 +52,7 @@ The key concepts in the Beam programming model are:
 
 Beam supports multiple language specific SDKs for writing pipelines against 
the Beam Model.
 
-Currently, this repository contains the Java SDK, as well as the Python SDK in 
a [feature branch](https://github.com/apache/beam/tree/python-sdk).
+Currently, this repository contains the Java and Python SDKs.
 
 Have ideas for new SDKs or DSLs? See the 
[JIRA](https://issues.apache.org/jira/browse/BEAM/component/12328909/).
 

http://git-wip-us.apache.org/repos/asf/beam/blob/1846a226/sdks/python/README.md
--
diff --git a/sdks/python/README.md b/sdks/python/README.md
index 8a5e408..d202ebc 100644
--- a/sdks/python/README.md
+++ b/sdks/python/README.md
@@ -107,7 +107,7 @@ directory you created.
  Download and install
 
 1. Clone the Apache Beam repo from GitHub: 
-  `git clone https://github.com/apache/beam.git --branch python-sdk`
+  `git clone https://github.com/apache/beam.git`
 
 2. Navigate to the `python` directory: 
   `cd beam/sdks/python/`



[GitHub] beam pull request #1880: [BEAM-1360] Remove reference to the obsolete python...

2017-01-31 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/1880


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[2/2] beam git commit: This closes #1880

2017-01-31 Thread dhalperi
This closes #1880


Project: http://git-wip-us.apache.org/repos/asf/beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/7da5a2c2
Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/7da5a2c2
Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/7da5a2c2

Branch: refs/heads/master
Commit: 7da5a2c2703f5185f4e1c0682a79583a66f68523
Parents: 7a235f9 1846a22
Author: Dan Halperin 
Authored: Tue Jan 31 09:12:45 2017 -0800
Committer: Dan Halperin 
Committed: Tue Jan 31 09:12:45 2017 -0800

--
 README.md | 2 +-
 sdks/python/README.md | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
--




[jira] (BEAM-1360) Reference to the obsolete python-sdk branch in the documentation.

2017-01-31 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/BEAM-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15847143#comment-15847143
 ] 

ASF GitHub Bot commented on BEAM-1360:
--

Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/1880


> Reference to the obsolete python-sdk branch in the documentation.
> -
>
> Key: BEAM-1360
> URL: https://issues.apache.org/jira/browse/BEAM-1360
> Project: Beam
>  Issue Type: Improvement
>  Components: project-management
>Reporter: Prabeesh K
>Assignee: Davor Bonaci
>
> Following documents contains reference to obsolete python-sdk
> - [README|https://github.com/apache/beam/blame/master/README.md#L55]
> - [PYTHON SDK 
> README|https://github.com/apache/beam/blame/master/sdks/python/README.md#L110]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] (BEAM-1360) Reference to the obsolete python-sdk branch in the documentation.

2017-01-31 Thread Daniel Halperin (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Halperin reassigned BEAM-1360:
-

Assignee: Prabeesh K  (was: Davor Bonaci)

> Reference to the obsolete python-sdk branch in the documentation.
> -
>
> Key: BEAM-1360
> URL: https://issues.apache.org/jira/browse/BEAM-1360
> Project: Beam
>  Issue Type: Improvement
>  Components: project-management
>Reporter: Prabeesh K
>Assignee: Prabeesh K
> Fix For: Not applicable
>
>
> Following documents contains reference to obsolete python-sdk
> - [README|https://github.com/apache/beam/blame/master/README.md#L55]
> - [PYTHON SDK 
> README|https://github.com/apache/beam/blame/master/sdks/python/README.md#L110]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] (BEAM-1360) Reference to the obsolete python-sdk branch in the documentation.

2017-01-31 Thread Daniel Halperin (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Halperin resolved BEAM-1360.
---
   Resolution: Fixed
Fix Version/s: Not applicable

> Reference to the obsolete python-sdk branch in the documentation.
> -
>
> Key: BEAM-1360
> URL: https://issues.apache.org/jira/browse/BEAM-1360
> Project: Beam
>  Issue Type: Improvement
>  Components: project-management
>Reporter: Prabeesh K
>Assignee: Prabeesh K
> Fix For: Not applicable
>
>
> Following documents contains reference to obsolete python-sdk
> - [README|https://github.com/apache/beam/blame/master/README.md#L55]
> - [PYTHON SDK 
> README|https://github.com/apache/beam/blame/master/sdks/python/README.md#L110]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] (BEAM-1346) Drop Late Data in ReduceFnRunner

2017-01-31 Thread Kenneth Knowles (JIRA)

[ 
https://issues.apache.org/jira/browse/BEAM-1346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15847156#comment-15847156
 ] 

Kenneth Knowles commented on BEAM-1346:
---

Looking at the history, the Apex runner never relied on 
{{DoFnRunners.createDefault}} to set up late data dropping. I think it was just 
never done.

One way to look at the gap in coverage here is that because all of our 
{{RunnableOnService}} tests (until very recently) have only bounded input, the 
integration testing of a runner + the triggering logic is rather poor. The test 
cases we want are in {{ReduceFnRunnerTest}}. One obvious approach (I am not 
claiming it is best) is for runners to implement {{TestStream}} and port these 
all these to {{RunnableOnService}} tests. They basically are already in that 
form.

There was a time that {{ReduceFnRunner}} did the dropping, I believe, and it 
was factored out to try to be reused in some other situation where late data 
would cause a problem. I've forgotten the details. Given the code coverage 
situation, it might be reasonable to have both.

> Drop Late Data in ReduceFnRunner
> 
>
> Key: BEAM-1346
> URL: https://issues.apache.org/jira/browse/BEAM-1346
> Project: Beam
>  Issue Type: Bug
>  Components: runner-core
>Affects Versions: 0.5.0
>Reporter: Aljoscha Krettek
>
> I think these two commits recently broke late-data dropping for the Flink 
> Runner (and maybe for other runners as well):
> - https://github.com/apache/beam/commit/2b26ec8
> - https://github.com/apache/beam/commit/8989473
> It boils down to the {{LateDataDroppingDoFnRunner}} not being used anymore  
> because {{DoFnRunners.lateDataDroppingRunner()}} is not called anymore when a 
> {{DoFn}} is a {{ReduceFnExecutor}} (because that interface was removed).
> Maybe we should think about dropping late data in another place, my 
> suggestion is {{ReduceFnRunner}} but that's open for discussion.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[GitHub] beam-site pull request #128: Restructure conceptual documentation on Beam si...

2017-01-31 Thread hadarhg
Github user hadarhg closed the pull request at:

https://github.com/apache/beam-site/pull/128


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] (BEAM-1360) Reference to the obsolete python-sdk branch in the documentation.

2017-01-31 Thread Ahmet Altay (JIRA)

[ 
https://issues.apache.org/jira/browse/BEAM-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15847185#comment-15847185
 ] 

Ahmet Altay commented on BEAM-1360:
---

There are actually a few more python-sdk references in the README.md in the 
links. I can fix those.

> Reference to the obsolete python-sdk branch in the documentation.
> -
>
> Key: BEAM-1360
> URL: https://issues.apache.org/jira/browse/BEAM-1360
> Project: Beam
>  Issue Type: Improvement
>  Components: project-management
>Reporter: Prabeesh K
>Assignee: Prabeesh K
> Fix For: Not applicable
>
>
> Following documents contains reference to obsolete python-sdk
> - [README|https://github.com/apache/beam/blame/master/README.md#L55]
> - [PYTHON SDK 
> README|https://github.com/apache/beam/blame/master/sdks/python/README.md#L110]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[GitHub] beam pull request #1740: [BEAM-246] Improve developer experience after strea...

2017-01-31 Thread dhalperi
Github user dhalperi closed the pull request at:

https://github.com/apache/beam/pull/1740


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] (BEAM-246) Streamline default Maven profile for user efficiency

2017-01-31 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/BEAM-246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15847194#comment-15847194
 ] 

ASF GitHub Bot commented on BEAM-246:
-

Github user dhalperi closed the pull request at:

https://github.com/apache/beam/pull/1740


> Streamline default Maven profile for user efficiency
> 
>
> Key: BEAM-246
> URL: https://issues.apache.org/jira/browse/BEAM-246
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Daniel Halperin
>Priority: Minor
> Fix For: 0.4.0
>
>
> We should optimize the default maven profile for interactive human use. Right 
> now it does a lot of things that waste time & CPU and do not speed 
> development, like building the source jar and javadoc.
> We certainly still want to build all of these things in release tests, 
> probably in post-commit tests, and case-by-case in pre-commit tests. But for 
> these we can set up a profile to activate them.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[GitHub] beam pull request #1881: [BEAM-1360] Remove python-sdk links from the sdks/p...

2017-01-31 Thread aaltay
GitHub user aaltay opened a pull request:

https://github.com/apache/beam/pull/1881

[BEAM-1360] Remove python-sdk links from the sdks/python/README.md



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/aaltay/incubator-beam read

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/1881.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1881


commit 993785701de7d02ac361a517cd832cf72f5eeae7
Author: Ahmet Altay 
Date:   2017-01-31T17:43:31Z

Remove python-sdk links from the sdks/python/README.md




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] (BEAM-1360) Reference to the obsolete python-sdk branch in the documentation.

2017-01-31 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/BEAM-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15847198#comment-15847198
 ] 

ASF GitHub Bot commented on BEAM-1360:
--

GitHub user aaltay opened a pull request:

https://github.com/apache/beam/pull/1881

[BEAM-1360] Remove python-sdk links from the sdks/python/README.md



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/aaltay/incubator-beam read

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/1881.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1881


commit 993785701de7d02ac361a517cd832cf72f5eeae7
Author: Ahmet Altay 
Date:   2017-01-31T17:43:31Z

Remove python-sdk links from the sdks/python/README.md




> Reference to the obsolete python-sdk branch in the documentation.
> -
>
> Key: BEAM-1360
> URL: https://issues.apache.org/jira/browse/BEAM-1360
> Project: Beam
>  Issue Type: Improvement
>  Components: project-management
>Reporter: Prabeesh K
>Assignee: Prabeesh K
> Fix For: Not applicable
>
>
> Following documents contains reference to obsolete python-sdk
> - [README|https://github.com/apache/beam/blame/master/README.md#L55]
> - [PYTHON SDK 
> README|https://github.com/apache/beam/blame/master/sdks/python/README.md#L110]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] (BEAM-804) Python Pipeline Option save_main_session non-functional

2017-01-31 Thread Sourabh Bajaj (JIRA)

[ 
https://issues.apache.org/jira/browse/BEAM-804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15847279#comment-15847279
 ] 

Sourabh Bajaj commented on BEAM-804:


This should be fixed by 
https://github.com/apache/beam/commit/aef4858b80dfacf3401e6672b9373c82a8e77027

> Python Pipeline Option save_main_session non-functional
> ---
>
> Key: BEAM-804
> URL: https://issues.apache.org/jira/browse/BEAM-804
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py
> Environment: OSX El Capitan, google-cloud-dataflow==0.4.3, python 
> 2.7.12
>Reporter: Zoran Bujila
>Priority: Critical
>
> When trying to use the option --save_main_session a pickling error occurs.
> pickle.PicklingError: Can't pickle  'apache_beam.internal.clients.dataflow.dataflow_v1b3_messages.TypeValueValuesEnum'>:
>  it's not found as 
> apache_beam.internal.clients.dataflow.dataflow_v1b3_messages.TypeValueValuesEnu
> This prevents the use of this option which is desirable as there is an 
> expensive object that needs to be created on each worker in my pipeline and I 
> would like to have this object created only once per worker. It is not 
> practical to have it inline with the ParDo function unless I make the batch 
> size sent to the ParDo quite large. Doing this seems to lead to idle workers 
> and I would ideally want to bring the batch size way down.
> The "Affects Version" option above doesn't have a 0.4.3 version in the drop 
> down so I did not populate it. However, this was a problem with 0.4.1 and has 
> not been corrected with 0.4.3.
> I don't see where I can attach a file, so here is the entire error.
> 2016-10-24 10:00:16,071 The 
> oauth2client.contrib.multistore_file module has been deprecated and will be 
> removed in the next release of oauth2client. Please migrate to 
> multiprocess_file_storage.
> 2016-10-24 10:00:16,127 __init__Direct usage of TextFileSink is 
> deprecated. Please use 'textio.WriteToText()' instead of directly 
> instantiating a TextFileSink object.
> Traceback (most recent call last):
>   File "00test.py", line 41, in 
> p.run()
>   File "/usr/local/lib/python2.7/site-packages/apache_beam/pipeline.py", line 
> 159, in run
> return self.runner.run(self)
>   File 
> "/usr/local/lib/python2.7/site-packages/apache_beam/runners/dataflow_runner.py",
>  line 172, in run
> self.dataflow_client.create_job(self.job))
>   File "/usr/local/lib/python2.7/site-packages/apache_beam/utils/retry.py", 
> line 160, in wrapper
> return fun(*args, **kwargs)
>   File 
> "/usr/local/lib/python2.7/site-packages/apache_beam/internal/apiclient.py", 
> line 375, in create_job
> job.options, file_copy=self._gcs_file_copy)
>   File 
> "/usr/local/lib/python2.7/site-packages/apache_beam/utils/dependency.py", 
> line 325, in stage_job_resources
> pickler.dump_session(pickled_session_file)
>   File 
> "/usr/local/lib/python2.7/site-packages/apache_beam/internal/pickler.py", 
> line 204, in dump_session
> return dill.dump_session(file_path)
>   File "/usr/local/lib/python2.7/site-packages/dill/dill.py", line 333, in 
> dump_session
> pickler.dump(main)
>   File 
> "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
>  line 224, in dump
> self.save(obj)
>   File 
> "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
>  line 286, in save
> f(self, obj) # Call unbound method with explicit self
>   File 
> "/usr/local/lib/python2.7/site-packages/apache_beam/internal/pickler.py", 
> line 123, in save_module
> return old_save_module(pickler, obj)
>   File "/usr/local/lib/python2.7/site-packages/dill/dill.py", line 1168, in 
> save_module
> state=_main_dict)
>   File 
> "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
>  line 425, in save_reduce
> save(state)
>   File 
> "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
>  line 286, in save
> f(self, obj) # Call unbound method with explicit self
>   File 
> "/usr/local/lib/python2.7/site-packages/apache_beam/internal/pickler.py", 
> line 159, in new_save_module_dict
> return old_save_module_dict(pickler, obj)
>   File "/usr/local/lib/python2.7/site-packages/dill/dill.py", line 835, in 
> save_module_dict
> StockPickler.save_dict(pickler, obj)
>   File 
> "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
>  line 655, in save_dict
> self._batch_setitems(obj.iteritems())
>   File 
> "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
>  line 687, in _batch_setitems
> save(v)
>   File 
> "

[jira] (BEAM-1346) Drop Late Data in ReduceFnRunner

2017-01-31 Thread Kenneth Knowles (JIRA)

[ 
https://issues.apache.org/jira/browse/BEAM-1346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15847293#comment-15847293
 ] 

Kenneth Knowles commented on BEAM-1346:
---

[~aljoscha] that is a very interesting point and I think I agree with you, 
though I need to ponder it a little more.

For context, at some point I switched late data dropping from {{(WM - 
timestamp) > allowedLateness}} to the simpler-to-understand and seemingly 
better-behaved {{(WM - EOW) > allowedLateness}} (big caveat that it doesn't 
work in global window, but you always have to stream with a trigger and 
discarding mode there to avoid leaks). In the first situation, the sort of data 
you are talking about was always dropped anyhow. When I switched it, I did not 
make any special consideration for tiny to-be-merged "proto"-windows. So they 
are still dropped somewhat like the original design, when {{(WM - gap duration 
- timestamp) > allowedLateness}}. I think that question fits this ticket, but 
doesn't relate to the release blocking question since it has been like this 
since before Beam I think.

> Drop Late Data in ReduceFnRunner
> 
>
> Key: BEAM-1346
> URL: https://issues.apache.org/jira/browse/BEAM-1346
> Project: Beam
>  Issue Type: Bug
>  Components: runner-core
>Affects Versions: 0.5.0
>Reporter: Aljoscha Krettek
>
> I think these two commits recently broke late-data dropping for the Flink 
> Runner (and maybe for other runners as well):
> - https://github.com/apache/beam/commit/2b26ec8
> - https://github.com/apache/beam/commit/8989473
> It boils down to the {{LateDataDroppingDoFnRunner}} not being used anymore  
> because {{DoFnRunners.lateDataDroppingRunner()}} is not called anymore when a 
> {{DoFn}} is a {{ReduceFnExecutor}} (because that interface was removed).
> Maybe we should think about dropping late data in another place, my 
> suggestion is {{ReduceFnRunner}} but that's open for discussion.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[2/2] beam git commit: This closes #1881

2017-01-31 Thread dhalperi
This closes #1881


Project: http://git-wip-us.apache.org/repos/asf/beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/e053207f
Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/e053207f
Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/e053207f

Branch: refs/heads/master
Commit: e053207f469d3090a38ed298e708b6a6acaf9de3
Parents: 7da5a2c c40c06e
Author: Dan Halperin 
Authored: Tue Jan 31 12:02:06 2017 -0800
Committer: Dan Halperin 
Committed: Tue Jan 31 12:02:06 2017 -0800

--
 sdks/python/README.md | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)
--




[1/2] beam git commit: Remove python-sdk links from the sdks/python/README.md

2017-01-31 Thread dhalperi
Repository: beam
Updated Branches:
  refs/heads/master 7da5a2c27 -> e053207f4


Remove python-sdk links from the sdks/python/README.md


Project: http://git-wip-us.apache.org/repos/asf/beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/c40c06eb
Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/c40c06eb
Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/c40c06eb

Branch: refs/heads/master
Commit: c40c06ebcd744d49de8f32ec886e90a8db921a9e
Parents: 7da5a2c
Author: Ahmet Altay 
Authored: Tue Jan 31 09:43:31 2017 -0800
Committer: Dan Halperin 
Committed: Tue Jan 31 12:02:02 2017 -0800

--
 sdks/python/README.md | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/beam/blob/c40c06eb/sdks/python/README.md
--
diff --git a/sdks/python/README.md b/sdks/python/README.md
index d202ebc..5b75db2 100644
--- a/sdks/python/README.md
+++ b/sdks/python/README.md
@@ -123,7 +123,7 @@ directory you created.
 
 ### Execute a pipeline locally
 
-The Apache Beam 
[examples](https://github.com/apache/beam/tree/python-sdk/sdks/python/apache_beam/examples)
 directory has many examples. All examples can be run locally by passing the 
arguments required by the example script.
+The Apache Beam 
[examples](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples)
 directory has many examples. All examples can be run locally by passing the 
arguments required by the example script.
 
 For example, to run `wordcount.py`, run:
 
@@ -157,7 +157,7 @@ Make sure you installed the package first. If not, run 
`python setup.py install`
 
 ## Simple Examples
 
-The following examples demonstrate some basic, fundamental concepts for using 
Apache Beam's Python SDK. For more detailed examples, Beam provides a 
[directory of 
examples](https://github.com/apache/beam/tree/python-sdk/sdks/python/apache_beam/examples)
 for Python.
+The following examples demonstrate some basic, fundamental concepts for using 
Apache Beam's Python SDK. For more detailed examples, Beam provides a 
[directory of 
examples](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples)
 for Python.
 
 ### Basic pipeline
 
@@ -357,13 +357,13 @@ SAMPLE_DATA = [('a', 1), ('b', 10), ('a', 2), ('a', 3), 
('b', 20)]
 p.run()
 ```
 
-The 
[combiners_test.py](https://github.com/apache/beam/blob/python-sdk/sdks/python/apache_beam/examples/cookbook/combiners_test.py)
 file contains more combiner examples.
+The 
[combiners_test.py](https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/cookbook/combiners_test.py)
 file contains more combiner examples.
 
 ## Organizing Your Code
 
 Many projects will grow to multiple source code files. It is recommended that 
you organize your project so that all code involved in running your pipeline 
can be built as a Python package. This way, the package can easily be installed 
in the VM workers executing the job.
 
-Follow the [Juliaset 
example](https://github.com/apache/beam/tree/python-sdk/sdks/python/apache_beam/examples/complete/juliaset).
 If the code is organized in this fashion, you can use the `--setup_file` 
command line option to create a source distribution out of the project files, 
stage the resulting tarball, and later install it in the workers executing the 
job.
+Follow the [Juliaset 
example](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/complete/juliaset).
 If the code is organized in this fashion, you can use the `--setup_file` 
command line option to create a source distribution out of the project files, 
stage the resulting tarball, and later install it in the workers executing the 
job.
 
 ## More Information
 



[GitHub] beam pull request #1881: [BEAM-1360] Remove python-sdk links from the sdks/p...

2017-01-31 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/1881


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] (BEAM-1360) Reference to the obsolete python-sdk branch in the documentation.

2017-01-31 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/BEAM-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15847413#comment-15847413
 ] 

ASF GitHub Bot commented on BEAM-1360:
--

Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/1881


> Reference to the obsolete python-sdk branch in the documentation.
> -
>
> Key: BEAM-1360
> URL: https://issues.apache.org/jira/browse/BEAM-1360
> Project: Beam
>  Issue Type: Improvement
>  Components: project-management
>Reporter: Prabeesh K
>Assignee: Prabeesh K
> Fix For: Not applicable
>
>
> Following documents contains reference to obsolete python-sdk
> - [README|https://github.com/apache/beam/blame/master/README.md#L55]
> - [PYTHON SDK 
> README|https://github.com/apache/beam/blame/master/sdks/python/README.md#L110]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[1/2] beam git commit: [BEAM-475] Build aggregate javadoc in the build.

2017-01-31 Thread davor
Repository: beam
Updated Branches:
  refs/heads/master e053207f4 -> b80aac5e3


[BEAM-475] Build aggregate javadoc in the build.

Having failed to make javadoc:aggregate work in a pom,
this commit uses antrun to, well, run ant in order to
create a Javadoc Monolith for publication to the website.


Project: http://git-wip-us.apache.org/repos/asf/beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/82df9b6b
Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/82df9b6b
Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/82df9b6b

Branch: refs/heads/master
Commit: 82df9b6b78863c1b7153eb7ff2a705c507dab05b
Parents: e053207
Author: bimargulies-google 
Authored: Fri Jan 13 15:24:30 2017 -0800
Committer: Davor Bonaci 
Committed: Tue Jan 31 12:19:07 2017 -0800

--
 pom.xml   | 111 ++--
 sdks/java/javadoc/ant.xml |  91 +
 sdks/java/javadoc/pom.xml | 291 +
 sdks/java/pom.xml |   3 +
 4 files changed, 486 insertions(+), 10 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/beam/blob/82df9b6b/pom.xml
--
diff --git a/pom.xml b/pom.xml
index 1c868a8..6776db1 100644
--- a/pom.xml
+++ b/pom.xml
@@ -130,6 +130,7 @@
 3.1.0
 v1-rev10-1.22.0
 1.7.14
+1.6.2
 3.1.4
 v1-rev71-1.22.0
 4.4.1
@@ -148,6 +149,8 @@
 sdks
 runners
 examples
+
+sdks/java/javadoc
   
 
   
@@ -162,9 +165,6 @@
 
   org.apache.maven.plugins
   maven-javadoc-plugin
-  
-${beam.javadoc_opts}
-  
   
 
   javadoc
@@ -322,11 +322,35 @@
 
   
 org.apache.beam
+beam-runners-flink_2.10-examples
+${project.version}
+  
+
+  
+org.apache.beam
 beam-sdks-java-core
 ${project.version}
   
 
   
+   org.apache.beam
+   beam-sdks-java-extensions-join-library
+${project.version}
+  
+
+  
+   org.apache.beam
+   beam-sdks-java-extensions-sorter
+${project.version}
+  
+
+  
+   org.apache.beam
+   beam-sdks-java-harness
+${project.version}
+  
+
+  
 org.apache.beam
 beam-sdks-java-io-google-cloud-platform
 ${project.version}
@@ -334,12 +358,54 @@
 
   
 org.apache.beam
+beam-sdks-java-io-elasticsearch
+${project.version}
+  
+
+  
+org.apache.beam
+beam-sdks-java-io-hdfs
+${project.version}
+  
+
+  
+org.apache.beam
+beam-sdks-java-io-jdbc
+${project.version}
+  
+
+  
+org.apache.beam
+beam-sdks-java-io-jms
+${project.version}
+  
+
+  
+org.apache.beam
 beam-sdks-java-io-kafka
 ${project.version}
   
 
   
 org.apache.beam
+beam-sdks-java-io-kinesis
+${project.version}
+  
+
+  
+org.apache.beam
+beam-sdks-java-io-mongodb
+${project.version}
+  
+
+  
+org.apache.beam
+beam-sdks-java-io-mqtt
+${project.version}
+  
+
+  
+org.apache.beam
 beam-runners-core-java
 ${project.version}
   
@@ -381,6 +447,12 @@
   
 
   
+org.apache.beam
+beam-examples-java8
+${project.version}
+  
+
+  
 io.grpc
 grpc-all
 ${grpc.version}
@@ -860,6 +932,12 @@
 
 
   org.apache.maven.plugins
+  maven-antrun-plugin
+  1.8
+
+
+
+  org.apache.maven.plugins
   maven-checkstyle-plugin
   2.17
   
@@ -974,9 +1052,22 @@
 
 
 
+  
   org.apache.maven.plugins
   maven-javadoc-plugin
   2.10.3
+  
+${beam.javadoc_opts}
+Apache Beam SDK for Java, version ${project.version} 
API
+Apache Beam SDK for Java, version 
${project.version}
+false
+true
+  
 
 
 
@@ -1020,7 +,13 @@
   versions-maven-plugin
   2.1
 
-
+   
+
+  org.codehaus.mojo
+  build-helper-maven-plugin
+  1.12
+
+   
 
   org.codehaus.mojo
   exec-maven-plugin
@@ -1160,12 +1257,6 @@
 
 
 
-  org.codehaus.mojo
-  build-helper-maven-plugin
-  1.10
-
-
-
   org.apache.maven.plugins
   maven-shade-plugin
   2.4.3

http://git-wip-us.apache.org/repos/asf/beam/blob/82df9b6b/sdks/java/javadoc/ant.xml
---

[2/2] beam git commit: This closes #1781

2017-01-31 Thread davor
This closes #1781


Project: http://git-wip-us.apache.org/repos/asf/beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/b80aac5e
Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/b80aac5e
Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/b80aac5e

Branch: refs/heads/master
Commit: b80aac5e33d6d4171d756d8caf67ec8f10dc4e86
Parents: e053207 82df9b6
Author: Davor Bonaci 
Authored: Tue Jan 31 12:19:22 2017 -0800
Committer: Davor Bonaci 
Committed: Tue Jan 31 12:19:22 2017 -0800

--
 pom.xml   | 111 ++--
 sdks/java/javadoc/ant.xml |  91 +
 sdks/java/javadoc/pom.xml | 291 +
 sdks/java/pom.xml |   3 +
 4 files changed, 486 insertions(+), 10 deletions(-)
--




[GitHub] beam pull request #1781: [BEAM-475] Improve Javadoc Aggregation

2017-01-31 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/1781


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] (BEAM-475) High-quality javadoc for Beam

2017-01-31 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/BEAM-475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15847437#comment-15847437
 ] 

ASF GitHub Bot commented on BEAM-475:
-

Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/1781


> High-quality javadoc for Beam
> -
>
> Key: BEAM-475
> URL: https://issues.apache.org/jira/browse/BEAM-475
> Project: Beam
>  Issue Type: Improvement
>  Components: project-management
>Reporter: Daniel Halperin
>Assignee: Benson Margulies
> Fix For: Not applicable
>
>
> We should have good Javadoc for Beam!
> Current snapshot: http://beam.incubator.apache.org/javadoc/0.1.0-incubating/



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] (BEAM-1346) Drop Late Data in ReduceFnRunner

2017-01-31 Thread Aljoscha Krettek (JIRA)

[ 
https://issues.apache.org/jira/browse/BEAM-1346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15847474#comment-15847474
 ] 

Aljoscha Krettek commented on BEAM-1346:


Yep, that fact (the Apex runner never using the DoFnRunners) is part of the 
reason why I suggested to put it into {{ReduceFnRunner}}.

Wherever we put it, in the end we need some good tests to ensure that dropping 
(and triggering) works correctly in all runners. :-)

> Drop Late Data in ReduceFnRunner
> 
>
> Key: BEAM-1346
> URL: https://issues.apache.org/jira/browse/BEAM-1346
> Project: Beam
>  Issue Type: Bug
>  Components: runner-core
>Affects Versions: 0.5.0
>Reporter: Aljoscha Krettek
>
> I think these two commits recently broke late-data dropping for the Flink 
> Runner (and maybe for other runners as well):
> - https://github.com/apache/beam/commit/2b26ec8
> - https://github.com/apache/beam/commit/8989473
> It boils down to the {{LateDataDroppingDoFnRunner}} not being used anymore  
> because {{DoFnRunners.lateDataDroppingRunner()}} is not called anymore when a 
> {{DoFn}} is a {{ReduceFnExecutor}} (because that interface was removed).
> Maybe we should think about dropping late data in another place, my 
> suggestion is {{ReduceFnRunner}} but that's open for discussion.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[GitHub] beam pull request #1882: Combiner lifting

2017-01-31 Thread robertwb
GitHub user robertwb opened a pull request:

https://github.com/apache/beam/pull/1882

Combiner lifting

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/robertwb/incubator-beam combiner-lifting

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/1882.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1882


commit d25b14fc74f24bc88debbf3fd42a3bc775249462
Author: Robert Bradshaw 
Date:   2017-01-31T19:41:09Z

Implement combiner lifting for direct runner.

commit 07e4154d4afbd0cbdda4419a7c18e6e338cef869
Author: Robert Bradshaw 
Date:   2017-01-31T20:41:22Z

Fix typehints tests.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] beam pull request #1883: Fix Release 0.5.0 with Reverts

2017-01-31 Thread aljoscha
GitHub user aljoscha opened a pull request:

https://github.com/apache/beam/pull/1883

Fix Release 0.5.0 with Reverts

I'm opening this just so that I can run PostCommit hooks on Jenkins. Sorry 
for the inconvenience.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/aljoscha/beam release-0.5.0

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/1883.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1883


commit 796649d70a3a2772a6b4b010b44563ca61b7c2d1
Author: Pei He 
Date:   2017-01-24T00:25:43Z

[BEAM-1235] BigQueryIO.Write: log failed load/copy jobs.

commit 8505e9312705ceb0bb5779580c930ab2113c0b3d
Author: Jesse Anderson 
Date:   2017-01-24T16:37:33Z

Refactored existing code. Added iterable and KV. Changed from element to of.

commit fccfa8ae06f6a29e208557d3181d26f6e0eb4bd5
Author: Aljoscha Krettek 
Date:   2017-01-31T20:27:17Z

Revert "Removes ReduceFnExecutor interface"

This reverts commit 8989473b8e379a40b888565aadead001379c9398.

commit d6ddce31aad92c4c61ff4614afd37cdb25b6e6aa
Author: Aljoscha Krettek 
Date:   2017-01-31T20:27:27Z

Revert "Removes some OldDoFn code from DoFnRunners"

This reverts commit 2b26ec8934725a600954ced9c4063766a582396a.

commit 27ad60323974eb4cd889e4586ab21697f6eff172
Author: Aljoscha Krettek 
Date:   2017-01-31T20:34:07Z

Revert "Moves OldDoFn to runners-core"

This reverts commit 5f8b8c5b06cfd49c4293a20dff2eea99f1076444.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] beam pull request #1884: Reduce cost of sample combine test

2017-01-31 Thread robertwb
GitHub user robertwb opened a pull request:

https://github.com/apache/beam/pull/1884

Reduce cost of sample combine test

There's no need to run this test 300 times, which takes tens of seconds.

Also split global and per-key sampling into separate tests.

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/robertwb/incubator-beam patch-4

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/1884.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1884


commit 7f266ff0835efb96e55785f2a2eb93e4eab9224c
Author: Robert Bradshaw 
Date:   2017-01-31T20:51:27Z

Reduce cost of sample combine test

There's no need to run this test 300 times, which takes tens of seconds.

Also split global and per-key sampling into separate tests.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


Jenkins build became unstable: beam_PostCommit_Java_MavenInstall #2486

2017-01-31 Thread Apache Jenkins Server
See 



Build failed in Jenkins: beam_PostCommit_Python_Verify #1126

2017-01-31 Thread Apache Jenkins Server
See 

--
Started by user dhalperi
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git -c core.askpass=true fetch --tags --progress 
 > https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/*:refs/remotes/origin/pr/*
 > git rev-parse refs/remotes/origin/pr/1864/head^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/pr/1864/head^{commit} # timeout=10
Checking out Revision 278b0c6aa8cf8bc62e2a65d2b95f259b61fc2270 
(refs/remotes/origin/pr/1864/head)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 278b0c6aa8cf8bc62e2a65d2b95f259b61fc2270
First time build. Skipping changelog.
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
[beam_PostCommit_Python_Verify] $ /bin/bash -xe 
/tmp/hudson8165036395902967873.sh
+ bash sdks/python/run_postcommit.sh
bash: sdks/python/run_postcommit.sh: No such file or directory
Build step 'Execute shell' marked build as failure


[GitHub] beam-site pull request #137: Cleanup of links: relative instead of absolute,...

2017-01-31 Thread davorbonaci
GitHub user davorbonaci opened a pull request:

https://github.com/apache/beam-site/pull/137

Cleanup of links: relative instead of absolute, https instead of http, etc.

R: @dhalperi 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/davorbonaci/beam-site https

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam-site/pull/137.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #137






---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[beam] Git Push Summary

2017-01-31 Thread davor
Repository: beam
Updated Branches:
  refs/heads/python-sdk [deleted] be0e32e36


[jira] (BEAM-1046) Travis (Linux) failing for python sdk

2017-01-31 Thread Ahmet Altay (JIRA)

 [ 
https://issues.apache.org/jira/browse/BEAM-1046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ahmet Altay closed BEAM-1046.
-
   Resolution: Fixed
Fix Version/s: Not applicable

> Travis (Linux) failing for python sdk
> -
>
> Key: BEAM-1046
> URL: https://issues.apache.org/jira/browse/BEAM-1046
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py
>Reporter: Ahmet Altay
> Fix For: Not applicable
>
>
> All PRs are failing with the same error:
> An example: https://travis-ci.org/apache/incubator-beam/builds/178435675
> $ if [ "$TEST_PYTHON" ] && ! pip list | grep tox; then travis_retry pip 
> install tox --user `whoami`; fi
> You are using pip version 6.0.8, however version 9.0.1 is available.
> You should consider upgrading via the 'pip install --upgrade pip' command.
> Collecting tox
>   Downloading tox-2.5.0-py2.py3-none-any.whl (42kB)
> 100% || 45kB 6.3MB/s 
> Collecting travis
>   Could not find any downloads that satisfy the requirement travis
>   No distributions at all found for travis
> The command "pip install tox --user travis" failed. Retrying, 2 of 3.
> You are using pip version 6.0.8, however version 9.0.1 is available.
> You should consider upgrading via the 'pip install --upgrade pip' command.
> Collecting tox
>   Using cached tox-2.5.0-py2.py3-none-any.whl
> Collecting travis
>   Could not find any downloads that satisfy the requirement travis
>   No distributions at all found for travis
> The command "pip install tox --user travis" failed. Retrying, 3 of 3.
> You are using pip version 6.0.8, however version 9.0.1 is available.
> You should consider upgrading via the 'pip install --upgrade pip' command.
> Collecting tox
>   Using cached tox-2.5.0-py2.py3-none-any.whl
> Collecting travis
>   Could not find any downloads that satisfy the requirement travis
>   No distributions at all found for travis
> The command "pip install tox --user travis" failed 3 times.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[2/3] beam-site git commit: Regenerate website

2017-01-31 Thread davor
Regenerate website


Project: http://git-wip-us.apache.org/repos/asf/beam-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam-site/commit/66c0a704
Tree: http://git-wip-us.apache.org/repos/asf/beam-site/tree/66c0a704
Diff: http://git-wip-us.apache.org/repos/asf/beam-site/diff/66c0a704

Branch: refs/heads/asf-site
Commit: 66c0a7042272d8e0f8944b5c8c655733a04a42ae
Parents: f67cdd1
Author: Davor Bonaci 
Authored: Tue Jan 31 14:38:28 2017 -0800
Committer: Davor Bonaci 
Committed: Tue Jan 31 14:38:28 2017 -0800

--
 content/contribute/team/index.html | 27 +++
 1 file changed, 27 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/beam-site/blob/66c0a704/content/contribute/team/index.html
--
diff --git a/content/contribute/team/index.html 
b/content/contribute/team/index.html
index 3281eb3..47dcc98 100644
--- a/content/contribute/team/index.html
+++ b/content/contribute/team/index.html
@@ -177,6 +177,15 @@
 
   
 
+  Ahmet Altay
+  altay
+  altay [at] apache [dot] org
+  Google
+  committer
+  -8
+
+  
+
   Jesse Anderson
   jesseanderson
   jesseanderson [at] apache [dot] org
@@ -249,6 +258,15 @@
 
   
 
+  Pei He
+  pei
+  pei [at] apache [dot] org
+  Google
+  committer
+  -8
+
+  
+
   Kenneth Knowles
   kenn
   kenn [at] apache [dot] org
@@ -267,6 +285,15 @@
 
   
 
+  Stas Levin
+  staslevin
+  staslevin [at] apache [dot] org
+  PayPal
+  committer
+  +2
+
+  
+
   Maximilian Michels
   mxm
   mxm [at] apache [dot] org



[1/3] beam-site git commit: Add new committers to the team.

2017-01-31 Thread davor
Repository: beam-site
Updated Branches:
  refs/heads/asf-site 689c36863 -> 7f9daf5d3


Add new committers to the team.


Project: http://git-wip-us.apache.org/repos/asf/beam-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam-site/commit/f67cdd1a
Tree: http://git-wip-us.apache.org/repos/asf/beam-site/tree/f67cdd1a
Diff: http://git-wip-us.apache.org/repos/asf/beam-site/diff/f67cdd1a

Branch: refs/heads/asf-site
Commit: f67cdd1a1b28242172b77bff33533215482d7a50
Parents: 689c368
Author: Ahmet Altay 
Authored: Tue Jan 31 14:29:28 2017 -0800
Committer: Ahmet Altay 
Committed: Tue Jan 31 14:29:28 2017 -0800

--
 src/_beam_team/team.md | 18 ++
 1 file changed, 18 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/beam-site/blob/f67cdd1a/src/_beam_team/team.md
--
diff --git a/src/_beam_team/team.md b/src/_beam_team/team.md
index 70c895d..7028d3e 100644
--- a/src/_beam_team/team.md
+++ b/src/_beam_team/team.md
@@ -8,6 +8,12 @@ members:
 organization: Google
 roles: committer, PMC
 time_zone: "-8"
+  - name: Ahmet Altay
+apache_id: altay
+email: altay [at] apache [dot] org
+organization: Google
+roles: committer
+time_zone: "-8"
   - name: Jesse Anderson
 apache_id: jesseanderson
 email: jesseanderson [at] apache [dot] org
@@ -56,6 +62,12 @@ members:
 organization: Google
 roles: committer, PMC
 time_zone: "-8"
+  - name: Pei He
+apache_id: pei
+email: pei [at] apache [dot] org
+organization: Google
+roles: committer
+time_zone: "-8"
   - name: Kenneth Knowles
 apache_id: kenn
 email: kenn [at] apache [dot] org
@@ -68,6 +80,12 @@ members:
 organization: data Artisans
 roles: committer, PMC
 time_zone: "+1"
+  - name: Stas Levin
+apache_id: staslevin
+email: staslevin [at] apache [dot] org
+organization: PayPal
+roles: committer
+time_zone: "+2"
   - name: Maximilian Michels
 apache_id: mxm
 email: mxm [at] apache [dot] org



[3/3] beam-site git commit: This closes #138

2017-01-31 Thread davor
This closes #138


Project: http://git-wip-us.apache.org/repos/asf/beam-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam-site/commit/7f9daf5d
Tree: http://git-wip-us.apache.org/repos/asf/beam-site/tree/7f9daf5d
Diff: http://git-wip-us.apache.org/repos/asf/beam-site/diff/7f9daf5d

Branch: refs/heads/asf-site
Commit: 7f9daf5d3e8c32f8cb560a49db4ed8f909df3d37
Parents: 689c368 66c0a70
Author: Davor Bonaci 
Authored: Tue Jan 31 14:38:28 2017 -0800
Committer: Davor Bonaci 
Committed: Tue Jan 31 14:38:28 2017 -0800

--
 content/contribute/team/index.html | 27 +++
 src/_beam_team/team.md | 18 ++
 2 files changed, 45 insertions(+)
--




[GitHub] beam-site pull request #138: Add new committers to the team.

2017-01-31 Thread aaltay
GitHub user aaltay opened a pull request:

https://github.com/apache/beam-site/pull/138

Add new committers to the team.

R: @davorbonaci 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/aaltay/beam-site asf-site

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam-site/pull/138.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #138


commit f67cdd1a1b28242172b77bff33533215482d7a50
Author: Ahmet Altay 
Date:   2017-01-31T22:29:28Z

Add new committers to the team.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] beam-site pull request #138: Add new committers to the team.

2017-01-31 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/beam-site/pull/138


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] beam-site pull request #137: Cleanup of links: relative instead of absolute,...

2017-01-31 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/beam-site/pull/137


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[4/4] beam-site git commit: This closes #137

2017-01-31 Thread davor
This closes #137


Project: http://git-wip-us.apache.org/repos/asf/beam-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam-site/commit/5b9e9ef6
Tree: http://git-wip-us.apache.org/repos/asf/beam-site/tree/5b9e9ef6
Diff: http://git-wip-us.apache.org/repos/asf/beam-site/diff/5b9e9ef6

Branch: refs/heads/asf-site
Commit: 5b9e9ef69b934232167d0afad3c609fdec82b89b
Parents: 7f9daf5 63f4840
Author: Davor Bonaci 
Authored: Tue Jan 31 14:39:27 2017 -0800
Committer: Davor Bonaci 
Committed: Tue Jan 31 14:39:27 2017 -0800

--
 README.md   | 13 ++---
 _config.yml |  2 +-
 .../2016/03/17/capability-matrix.html   |  4 +-
 .../2016/04/03/presentation-materials.html  |  4 +-
 .../sdk/2016/02/25/python-sdk-now-public.html   |  4 +-
 .../beam/release/2016/06/15/first-release.html  |  4 +-
 .../10/11/strata-hadoop-world-and-beam.html |  4 +-
 .../website/2016/02/22/beam-has-a-logo.html |  4 +-
 .../blog/2016/05/18/splitAtFraction-method.html |  4 +-
 .../05/27/where-is-my-pcollection-dot-map.html  |  4 +-
 .../06/13/flink-batch-runner-milestone.html |  4 +-
 content/blog/2016/08/03/six-months.html |  4 +-
 content/blog/2016/10/20/test-stream.html|  4 +-
 content/blog/2017/01/09/added-apex-runner.html  | 10 ++--
 content/blog/2017/01/10/beam-graduates.html |  4 +-
 content/blog/index.html |  6 +--
 content/capability-matrix/index.html|  8 ++--
 content/coming-soon.html|  4 +-
 .../contribute/contribution-guide/index.html|  4 +-
 content/contribute/design-principles/index.html |  4 +-
 content/contribute/index.html   |  4 +-
 content/contribute/logos/index.html |  4 +-
 content/contribute/maturity-model/index.html|  4 +-
 .../presentation-materials/index.html   |  4 +-
 .../ptransform-style-guide/index.html   |  4 +-
 content/contribute/release-guide/index.html |  4 +-
 content/contribute/source-repository/index.html |  4 +-
 content/contribute/team/index.html  |  4 +-
 content/contribute/testing/index.html   |  4 +-
 content/contribute/work-in-progress/index.html  |  4 +-
 content/contribution-guide/index.html   |  8 ++--
 content/docs/contribute.html|  8 ++--
 content/docs/learn/index.html   |  8 ++--
 content/docs/learn/programming-guide/index.html |  8 ++--
 content/docs/use/beam-overview/index.html   |  8 ++--
 content/docs/use/index.html |  8 ++--
 content/documentation/index.html|  4 +-
 .../pipelines/create-your-pipeline/index.html   |  4 +-
 .../pipelines/design-your-pipeline/index.html   |  4 +-
 .../pipelines/test-your-pipeline/index.html |  4 +-
 .../documentation/programming-guide/index.html  |  4 +-
 content/documentation/resources/index.html  |  4 +-
 content/documentation/runners/apex/index.html   |  4 +-
 .../runners/capability-matrix/index.html|  4 +-
 .../documentation/runners/dataflow/index.html   |  4 +-
 content/documentation/runners/direct/index.html |  4 +-
 content/documentation/runners/flink/index.html  |  4 +-
 content/documentation/runners/spark/index.html  |  4 +-
 content/documentation/sdks/java/index.html  |  4 +-
 content/documentation/sdks/python/index.html|  4 +-
 content/downloads/beam-doap.rdf | 10 ++--
 content/feed.xml| 50 ++--
 content/get-started/beam-overview/index.html|  4 +-
 content/get-started/downloads/index.html|  4 +-
 content/get-started/index.html  |  4 +-
 .../mobile-gaming-example/index.html|  4 +-
 content/get-started/quickstart-java/index.html  |  4 +-
 content/get-started/quickstart-py/index.html|  4 +-
 content/get-started/quickstart/index.html   |  8 ++--
 content/get-started/releases/index.html |  8 ++--
 content/get-started/support/index.html  |  4 +-
 .../get-started/wordcount-example/index.html|  4 +-
 content/getting-started/index.html  |  8 ++--
 content/index.html  |  4 +-
 content/learn/index.html|  8 ++--
 content/learn/presentation-materials/index.html |  8 ++--
 content/learn/programming-guide/index.html  |  8 ++--
 content/learn/resources/index.html  |  8 ++--
 .../learn/runners/capability-matrix/index.html  |  8 ++--
 content/learn/runners/dataflow/index.html   |  8 ++--
 content/learn/runners/direct/index.html |  8 ++--
 content/learn/runners/flink/index.html  |  8 ++--
 content/learn/runners/spark/index.html  |  8 ++--
 content/learn/sdks/java/index.html  |  8 ++--
 content/learn/sdks/javadoc/index.html   |  8 ++--
 content/material/index.html |  8 ++--

[2/4] beam-site git commit: Regenerate website

2017-01-31 Thread davor
http://git-wip-us.apache.org/repos/asf/beam-site/blob/63f48409/content/get-started/index.html
--
diff --git a/content/get-started/index.html b/content/get-started/index.html
index 76ba604..69a71c0 100644
--- a/content/get-started/index.html
+++ b/content/get-started/index.html
@@ -15,8 +15,8 @@
   https://ajax.googleapis.com/ajax/libs/jquery/2.2.0/jquery.min.js";>
   
   
-  http://beam.apache.org/get-started/"; 
data-proofer-ignore>
-  http://beam.apache.org/feed.xml";>
+  https://beam.apache.org/get-started/"; 
data-proofer-ignore>
+  https://beam.apache.org/feed.xml";>
   
 
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
 (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new 
Date();a=s.createElement(o),

http://git-wip-us.apache.org/repos/asf/beam-site/blob/63f48409/content/get-started/mobile-gaming-example/index.html
--
diff --git a/content/get-started/mobile-gaming-example/index.html 
b/content/get-started/mobile-gaming-example/index.html
index ced93cb..6c46596 100644
--- a/content/get-started/mobile-gaming-example/index.html
+++ b/content/get-started/mobile-gaming-example/index.html
@@ -15,8 +15,8 @@