[GitHub] incubator-beam pull request #1369: [Ignore] DatastoreIO with ValueProvider

GitHub user vikkyrk opened a pull request: https://github.com/apache/incubator-beam/pull/1369 [Ignore] DatastoreIO with ValueProvider Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatte

[jira] [Commented] (BEAM-825) Fill in the documentation/runners/apex portion of the website

[ https://issues.apache.org/jira/browse/BEAM-825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15669868#comment-15669868 ] Daniel Halperin commented on BEAM-825: -- Hi [~thw], I hope that as a committer you shoul

[jira] [Created] (BEAM-990) KafkaIO does not commit offsets to Kafka

Alban Perillat-Merceroz created BEAM-990: Summary: KafkaIO does not commit offsets to Kafka Key: BEAM-990 URL: https://issues.apache.org/jira/browse/BEAM-990 Project: Beam Issue Type:

[jira] [Commented] (BEAM-302) Add Scio Scala DSL to Beam

[ https://issues.apache.org/jira/browse/BEAM-302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15670335#comment-15670335 ] Jean-Baptiste Onofré commented on BEAM-302: --- Resuming tests and changes on Scio.

[GitHub] incubator-beam pull request #1370: [BEAM-918] Allow users to define the stor...

GitHub user jbonofre opened a pull request: https://github.com/apache/incubator-beam/pull/1370 [BEAM-918] Allow users to define the storage level via pipeline options Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [X] M

[jira] [Commented] (BEAM-918) Let users set STORAGE_LEVEL via SparkPipelineOptions.

[ https://issues.apache.org/jira/browse/BEAM-918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15670729#comment-15670729 ] ASF GitHub Bot commented on BEAM-918: - GitHub user jbonofre opened a pull request:

[jira] [Commented] (BEAM-825) Fill in the documentation/runners/apex portion of the website

[ https://issues.apache.org/jira/browse/BEAM-825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15670760#comment-15670760 ] Daniel Halperin commented on BEAM-825: -- I believe Thomas is traveling now, so I will go

[jira] [Updated] (BEAM-825) Fill in the documentation/runners/apex portion of the website

[ https://issues.apache.org/jira/browse/BEAM-825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin updated BEAM-825: - Assignee: Sandeep Deshmukh > Fill in the documentation/runners/apex portion of the website > ---

[jira] [Commented] (BEAM-825) Fill in the documentation/runners/apex portion of the website

[ https://issues.apache.org/jira/browse/BEAM-825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15670773#comment-15670773 ] Thomas Weise commented on BEAM-825: --- Just arrived and it is good you made the change (add

[jira] [Updated] (BEAM-825) Fill in the documentation/runners/apex portion of the website

[ https://issues.apache.org/jira/browse/BEAM-825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated BEAM-825: -- Issue Type: Task (was: Bug) > Fill in the documentation/runners/apex portion of the website > ---

[GitHub] incubator-beam pull request #1371: [Ignore] Change build order to first pick...

GitHub user vikkyrk opened a pull request: https://github.com/apache/incubator-beam/pull/1371 [Ignore] Change build order to first pick up python Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[jira] [Resolved] (BEAM-957) Flake in FlattenTest under DirectRunner

[ https://issues.apache.org/jira/browse/BEAM-957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-957. -- Resolution: Fixed Fix Version/s: 0.4.0-incubating > Flake in FlattenTest under DirectRunner > -

[3/3] incubator-beam-site git commit: This closes #79

This closes #79 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/commit/4e6947c6 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/tree/4e6947c6 Diff: http://git-wip-us.apache.org/repos/asf/

[1/3] incubator-beam-site git commit: [BEAM-899] Add Flink Instructions to quickstart.md

Repository: incubator-beam-site Updated Branches: refs/heads/asf-site 3b93d1b56 -> 4e6947c63 [BEAM-899] Add Flink Instructions to quickstart.md Project: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/commit/a

[2/3] incubator-beam-site git commit: Regenerate website

Regenerate website Project: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/commit/bfbd64c3 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/tree/bfbd64c3 Diff: http://git-wip-us.apache.org/repos/a

[jira] [Resolved] (BEAM-839) The DirectRunner slows down significantly as the number of keys increases

[ https://issues.apache.org/jira/browse/BEAM-839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-839. -- Resolution: Fixed Fix Version/s: 0.4.0-incubating This was linear time (with regards to the number

[GitHub] incubator-beam-site pull request #79: [BEAM-899] Add Flink Instructions to q...

Github user asfgit closed the pull request at: https://github.com/apache/incubator-beam-site/pull/79 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fea

[jira] [Commented] (BEAM-899) Flink quickstart instructions

[ https://issues.apache.org/jira/browse/BEAM-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15671086#comment-15671086 ] ASF GitHub Bot commented on BEAM-899: - Github user asfgit closed the pull request at:

[GitHub] incubator-beam-site pull request #87: Remove Apex runner from work in progre...

GitHub user kennknowles opened a pull request: https://github.com/apache/incubator-beam-site/pull/87 Remove Apex runner from work in progress R: @davorbonaci You can merge this pull request into a Git repository by running: $ git pull https://github.com/kennknowles/incubator-b

[jira] [Created] (BEAM-991) DatastoreIO Write should flush early for large batches

Vikas Kedigehalli created BEAM-991: -- Summary: DatastoreIO Write should flush early for large batches Key: BEAM-991 URL: https://issues.apache.org/jira/browse/BEAM-991 Project: Beam Issue Typ

[3/3] incubator-beam-site git commit: This closes #87

This closes #87 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/commit/f5a08479 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/tree/f5a08479 Diff: http://git-wip-us.apache.org/repos/asf/

[GitHub] incubator-beam-site pull request #87: Remove Apex runner from work in progre...

Github user asfgit closed the pull request at: https://github.com/apache/incubator-beam-site/pull/87 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fea

[1/3] incubator-beam-site git commit: Remove Apex runner from work in progress

Repository: incubator-beam-site Updated Branches: refs/heads/asf-site 4e6947c63 -> f5a084799 Remove Apex runner from work in progress Project: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/commit/99ad7878 Tr

[2/3] incubator-beam-site git commit: Regenerate website

Regenerate website Project: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/commit/c219b493 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/tree/c219b493 Diff: http://git-wip-us.apache.org/repos/a

[2/2] incubator-beam git commit: This closes #1254

This closes #1254 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/938ac91a Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/938ac91a Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam

[1/2] incubator-beam git commit: Exercise Dynamic Splitting in the DirectRunner

Repository: incubator-beam Updated Branches: refs/heads/master dc94dbdd7 -> 938ac91a9 Exercise Dynamic Splitting in the DirectRunner For sources that are above a certain size, the DirectRunner will run a Thread which will split off half of the remaining work. This exercises the concurrent beha

[jira] [Commented] (BEAM-833) Exercise Dynamic Splitting in the DirectRunner

[ https://issues.apache.org/jira/browse/BEAM-833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15671180#comment-15671180 ] ASF GitHub Bot commented on BEAM-833: - Github user asfgit closed the pull request at:

[jira] [Closed] (BEAM-899) Flink quickstart instructions

[ https://issues.apache.org/jira/browse/BEAM-899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aljoscha Krettek closed BEAM-899. - Resolution: Fixed Fix Version/s: 0.4.0-incubating > Flink quickstart instructions > -

[GitHub] incubator-beam pull request #1254: [BEAM-833] Exercise Dynamic Splitting in ...

Github user asfgit closed the pull request at: https://github.com/apache/incubator-beam/pull/1254 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the featur

[jira] [Resolved] (BEAM-833) Exercise Dynamic Splitting in the DirectRunner

[ https://issues.apache.org/jira/browse/BEAM-833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-833. -- Resolution: Fixed Fix Version/s: 0.4.0-incubating > Exercise Dynamic Splitting in the DirectRunner

[2/2] incubator-beam git commit: This closes #1363

This closes #1363 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/c695ef48 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/c695ef48 Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam

[1/2] incubator-beam git commit: Remove Pipeline#getFullNameForTesting

Repository: incubator-beam Updated Branches: refs/heads/master 938ac91a9 -> c695ef48b Remove Pipeline#getFullNameForTesting Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/1cdaa4e7 Tree: http://git-wip-us

[GitHub] incubator-beam pull request #1363: Remove Pipeline#getFullNameForTesting

Github user asfgit closed the pull request at: https://github.com/apache/incubator-beam/pull/1363 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the featur

[jira] [Created] (BEAM-993) Python SDK build broken due to datastore dependency

Vikas Kedigehalli created BEAM-993: -- Summary: Python SDK build broken due to datastore dependency Key: BEAM-993 URL: https://issues.apache.org/jira/browse/BEAM-993 Project: Beam Issue Type:

[jira] [Created] (BEAM-992) Python SDK build broken due to datastore dependency

Vikas Kedigehalli created BEAM-992: -- Summary: Python SDK build broken due to datastore dependency Key: BEAM-992 URL: https://issues.apache.org/jira/browse/BEAM-992 Project: Beam Issue Type:

[GitHub] incubator-beam pull request #1372: Remove datastore dependency and disable t...

GitHub user vikkyrk opened a pull request: https://github.com/apache/incubator-beam/pull/1372 Remove datastore dependency and disable tests Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is fo

[2/4] incubator-beam git commit: Remove unused Thread variable in TransformExecutor

Remove unused Thread variable in TransformExecutor Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/5ba4d181 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/5ba4d181 Diff: http://git-wip-us.ap

[3/4] incubator-beam git commit: Reduce Visibility of PCollectionViewWriter

Reduce Visibility of PCollectionViewWriter This is an internal implementation detail of the DirectRunner and should not be exposed. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/bf4c504d Tree: http://git-w

[4/4] incubator-beam git commit: Reduce incidence of Namespace StringKey comparisons

Reduce incidence of Namespace StringKey comparisons If the Namespace of a TimerData reports itself as being equal to the other namespace, immediately return 0 rather than generating the string keys and comparing them. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: ht

[1/4] incubator-beam git commit: This closes #1319

Repository: incubator-beam Updated Branches: refs/heads/master c695ef48b -> 15e93c58e This closes #1319 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/15e93c58 Tree: http://git-wip-us.apache.org/repos/as

[GitHub] incubator-beam pull request #1319: Misc TimerData, TransformExecutor improve...

Github user asfgit closed the pull request at: https://github.com/apache/incubator-beam/pull/1319 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the featur

[GitHub] incubator-beam pull request #1371: Change maven build order to first pick up...

Github user vikkyrk closed the pull request at: https://github.com/apache/incubator-beam/pull/1371 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the featu

[jira] [Created] (BEAM-994) Support for S3 file source and sink

Sourabh Bajaj created BEAM-994: -- Summary: Support for S3 file source and sink Key: BEAM-994 URL: https://issues.apache.org/jira/browse/BEAM-994 Project: Beam Issue Type: New Feature

[jira] [Commented] (BEAM-25) Add user-ready API for interacting with state

[ https://issues.apache.org/jira/browse/BEAM-25?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15671914#comment-15671914 ] ASF GitHub Bot commented on BEAM-25: Github user kennknowles closed the pull request at:

[GitHub] incubator-beam pull request #1347: [BEAM-25] Allow stateful DoFn in DirectRu...

Github user kennknowles closed the pull request at: https://github.com/apache/incubator-beam/pull/1347 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the f

[GitHub] incubator-beam pull request #1373: Properly uses result of transform overrid...

GitHub user jkff opened a pull request: https://github.com/apache/incubator-beam/pull/1373 Properly uses result of transform override Previously direct runner would use the transform override to .apply(), but would keep the original transform in the pipeline, e.g. it would use

Jenkins build became unstable: beam_PostCommit_RunnableOnService_SparkLocal #164

See

Jenkins build became unstable: beam_PostCommit_MavenVerify #1838

See

[1/2] incubator-beam git commit: Properly apply Transform Overrides in the Direct Runner

Repository: incubator-beam Updated Branches: refs/heads/master 15e93c58e -> abd9fb3d7 Properly apply Transform Overrides in the Direct Runner Previously the direct runner would use the transform override to .apply(), but would keep the original transform in the pipeline, e.g. it would use the

[GitHub] incubator-beam pull request #1373: Properly uses result of transform overrid...

Github user asfgit closed the pull request at: https://github.com/apache/incubator-beam/pull/1373 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the featur

[2/2] incubator-beam git commit: This closes #1373

This closes #1373 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/abd9fb3d Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/abd9fb3d Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam

Jenkins build is back to stable : beam_PostCommit_RunnableOnService_SparkLocal #165

See

Jenkins build is back to stable : beam_PostCommit_MavenVerify #1839

See

[GitHub] incubator-beam pull request #1264: [BEAM-884] Adding display data to Pipelin...

Github user pabloem closed the pull request at: https://github.com/apache/incubator-beam/pull/1264 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the featu

[jira] [Commented] (BEAM-884) Add Display Data to the Python SDK's PipelineOptions, Avro io and other transforms

[ https://issues.apache.org/jira/browse/BEAM-884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15672717#comment-15672717 ] ASF GitHub Bot commented on BEAM-884: - Github user pabloem closed the pull request at:

Jenkins build became unstable: beam_PostCommit_RunnableOnService_SparkLocal #167

See