[GitHub] incubator-beam pull request #980: TextIO/CompressedSource: support splitting...

GitHub user dhalperi opened a pull request: https://github.com/apache/incubator-beam/pull/980 TextIO/CompressedSource: support splitting AUTO mode files into bundles R: @jkff please take a look. This code has been the cause of major bugs in the past. I am still planning a

[jira] [Commented] (BEAM-643) Allow users to specify a custom service account

[ https://issues.apache.org/jira/browse/BEAM-643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505641#comment-15505641 ] ASF GitHub Bot commented on BEAM-643: - GitHub user chamikaramj opened a pull request:

[GitHub] incubator-beam pull request #979: [BEAM-643] Updates lint configurations to ...

GitHub user chamikaramj opened a pull request: https://github.com/apache/incubator-beam/pull/979 [BEAM-643] Updates lint configurations to ignore generated files. Adds ability to ignore certain generated files when running pylint and pep8. You can merge this pull request into a

[GitHub] incubator-beam pull request #951: Check Dataflow Job Status Before Terminate

Github user asfgit closed the pull request at: https://github.com/apache/incubator-beam/pull/951 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[jira] [Commented] (BEAM-643) Allow users to specify a custom service account

[ https://issues.apache.org/jira/browse/BEAM-643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505616#comment-15505616 ] ASF GitHub Bot commented on BEAM-643: - GitHub user chamikaramj opened a pull request:

[2/2] incubator-beam git commit: Check Dataflow Job Status Before Terminate

Check Dataflow Job Status Before Terminate This closes #951 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/9e7ed292 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/9e7ed292 Diff:

[1/2] incubator-beam git commit: Check Dataflow Job Status Before Terminate

Repository: incubator-beam Updated Branches: refs/heads/master 9c8e19c1c -> 9e7ed2929 Check Dataflow Job Status Before Terminate Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/e776ae73 Tree:

[GitHub] incubator-beam pull request #978: [BEAM-643] Updates Dataflow API client.

GitHub user chamikaramj opened a pull request: https://github.com/apache/incubator-beam/pull/978 [BEAM-643] Updates Dataflow API client. Updates Cloud Dataflow API client files to the latest version. You can merge this pull request into a Git repository by running: $ git pull

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #1173

See

[GitHub] incubator-beam pull request #977: Support BigQuery BYTES type

GitHub user peihe opened a pull request: https://github.com/apache/incubator-beam/pull/977 Support BigQuery BYTES type You can merge this pull request into a Git repository by running: $ git pull https://github.com/peihe/incubator-beam bq-new-types Alternatively you can

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #1172

See

[GitHub] incubator-beam pull request #976: [BEAM-625] Making Dataflow Python Material...

GitHub user katsiapis opened a pull request: https://github.com/apache/incubator-beam/pull/976 [BEAM-625] Making Dataflow Python Materialized PCollection representation more efficient (5 of several). Changing ToStringCoder to BytesCoder since the former can't be used with sources

[1/2] incubator-beam git commit: Changed ToStringCoder to BytesCoder in test

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 2f09003e3 -> adda16320 Changed ToStringCoder to BytesCoder in test The former can't be used with sources, as it can only encode. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[2/2] incubator-beam git commit: Closes #976

Closes #976 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/adda1632 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/adda1632 Diff:

[1/2] incubator-beam git commit: Use sys.executable and "-m pip" to ensure we use the same Python and pip as the currently running one.

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 29b55e956 -> 2f09003e3 Use sys.executable and "-m pip" to ensure we use the same Python and pip as the currently running one. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[2/2] incubator-beam git commit: Closes #962

Closes #962 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/2f09003e Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/2f09003e Diff:

[2/2] incubator-beam git commit: Removed unnecessary throttling of rename parallelism.

Removed unnecessary throttling of rename parallelism. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/24bb8f19 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/24bb8f19 Diff:

[1/2] incubator-beam git commit: Closes #965

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 4b7fe2dc5 -> 29b55e956 Closes #965 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/29b55e95 Tree:

[jira] [Commented] (BEAM-625) Make Dataflow Python Materialized PCollection representation more efficient

[ https://issues.apache.org/jira/browse/BEAM-625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505137#comment-15505137 ] ASF GitHub Bot commented on BEAM-625: - GitHub user katsiapis opened a pull request:

[GitHub] incubator-beam pull request #973: Deprecate TeardownPolicy for Dataflow serv...

Github user asfgit closed the pull request at: https://github.com/apache/incubator-beam/pull/973 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[2/2] incubator-beam git commit: Closes #973

Closes #973 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/9c8e19c1 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/9c8e19c1 Diff:

[1/2] incubator-beam git commit: Deprecate TeardownPolicy for Dataflow service

Repository: incubator-beam Updated Branches: refs/heads/master 62c56c99b -> 9c8e19c1c Deprecate TeardownPolicy for Dataflow service We are moving towards supporting only TEARDOWN_ALWAYS. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #1171

See

[jira] [Commented] (BEAM-643) Allow users to specify a custom service account

[ https://issues.apache.org/jira/browse/BEAM-643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504940#comment-15504940 ] ASF GitHub Bot commented on BEAM-643: - GitHub user chamikaramj opened a pull request:

[GitHub] incubator-beam pull request #975: [BEAM-643] Adds support for specifying a c...

GitHub user chamikaramj opened a pull request: https://github.com/apache/incubator-beam/pull/975 [BEAM-643] Adds support for specifying a custom service account. Adds support for specifying a custom service account when using DataflowPipelineRunner. Updates Dataflow API

[jira] [Created] (BEAM-643) Allow users to specify a custom service account

Chamikara Jayalath created BEAM-643: --- Summary: Allow users to specify a custom service account Key: BEAM-643 URL: https://issues.apache.org/jira/browse/BEAM-643 Project: Beam Issue Type:

[jira] [Commented] (BEAM-642) FlinkRunner does not support Detached Mode

[ https://issues.apache.org/jira/browse/BEAM-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504855#comment-15504855 ] ASF GitHub Bot commented on BEAM-642: - GitHub user sumitchawla opened a pull request:

[GitHub] incubator-beam pull request #974: BEAM-642 - Support Flink Detached Mode for...

GitHub user sumitchawla opened a pull request: https://github.com/apache/incubator-beam/pull/974 BEAM-642 - Support Flink Detached Mode for JOB execution FlinkRunner gives exception when job is submitted in detached mode. Following code will throw exception: {code}

[jira] [Updated] (BEAM-639) BigtableIO.Read: support for user specified row range

[ https://issues.apache.org/jira/browse/BEAM-639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin updated BEAM-639: - Summary: BigtableIO.Read: support for user specified row range (was: BigtableIO.Read: expose

[jira] [Updated] (BEAM-642) FlinkRunner does not support Detached Mode

[ https://issues.apache.org/jira/browse/BEAM-642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Chawla updated BEAM-642: -- Description: FlinkRunner gives exception when job is submitted in detached mode. Following code will

[jira] [Closed] (BEAM-639) BigtableIO.Read: support for user specified row range

[ https://issues.apache.org/jira/browse/BEAM-639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin closed BEAM-639. Resolution: Fixed Fix Version/s: 0.3.0-incubating > BigtableIO.Read: support for user

[GitHub] incubator-beam pull request #973: Deprecate TeardownPolicy for Dataflow serv...

GitHub user jkff opened a pull request: https://github.com/apache/incubator-beam/pull/973 Deprecate TeardownPolicy for Dataflow service We are moving towards supporting only TEARDOWN_ALWAYS. (This is a clone of

[jira] [Commented] (BEAM-642) FlinkRunner does not support Detached Mode

[ https://issues.apache.org/jira/browse/BEAM-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504842#comment-15504842 ] Sumit Chawla commented on BEAM-642: --- Can somebody please assign this issue to me? > FlinkRunner does not

[jira] [Updated] (BEAM-642) FlinkRunner does not support Detached Mode

[ https://issues.apache.org/jira/browse/BEAM-642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Chawla updated BEAM-642: -- Description: FlinkRunner gives exception when job is submitted in detached mode. Following code will

[jira] [Created] (BEAM-642) FlinkRunner does not support Detached Mode

Sumit Chawla created BEAM-642: - Summary: FlinkRunner does not support Detached Mode Key: BEAM-642 URL: https://issues.apache.org/jira/browse/BEAM-642 Project: Beam Issue Type: Bug

[jira] [Updated] (BEAM-642) FlinkRunner does not support Detached Mode

[ https://issues.apache.org/jira/browse/BEAM-642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Chawla updated BEAM-642: -- Component/s: (was: runner-spark) runner-flink > FlinkRunner does not support

[jira] [Commented] (BEAM-639) BigtableIO.Read: expose withStartKey/withEndKey

[ https://issues.apache.org/jira/browse/BEAM-639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504834#comment-15504834 ] ASF GitHub Bot commented on BEAM-639: - Github user asfgit closed the pull request at:

[GitHub] incubator-beam pull request #972: checkstyle.xml: blacklist imports of two s...

Github user asfgit closed the pull request at: https://github.com/apache/incubator-beam/pull/972 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] incubator-beam pull request #969: [BEAM-639] BigtableIO: add support for use...

Github user asfgit closed the pull request at: https://github.com/apache/incubator-beam/pull/969 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[2/2] incubator-beam git commit: Closes #969

Closes #969 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/62c56c99 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/62c56c99 Diff:

[1/2] incubator-beam git commit: [BEAM-639] BigtableIO: add support for users to scan table subranges

Repository: incubator-beam Updated Branches: refs/heads/master 984b32ff2 -> 62c56c99b [BEAM-639] BigtableIO: add support for users to scan table subranges Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[2/2] incubator-beam git commit: Closes #972

Closes #972 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/984b32ff Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/984b32ff Diff:

[1/2] incubator-beam git commit: checkstyle.xml: blacklist imports of two shaded dependencies

Repository: incubator-beam Updated Branches: refs/heads/master c4036753f -> 984b32ff2 checkstyle.xml: blacklist imports of two shaded dependencies Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/a9b68aa2

[GitHub] incubator-beam pull request #972: checkstyle.xml: blacklist imports of two s...

GitHub user dhalperi opened a pull request: https://github.com/apache/incubator-beam/pull/972 checkstyle.xml: blacklist imports of two shaded dependencies Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure

[jira] [Commented] (BEAM-606) Create MqttIO

[ https://issues.apache.org/jira/browse/BEAM-606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504693#comment-15504693 ] ASF GitHub Bot commented on BEAM-606: - GitHub user jbonofre opened a pull request:

[GitHub] incubator-beam pull request #971: [BEAM-606] Create MqttIO

GitHub user jbonofre opened a pull request: https://github.com/apache/incubator-beam/pull/971 [BEAM-606] Create MqttIO Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [X] Make sure the PR title is formatted like:

[jira] [Commented] (BEAM-640) Update Python SDK README text

[ https://issues.apache.org/jira/browse/BEAM-640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504496#comment-15504496 ] ASF GitHub Bot commented on BEAM-640: - GitHub user hadarhg opened a pull request:

[GitHub] incubator-beam pull request #970: [BEAM-640] Update Python SDK README text

GitHub user hadarhg opened a pull request: https://github.com/apache/incubator-beam/pull/970 [BEAM-640] Update Python SDK README text Update the README for Beam Python SDK: - [x] Remove references to Alpha release - [x] Change "Dataflow" references to "Beam" - [x] Update

[jira] [Created] (BEAM-641) Need to test the generated archetypes projects

Pei He created BEAM-641: --- Summary: Need to test the generated archetypes projects Key: BEAM-641 URL: https://issues.apache.org/jira/browse/BEAM-641 Project: Beam Issue Type: Test Reporter:

[jira] [Updated] (BEAM-641) Need to test the generated archetypes projects

[ https://issues.apache.org/jira/browse/BEAM-641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pei He updated BEAM-641: Component/s: testing > Need to test the generated archetypes projects >

[jira] [Created] (BEAM-640) Update Python SDK README text

Hadar Hod created BEAM-640: -- Summary: Update Python SDK README text Key: BEAM-640 URL: https://issues.apache.org/jira/browse/BEAM-640 Project: Beam Issue Type: Task Components: sdk-py

[jira] [Commented] (BEAM-639) BigtableIO.Read: expose withStartKey/withEndKey

[ https://issues.apache.org/jira/browse/BEAM-639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504349#comment-15504349 ] ASF GitHub Bot commented on BEAM-639: - GitHub user dhalperi opened a pull request:

[GitHub] incubator-beam pull request #969: [BEAM-639] BigtableIO: add support for use...

GitHub user dhalperi opened a pull request: https://github.com/apache/incubator-beam/pull/969 [BEAM-639] BigtableIO: add support for users to scan table subranges Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make

[jira] [Created] (BEAM-639) BigtableIO.Read: expose withStartKey/withEndKey

Daniel Halperin created BEAM-639: Summary: BigtableIO.Read: expose withStartKey/withEndKey Key: BEAM-639 URL: https://issues.apache.org/jira/browse/BEAM-639 Project: Beam Issue Type:

[jira] [Resolved] (BEAM-633) Be able to import Beam codebase in Eclipse and support m2e

[ https://issues.apache.org/jira/browse/BEAM-633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Baptiste Onofré resolved BEAM-633. --- Resolution: Duplicate Fix Version/s: 0.3.0-incubating > Be able to import Beam

[jira] [Commented] (BEAM-635) Release 0.2.0-incubating - Support Flink Release Version 1.1.2

[ https://issues.apache.org/jira/browse/BEAM-635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503779#comment-15503779 ] Jean-Baptiste Onofré commented on BEAM-635: --- I would recommend to wait 0.3.0 release containing

[jira] [Created] (BEAM-638) Add a Window function to create a bounded PCollection from an unbounded one

Jean-Baptiste Onofré created BEAM-638: - Summary: Add a Window function to create a bounded PCollection from an unbounded one Key: BEAM-638 URL: https://issues.apache.org/jira/browse/BEAM-638

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #1169

See

[jira] [Commented] (BEAM-637) WindowedValue$ValueInGlobalWindow is not serializable when using JavaSerializer instead of Kryo

[ https://issues.apache.org/jira/browse/BEAM-637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502734#comment-15502734 ] Amit Sela commented on BEAM-637: By default, the SparkRunner is hard-coded to use Kryo for Serialisation. In

[jira] [Commented] (BEAM-621) Add MapValues and MapKeys functions

[ https://issues.apache.org/jira/browse/BEAM-621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502707#comment-15502707 ] Marco Buccini commented on BEAM-621: Another user opened a similar issue:

[jira] [Created] (BEAM-637) WindowedValue$ValueInGlobalWindow is not serializable when using JavaSerializer instead of Kryo

Stas Levin created BEAM-637: --- Summary: WindowedValue$ValueInGlobalWindow is not serializable when using JavaSerializer instead of Kryo Key: BEAM-637 URL: https://issues.apache.org/jira/browse/BEAM-637

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #1168

See