[jira] [Commented] (BEAM-461) Create custom Kinesis source

[ https://issues.apache.org/jira/browse/BEAM-461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15383977#comment-15383977 ] ASF GitHub Bot commented on BEAM-461: - GitHub user ppastuszkaocado opened a pull request

[GitHub] incubator-beam pull request #687: [BEAM-461] Create custom Kinesis source

GitHub user ppastuszkaocado opened a pull request: https://github.com/apache/incubator-beam/pull/687 [BEAM-461] Create custom Kinesis source This adds custom Kinesis source as new beam-sdks-java-io-kinesis module. Build on Travis has passed. This has been tested with uni

[jira] [Created] (BEAM-470) Spark Runner does not send the job execution information into the Spark History Server

Ismaël Mejía created BEAM-470: - Summary: Spark Runner does not send the job execution information into the Spark History Server Key: BEAM-470 URL: https://issues.apache.org/jira/browse/BEAM-470 Project: B

[jira] [Commented] (BEAM-470) Spark Runner does not send the job execution information into the Spark History Server

[ https://issues.apache.org/jira/browse/BEAM-470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15384390#comment-15384390 ] Amit Sela commented on BEAM-470: This is something to consider. On one hand, you could say t

[2/2] incubator-beam git commit: Closes #667

Closes #667 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/b75c0c0d Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/b75c0c0d Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/

[1/2] incubator-beam git commit: Adds a test harnesses and utilities framework for sources.

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 062af66f8 -> b75c0c0d4 Adds a test harnesses and utilities framework for sources. Helper functions and test harnesses for checking correctness of source (``iobase.BoundedSource``) and range tracker (``iobase.RangeTracker``) imp

[jira] [Created] (BEAM-471) RAT misconfigured except in release profile

Kenneth Knowles created BEAM-471: Summary: RAT misconfigured except in release profile Key: BEAM-471 URL: https://issues.apache.org/jira/browse/BEAM-471 Project: Beam Issue Type: Bug

[jira] [Created] (BEAM-472) RAT configuration fails on Eclipse files in subprojects

Kenneth Knowles created BEAM-472: Summary: RAT configuration fails on Eclipse files in subprojects Key: BEAM-472 URL: https://issues.apache.org/jira/browse/BEAM-472 Project: Beam Issue Type:

[GitHub] incubator-beam pull request #688: [BEAM-471][BEAM-472] Fixes to RAT configur...

GitHub user kennknowles opened a pull request: https://github.com/apache/incubator-beam/pull/688 [BEAM-471][BEAM-472] Fixes to RAT configuration Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[jira] [Commented] (BEAM-471) RAT misconfigured except in release profile

[ https://issues.apache.org/jira/browse/BEAM-471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15384552#comment-15384552 ] ASF GitHub Bot commented on BEAM-471: - GitHub user kennknowles opened a pull request:

[jira] [Commented] (BEAM-455) Implement a test harness for verifying correctness of custom sources

[ https://issues.apache.org/jira/browse/BEAM-455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15384568#comment-15384568 ] ASF GitHub Bot commented on BEAM-455: - Github user chamikaramj closed the pull request a

[GitHub] incubator-beam pull request #667: [BEAM-455] Adds a test harnesses and utili...

Github user chamikaramj closed the pull request at: https://github.com/apache/incubator-beam/pull/667 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fe

[GitHub] incubator-beam pull request #689: Express ParDo.Bound in terms of ParDo.Boun...

GitHub user tgroh opened a pull request: https://github.com/apache/incubator-beam/pull/689 Express ParDo.Bound in terms of ParDo.BoundMulti Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is fo

[jira] [Created] (BEAM-473) DatastoreIO querying latest timestamp statistics doesn't work with namespace

Vikas Kedigehalli created BEAM-473: -- Summary: DatastoreIO querying latest timestamp statistics doesn't work with namespace Key: BEAM-473 URL: https://issues.apache.org/jira/browse/BEAM-473 Project: B

[jira] [Reopened] (BEAM-145) OutputTimeFn#assignOutputTime overrides WindowFn#getOutputTime in unfortunate ways

[ https://issues.apache.org/jira/browse/BEAM-145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles reopened BEAM-145: -- There's an unfortunate mutual default-invocation loop left over from the fix to this. It will go away

[GitHub] incubator-beam pull request #690: [BEAM-452] Add DoFn setup and teardown met...

GitHub user tgroh opened a pull request: https://github.com/apache/incubator-beam/pull/690 [BEAM-452] Add DoFn setup and teardown methods Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is form

[jira] [Commented] (BEAM-452) Implement DoFn per-instance setup and teardown methods

[ https://issues.apache.org/jira/browse/BEAM-452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15384666#comment-15384666 ] ASF GitHub Bot commented on BEAM-452: - GitHub user tgroh opened a pull request: htt

[GitHub] incubator-beam pull request #691: Add low-level data structures to be shared...

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/691 Add low-level data structures to be shared with the worker. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the

[jira] [Created] (BEAM-474) Tests which evaluate DisplayData but don't run a pipeline should be marked @NeedsRunner not @RunnableOnService

Luke Cwik created BEAM-474: -- Summary: Tests which evaluate DisplayData but don't run a pipeline should be marked @NeedsRunner not @RunnableOnService Key: BEAM-474 URL: https://issues.apache.org/jira/browse/BEAM-474

[2/3] incubator-beam git commit: Correctly configure RAT in all profiles

Correctly configure RAT in all profiles Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/1e162128 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/1e162128 Diff: http://git-wip-us.apache.org/re

[1/3] incubator-beam git commit: Add fixed Eclipse RAT excludes

Repository: incubator-beam Updated Branches: refs/heads/master 69e22023f -> eb69e107d Add fixed Eclipse RAT excludes Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/fffa7996 Tree: http://git-wip-us.apache

[3/3] incubator-beam git commit: This closes #688

This closes #688 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/eb69e107 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/eb69e107 Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/

[jira] [Commented] (BEAM-471) RAT misconfigured except in release profile

[ https://issues.apache.org/jira/browse/BEAM-471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15384788#comment-15384788 ] ASF GitHub Bot commented on BEAM-471: - Github user asfgit closed the pull request at:

[GitHub] incubator-beam pull request #688: [BEAM-471][BEAM-472] Fixes to RAT configur...

Github user asfgit closed the pull request at: https://github.com/apache/incubator-beam/pull/688 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[jira] [Resolved] (BEAM-472) RAT configuration fails on Eclipse files in subprojects

[ https://issues.apache.org/jira/browse/BEAM-472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles resolved BEAM-472. -- Resolution: Fixed Fix Version/s: Not applicable > RAT configuration fails on Eclipse files

[jira] [Resolved] (BEAM-471) RAT misconfigured except in release profile

[ https://issues.apache.org/jira/browse/BEAM-471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles resolved BEAM-471. -- Resolution: Fixed Fix Version/s: Not applicable > RAT misconfigured except in release profi

[jira] [Created] (BEAM-475) High-quality javadoc for Beam

Daniel Halperin created BEAM-475: Summary: High-quality javadoc for Beam Key: BEAM-475 URL: https://issues.apache.org/jira/browse/BEAM-475 Project: Beam Issue Type: Improvement Comp

[GitHub] incubator-beam pull request #692: [BEAM-475] Checkstyle: enforce package-inf...

GitHub user dhalperi opened a pull request: https://github.com/apache/incubator-beam/pull/692 [BEAM-475] Checkstyle: enforce package-info.java presence R: @kennknowles for runners-core and misc R: @amitsela for Spark You can merge this pull request into a Git repository

[jira] [Commented] (BEAM-475) High-quality javadoc for Beam

[ https://issues.apache.org/jira/browse/BEAM-475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15384802#comment-15384802 ] ASF GitHub Bot commented on BEAM-475: - GitHub user dhalperi opened a pull request:

[GitHub] incubator-beam pull request #693: [BEAM-473] Fix stats querying for Datastor...

GitHub user vikkyrk opened a pull request: https://github.com/apache/incubator-beam/pull/693 [BEAM-473] Fix stats querying for DatastoreIO Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is for

[jira] [Commented] (BEAM-473) DatastoreIO querying latest timestamp statistics doesn't work with namespace

[ https://issues.apache.org/jira/browse/BEAM-473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15384928#comment-15384928 ] ASF GitHub Bot commented on BEAM-473: - GitHub user vikkyrk opened a pull request: h

[GitHub] incubator-beam pull request #694: Remove "beamhead-02" container workaround

GitHub user charlesccychen opened a pull request: https://github.com/apache/incubator-beam/pull/694 Remove "beamhead-02" container workaround You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/incubator-beam remove-b

[GitHub] incubator-beam pull request #695: BEAM-372 added a test that verifies if a c...

GitHub user chandnisingh opened a pull request: https://github.com/apache/incubator-beam/pull/695 BEAM-372 added a test that verifies if a coder consumes bytes equal t… Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [

[jira] [Commented] (BEAM-372) CoderProperties: Test that the coder doesn't consume more bytes than it produces

[ https://issues.apache.org/jira/browse/BEAM-372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15385138#comment-15385138 ] ASF GitHub Bot commented on BEAM-372: - GitHub user chandnisingh opened a pull request:

[GitHub] incubator-beam pull request #696: Cythonize WindowedValue class.

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/696 Cythonize WindowedValue class. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like:

[1/2] incubator-beam git commit: Closes #686

Repository: incubator-beam Updated Branches: refs/heads/python-sdk b75c0c0d4 -> ce29ac7ab Closes #686 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/ce29ac7a Tree: http://git-wip-us.apache.org/repos/asf/

[2/2] incubator-beam git commit: Check type of coder for step feeding into GroupByKey in Dataflow runner

Check type of coder for step feeding into GroupByKey in Dataflow runner Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/e3b53028 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/e3b53028 Diff:

[GitHub] incubator-beam pull request #697: [BEAM-362] [BEAM-25] Implement GroupAlsoBy...

GitHub user kennknowles opened a pull request: https://github.com/apache/incubator-beam/pull/697 [BEAM-362] [BEAM-25] Implement GroupAlsoByWindows by directly providing StateInternals Be sure to do all of the following to help us incorporate your contribution quickly and easily:

[jira] [Commented] (BEAM-362) Move shared runner functionality out of SDK and into runners/core-java

[ https://issues.apache.org/jira/browse/BEAM-362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15385262#comment-15385262 ] ASF GitHub Bot commented on BEAM-362: - GitHub user kennknowles opened a pull request:

[GitHub] incubator-beam pull request #698: Minor javadoc fixes

GitHub user dhalperi opened a pull request: https://github.com/apache/incubator-beam/pull/698 Minor javadoc fixes You can merge this pull request into a Git repository by running: $ git pull https://github.com/dhalperi/incubator-beam javadoc-fixes Alternatively you can review

[2/2] incubator-beam git commit: Closes #672

Closes #672 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/71214b37 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/71214b37 Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/

[1/2] incubator-beam git commit: Adds a PTransform for Avro source.

Repository: incubator-beam Updated Branches: refs/heads/python-sdk ce29ac7ab -> 71214b37a Adds a PTransform for Avro source. Wrapping a custom source as a PTransform is better than directly using the source using df.Read since the PTransform can be extended without breaking end-user code. U

[GitHub] incubator-beam pull request #672: [BEAM-360] Adds a PTransform for Avro sour...

Github user chamikaramj closed the pull request at: https://github.com/apache/incubator-beam/pull/672 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fe

[jira] [Commented] (BEAM-360) Add a framework for creating Python-SDK sources for new file types

[ https://issues.apache.org/jira/browse/BEAM-360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15385300#comment-15385300 ] ASF GitHub Bot commented on BEAM-360: - Github user chamikaramj closed the pull request a

[GitHub] incubator-beam pull request #699: [BEAM-362] Move aggregator-scraping to run...

GitHub user kennknowles opened a pull request: https://github.com/apache/incubator-beam/pull/699 [BEAM-362] Move aggregator-scraping to runners-core Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR ti

[jira] [Commented] (BEAM-362) Move shared runner functionality out of SDK and into runners/core-java

[ https://issues.apache.org/jira/browse/BEAM-362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15385303#comment-15385303 ] ASF GitHub Bot commented on BEAM-362: - GitHub user kennknowles opened a pull request: