[GitHub] beam pull request #1756: [BEAM-1188] Python File Verifer For E2E Tests
GitHub user markflyhigh opened a pull request: https://github.com/apache/beam/pull/1756 [BEAM-1188] Python File Verifer For E2E Tests Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- Add FileChecksumVerifier to verify E2E test output file(s) locally or on GCS: - Refactor TestPipeline to be clean and able to get option value by name - Create FileChecksumVerifier with retry when IO failed. - Add FileChecksumVerifier to wordcount e2e test - Create test_utils to hold utility method for testing. Test is done by running wordcount_it against Dataflow service. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markflyhigh/incubator-beam file-checksum-verifier Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/1756.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1756 commit 16109f1651dd9537464687129d8d0c0f3a6a4a91 Author: Mark LiuDate: 2017-01-10T07:48:42Z [BEAM-1188] Python File Verifer For E2E Tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (BEAM-519) fileio.CompressionType requires a __ne__ method
[ https://issues.apache.org/jira/browse/BEAM-519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15814156#comment-15814156 ] Kai Jiang commented on BEAM-519: This has been implemented. https://github.com/apache/beam/blob/81e44b833ce54e44e6506eb028afe8564c46cf18/sdks/python/apache_beam/io/fileio.py#L59 And This issue could be marked as 'RESOLVED' > fileio.CompressionType requires a __ne__ method > > > Key: BEAM-519 > URL: https://issues.apache.org/jira/browse/BEAM-519 > Project: Beam > Issue Type: Bug > Components: sdk-py >Reporter: Ahmet Altay >Priority: Minor > Labels: starter > > This code: > https://github.com/apache/incubator-beam/blob/python-sdk/sdks/python/apache_beam/io/fileio.py#L279 > Without the __ne__ operator instances of this class cannot be used in != > expressions (only ==). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] beam-site pull request #125: Update maturity model to reflect graduation
GitHub user davorbonaci opened a pull request: https://github.com/apache/beam-site/pull/125 Update maturity model to reflect graduation R: @francesperry You can merge this pull request into a Git repository by running: $ git pull https://github.com/davorbonaci/beam-site matmodel Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam-site/pull/125.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #125 commit 6dac95ed25728addd7d09092b169ceef0ca64fbd Author: Davor BonaciDate: 2017-01-10T02:26:12Z Update maturity model to reflect graduation --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam-site pull request #124: Blog post on Beam graduation
GitHub user davorbonaci opened a pull request: https://github.com/apache/beam-site/pull/124 Blog post on Beam graduation R: @francesperry You can merge this pull request into a Git repository by running: $ git pull https://github.com/davorbonaci/beam-site graduation-blog Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam-site/pull/124.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #124 commit 85ba754e20af4495e28932990eac9b978d05cd93 Author: Davor BonaciDate: 2017-01-09T21:12:54Z Blog post on Beam graduation --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #1754: [BEAM] Use the warning module in typecoders to log ...
GitHub user N3da opened a pull request: https://github.com/apache/beam/pull/1754 [BEAM] Use the warning module in typecoders to log only the first occurrence⦠⦠of each typehint related warning message. You can merge this pull request into a Git repository by running: $ git pull https://github.com/N3da/beam python-sdk Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/1754.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1754 commit a064e548dec965b1c4ded21ee4856c09778491ad Author: Neda MirianDate: 2017-01-10T02:21:43Z Use the warning module in typecoders to log only the first occurrence of each typehint related warning message. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (BEAM-1220) Graduation items
[ https://issues.apache.org/jira/browse/BEAM-1220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15813553#comment-15813553 ] ASF GitHub Bot commented on BEAM-1220: -- Github user asfgit closed the pull request at: https://github.com/apache/beam-site/pull/122 > Graduation items > > > Key: BEAM-1220 > URL: https://issues.apache.org/jira/browse/BEAM-1220 > Project: Beam > Issue Type: Task > Components: project-management >Reporter: Davor Bonaci >Assignee: Davor Bonaci > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[3/3] beam-site git commit: This closes #122
This closes #122 Project: http://git-wip-us.apache.org/repos/asf/beam-site/repo Commit: http://git-wip-us.apache.org/repos/asf/beam-site/commit/a46c512c Tree: http://git-wip-us.apache.org/repos/asf/beam-site/tree/a46c512c Diff: http://git-wip-us.apache.org/repos/asf/beam-site/diff/a46c512c Branch: refs/heads/asf-site Commit: a46c512c31d2a8cb4065931f27b5362ccb63494a Parents: fe02314 bb97dd3 Author: Davor BonaciAuthored: Mon Jan 9 18:03:28 2017 -0800 Committer: Davor Bonaci Committed: Mon Jan 9 18:03:28 2017 -0800 -- .../contribute/contribution-guide/index.html| 36 +++ content/contribute/testing/index.html | 9 ++-- src/contribute/contribution-guide.md| 46 ++-- 3 files changed, 45 insertions(+), 46 deletions(-) --
[jira] [Commented] (BEAM-1220) Graduation items
[ https://issues.apache.org/jira/browse/BEAM-1220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15813547#comment-15813547 ] ASF GitHub Bot commented on BEAM-1220: -- Github user asfgit closed the pull request at: https://github.com/apache/beam-site/pull/123 > Graduation items > > > Key: BEAM-1220 > URL: https://issues.apache.org/jira/browse/BEAM-1220 > Project: Beam > Issue Type: Task > Components: project-management >Reporter: Davor Bonaci >Assignee: Davor Bonaci > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[1/2] beam-site git commit: testing.md: fix links to project
Repository: beam-site Updated Branches: refs/heads/asf-site 973bd73a7 -> fe0231494 testing.md: fix links to project Project: http://git-wip-us.apache.org/repos/asf/beam-site/repo Commit: http://git-wip-us.apache.org/repos/asf/beam-site/commit/f128b700 Tree: http://git-wip-us.apache.org/repos/asf/beam-site/tree/f128b700 Diff: http://git-wip-us.apache.org/repos/asf/beam-site/diff/f128b700 Branch: refs/heads/asf-site Commit: f128b700f177f0eefd6b4780c3b3235bf8c89ead Parents: 973bd73 Author: Dan HalperinAuthored: Mon Jan 9 17:56:27 2017 -0800 Committer: Davor Bonaci Committed: Mon Jan 9 18:00:59 2017 -0800 -- src/contribute/testing.md | 9 - 1 file changed, 4 insertions(+), 5 deletions(-) -- http://git-wip-us.apache.org/repos/asf/beam-site/blob/f128b700/src/contribute/testing.md -- diff --git a/src/contribute/testing.md b/src/contribute/testing.md index 6d43337..1d54b0e 100644 --- a/src/contribute/testing.md +++ b/src/contribute/testing.md @@ -244,7 +244,7 @@ details on those testing types. ### Python SDK The Python SDK is currently under development on a feature branch. We have initial -postcommit tests by a Jenkins build; precommit testing and a full testing +postcommit tests by a Jenkins build; precommit testing and a full testing matrix will be coming soon. ## Testing Scenarios @@ -257,8 +257,8 @@ precommit and postcommit testing. For precommit testing, Beam uses [Jenkins](https://builds.apache.org/view/Beam/), -[Travis](http://travis-ci.org/apache/incubator-beam), and a code coverage tool -called [Coveralls](https://coveralls.io/github/apache/incubator-beam), hooked up +[Travis](http://travis-ci.org/apache/beam), and a code coverage tool +called [Coveralls](https://coveralls.io/github/apache/beam), hooked up to [Github](https://github.com/apache/beam), to ensure that pull requests meet a certain quality bar. These precommits verify correctness via two of the below testing tools: unit tests (with coverage monitored by Coveralls) @@ -271,7 +271,7 @@ that this hits the appropriate tradeoff between a desire for short (ideally function in the way in which they are intended. Precommit tests are kicked off when a user makes a Pull Request against the -apache/incubator-beam repository and the Travis, Jenkins, and Coveralls statuses +`apache/beam` repository and the Travis, Jenkins, and Coveralls statuses are displayed at the bottom of the pull request page. Clicking on âDetailsâ will open the status page in the selected tool; there, test status and output can be viewed. @@ -371,4 +371,3 @@ RunnableOnService tests are tests built to use the Beam TestPipeline class, which enables test authors to write simple functionality verification. They are meant to use some of the built-in utilities of the SDK, namely PAssert, to verify that the simple pipelines they run end in the correct state. -
[GitHub] beam-site pull request #123: [BEAM-1220] testing.md: fix links to project
Github user asfgit closed the pull request at: https://github.com/apache/beam-site/pull/123 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[2/2] beam-site git commit: This closes #123
This closes #123 Project: http://git-wip-us.apache.org/repos/asf/beam-site/repo Commit: http://git-wip-us.apache.org/repos/asf/beam-site/commit/fe023149 Tree: http://git-wip-us.apache.org/repos/asf/beam-site/tree/fe023149 Diff: http://git-wip-us.apache.org/repos/asf/beam-site/diff/fe023149 Branch: refs/heads/asf-site Commit: fe02314944300090aa6da6c2595192b7aaef4c0f Parents: 973bd73 f128b70 Author: Davor BonaciAuthored: Mon Jan 9 18:01:08 2017 -0800 Committer: Davor Bonaci Committed: Mon Jan 9 18:01:08 2017 -0800 -- src/contribute/testing.md | 9 - 1 file changed, 4 insertions(+), 5 deletions(-) --
[jira] [Commented] (BEAM-1220) Graduation items
[ https://issues.apache.org/jira/browse/BEAM-1220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15813534#comment-15813534 ] ASF GitHub Bot commented on BEAM-1220: -- GitHub user dhalperi opened a pull request: https://github.com/apache/beam-site/pull/123 [BEAM-1220] testing.md: fix links to project You can merge this pull request into a Git repository by running: $ git pull https://github.com/dhalperi/beam-site testing-pass Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam-site/pull/123.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #123 commit d2079352c84640e4ebf1874c9ae178507bab7350 Author: Dan HalperinDate: 2017-01-10T01:56:27Z testing.md: fix links to project > Graduation items > > > Key: BEAM-1220 > URL: https://issues.apache.org/jira/browse/BEAM-1220 > Project: Beam > Issue Type: Task > Components: project-management >Reporter: Davor Bonaci >Assignee: Davor Bonaci > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] beam-site pull request #122: Pass over the contribution guide
GitHub user dhalperi opened a pull request: https://github.com/apache/beam-site/pull/122 Pass over the contribution guide * Mostly incubating- cleanup * Minor prose fixes * Fix markdown: `something-` does not properly escape ``. You can merge this pull request into a Git repository by running: $ git pull https://github.com/dhalperi/beam-site prose-pass Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam-site/pull/122.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #122 commit 6d55a55c9a1f2e697978b84fbf19b8a74c290f96 Author: Dan HalperinDate: 2017-01-10T01:51:30Z Pass over the contribution guide * Mostly incubating- cleanup * Minor prose fixes * Fix markdown: `something-` does not properly escape ``. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
beam-site git commit: Publish a blog about Apache Apex, second attempt
Repository: beam-site Updated Branches: refs/heads/asf-site ba5fe2b2a -> 973bd73a7 Publish a blog about Apache Apex, second attempt The issue in the previous commit was usage of year 2016, instead of 2017 ;-) Project: http://git-wip-us.apache.org/repos/asf/beam-site/repo Commit: http://git-wip-us.apache.org/repos/asf/beam-site/commit/973bd73a Tree: http://git-wip-us.apache.org/repos/asf/beam-site/tree/973bd73a Diff: http://git-wip-us.apache.org/repos/asf/beam-site/diff/973bd73a Branch: refs/heads/asf-site Commit: 973bd73a7e90f8eeda0781a11b41f4a169fb1fc7 Parents: ba5fe2b Author: Davor BonaciAuthored: Mon Jan 9 17:47:15 2017 -0800 Committer: Davor Bonaci Committed: Mon Jan 9 17:47:15 2017 -0800 -- content/blog/2016/01/09/added-apex-runner.html | 211 content/blog/2017/01/09/added-apex-runner.html | 211 content/blog/index.html| 32 +-- content/feed.xml | 69 --- content/index.html | 4 +- src/_posts/2017-01-09-added-apex-runner.md | 2 +- 6 files changed, 271 insertions(+), 258 deletions(-) -- http://git-wip-us.apache.org/repos/asf/beam-site/blob/973bd73a/content/blog/2016/01/09/added-apex-runner.html -- diff --git a/content/blog/2016/01/09/added-apex-runner.html b/content/blog/2016/01/09/added-apex-runner.html deleted file mode 100644 index 3d0ab40..000 --- a/content/blog/2016/01/09/added-apex-runner.html +++ /dev/null @@ -1,211 +0,0 @@ - - - - - - - - - Release 0.4.0 adds a runner for Apache Apex - - - - - https://ajax.googleapis.com/ajax/libs/jquery/2.2.0/jquery.min.js"> - - - http://beam.apache.org/blog/2016/01/09/added-apex-runner.html; data-proofer-ignore> - http://beam.apache.org/feed.xml;> - - (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ -(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), - m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) - })(window,document,'script','//www.google-analytics.com/analytics.js','ga'); - -ga('create', 'UA-73650088-1', 'auto'); -ga('send', 'pageview'); - - - - - - - - - - - - - - - -Toggle navigation - - - - - - - - - Get Started - - Beam Overview - Quickstart - - Example Walkthroughs - WordCount - Mobile Gaming - - Resources - Downloads - Support - - - - Documentation - - Using the Documentation - - Beam Concepts - Programming Guide - Additional Resources - - Pipeline Fundamentals - Design Your Pipeline - Create Your Pipeline - Test Your Pipeline - - SDKs - Java SDK - Java SDK API Reference - -Python SDK - - Runners - Capability Matrix - Direct Runner - Apache Apex Runner - Apache Flink Runner - Apache Spark Runner - Cloud Dataflow Runner - - - - Contribute - - Get Started Contributing - -Guides - Contribution Guide -Testing Guide -Release Guide - -Technical References -Design Principles - Ongoing Projects -Source Repository - - Promotion -Presentation Materials -Logos and Design - -Maturity Model -Team - - - -Blog - - - - https://www.apache.org/foundation/press/kit/feather_small.png; alt="Apache Logo" style="height:24px;">Apache Software Foundation - -http://www.apache.org/;>ASF Homepage -http://www.apache.org/licenses/;>License -http://www.apache.org/security/;>Security -http://www.apache.org/foundation/thanks.html;>Thanks -
beam-site git commit: Publish a blog about Apache Apex runner
Repository: beam-site Updated Branches: refs/heads/asf-site 5de55f266 -> ba5fe2b2a Publish a blog about Apache Apex runner Project: http://git-wip-us.apache.org/repos/asf/beam-site/repo Commit: http://git-wip-us.apache.org/repos/asf/beam-site/commit/ba5fe2b2 Tree: http://git-wip-us.apache.org/repos/asf/beam-site/tree/ba5fe2b2 Diff: http://git-wip-us.apache.org/repos/asf/beam-site/diff/ba5fe2b2 Branch: refs/heads/asf-site Commit: ba5fe2b2a1827fed3290c0418c7af873d64b975b Parents: 5de55f2 Author: Davor BonaciAuthored: Mon Jan 9 17:43:02 2017 -0800 Committer: Davor Bonaci Committed: Mon Jan 9 17:43:02 2017 -0800 -- content/blog/2016/01/08/added-apex-runner.html | 211 content/blog/2016/01/09/added-apex-runner.html | 211 content/blog/index.html| 6 +- content/index.html | 2 +- src/_posts/2017-01-09-added-apex-runner.md | 2 +- 5 files changed, 216 insertions(+), 216 deletions(-) -- http://git-wip-us.apache.org/repos/asf/beam-site/blob/ba5fe2b2/content/blog/2016/01/08/added-apex-runner.html -- diff --git a/content/blog/2016/01/08/added-apex-runner.html b/content/blog/2016/01/08/added-apex-runner.html deleted file mode 100644 index abb22bb..000 --- a/content/blog/2016/01/08/added-apex-runner.html +++ /dev/null @@ -1,211 +0,0 @@ - - - - - - - - - Release 0.4.0 adds a runner for Apache Apex - - - - - https://ajax.googleapis.com/ajax/libs/jquery/2.2.0/jquery.min.js"> - - - http://beam.apache.org/blog/2016/01/08/added-apex-runner.html; data-proofer-ignore> - http://beam.apache.org/feed.xml;> - - (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ -(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), - m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) - })(window,document,'script','//www.google-analytics.com/analytics.js','ga'); - -ga('create', 'UA-73650088-1', 'auto'); -ga('send', 'pageview'); - - - - - - - - - - - - - - - -Toggle navigation - - - - - - - - - Get Started - - Beam Overview - Quickstart - - Example Walkthroughs - WordCount - Mobile Gaming - - Resources - Downloads - Support - - - - Documentation - - Using the Documentation - - Beam Concepts - Programming Guide - Additional Resources - - Pipeline Fundamentals - Design Your Pipeline - Create Your Pipeline - Test Your Pipeline - - SDKs - Java SDK - Java SDK API Reference - -Python SDK - - Runners - Capability Matrix - Direct Runner - Apache Apex Runner - Apache Flink Runner - Apache Spark Runner - Cloud Dataflow Runner - - - - Contribute - - Get Started Contributing - -Guides - Contribution Guide -Testing Guide -Release Guide - -Technical References -Design Principles - Ongoing Projects -Source Repository - - Promotion -Presentation Materials -Logos and Design - -Maturity Model -Team - - - -Blog - - - - https://www.apache.org/foundation/press/kit/feather_small.png; alt="Apache Logo" style="height:24px;">Apache Software Foundation - -http://www.apache.org/;>ASF Homepage -http://www.apache.org/licenses/;>License -http://www.apache.org/security/;>Security -http://www.apache.org/foundation/thanks.html;>Thanks -http://www.apache.org/foundation/sponsorship.html;>Sponsorship -https://www.apache.org/foundation/policies/conduct;>Code of Conduct - -
[GitHub] beam-site pull request #119: Added post for 0.4.0 release with Apex runner a...
Github user asfgit closed the pull request at: https://github.com/apache/beam-site/pull/119 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[3/3] beam-site git commit: This closes #119
This closes #119 Project: http://git-wip-us.apache.org/repos/asf/beam-site/repo Commit: http://git-wip-us.apache.org/repos/asf/beam-site/commit/5de55f26 Tree: http://git-wip-us.apache.org/repos/asf/beam-site/tree/5de55f26 Diff: http://git-wip-us.apache.org/repos/asf/beam-site/diff/5de55f26 Branch: refs/heads/asf-site Commit: 5de55f26657de26ef3c1542d9565c635121918bd Parents: fe57db9 e754782 Author: Davor BonaciAuthored: Mon Jan 9 17:37:53 2017 -0800 Committer: Davor Bonaci Committed: Mon Jan 9 17:37:53 2017 -0800 -- content/blog/2016/01/08/added-apex-runner.html | 211 content/blog/index.html| 16 ++ content/index.html | 2 + src/_data/authors.yml | 4 + src/_posts/2017-01-09-added-apex-runner.md | 39 5 files changed, 272 insertions(+) --
[1/3] beam-site git commit: Added post for 0.4.0 release with Apex runner addition.
Repository: beam-site Updated Branches: refs/heads/asf-site fe57db99a -> 5de55f266 Added post for 0.4.0 release with Apex runner addition. Project: http://git-wip-us.apache.org/repos/asf/beam-site/repo Commit: http://git-wip-us.apache.org/repos/asf/beam-site/commit/430a84c3 Tree: http://git-wip-us.apache.org/repos/asf/beam-site/tree/430a84c3 Diff: http://git-wip-us.apache.org/repos/asf/beam-site/diff/430a84c3 Branch: refs/heads/asf-site Commit: 430a84c3ae9689be7ba3ba4e3b5ca83775118e1e Parents: fe57db9 Author: Thomas WeiseAuthored: Sat Dec 31 10:10:28 2016 -0800 Committer: Davor Bonaci Committed: Mon Jan 9 17:35:11 2017 -0800 -- src/_data/authors.yml | 4 +++ src/_posts/2017-01-09-added-apex-runner.md | 39 + 2 files changed, 43 insertions(+) -- http://git-wip-us.apache.org/repos/asf/beam-site/blob/430a84c3/src/_data/authors.yml -- diff --git a/src/_data/authors.yml b/src/_data/authors.yml index ffcb419..147dd7b 100644 --- a/src/_data/authors.yml +++ b/src/_data/authors.yml @@ -32,3 +32,7 @@ tgroh: jesseanderson: name: Jesse Anderson twitter: jessetanderson +thw: +name: Thomas Weise +email: t...@apache.org +twitter: thweise http://git-wip-us.apache.org/repos/asf/beam-site/blob/430a84c3/src/_posts/2017-01-09-added-apex-runner.md -- diff --git a/src/_posts/2017-01-09-added-apex-runner.md b/src/_posts/2017-01-09-added-apex-runner.md new file mode 100644 index 000..93c00ed --- /dev/null +++ b/src/_posts/2017-01-09-added-apex-runner.md @@ -0,0 +1,39 @@ +--- +layout: post +title: "Release 0.4.0 adds a runner for Apache Apex" +date: 2016-01-09 00:00:01 -0700 +excerpt_separator: +categories: blog +authors: + - thw +--- + +The latest release 0.4.0 of [Apache Beam](https://beam.apache.org) adds a new runner for [Apache Apex](http://apex.apache.org/). We are excited to reach this initial milestone and are looking forward to continued collaboration between the Beam and Apex communities to advance the runner. + + + +Beam evolved from the Google Dataflow SDK and as incubator project has quickly adapted the Apache way, grown the community and attracts increasing interest from users that hope to benefit from a conceptual strong unified programming model that is portable between different big data processing frameworks (see [Streaming-101](https://www.oreilly.com/ideas/the-world-beyond-batch-streaming-101) and [Streaming-102](https://www.oreilly.com/ideas/the-world-beyond-batch-streaming-102)). Multiple Apache projects already provide runners for Beam (see [runners and capabilities matrix](http://beam.apache.org/documentation/runners/capability-matrix/)). + +Apex is a stream processing framework for low-latency, high-throughput, stateful and reliable processing of complex analytics pipelines on clusters. Apex was developed since 2012 and is used in production by large companies for real-time and batch processing at scale. + +The initial revision of the runner was focussed on broad coverage of the Beam model on a functional level. That means, there will be follow up work in several areas to take the runner from functional to scalable and high performance to match the capabilities of Apex and its native API. The runner capabilities matrix shows that the Apex capabilities are well aligned with the Beam model. Specifically, the ability to track computational state in a fault tolerant and efficient manner is needed to broadly support the windowing concepts, including event time based processing. + +## Stateful Stream Processor + +Apex was built as stateful stream processor from the ground up. Operators [checkpoint](https://www.datatorrent.com/blog/blog-introduction-to-checkpoint/) state in a distributed and asynchronous manner that produces a consistent snapshot for the entire processing graph, which can be used for recovery. Apex also supports such recovery in an incremental, or fine grained, manner. This means only the portion of the DAG that is actually affected by a failure will be recovered while the remaining pipeline continues processing (this can be leveraged to implement use cases with special needs, such as speculative execution to achieve SLA on the processing latency). The state checkpointing along with idempotent processing guarantee is the basis for [exactly-once results](https://www.datatorrent.com/blog/end-to-end-exactly-once-with-apache-apex/) support in Apex. + +## Translation to Apex DAG + +A Beam runner needs to implement the translation from the Beam model to the underlying frameworks execution model. In the case of Apex, the runner will translate the pipeline into the [native
[2/3] beam-site git commit: Regenerate website
Regenerate website Project: http://git-wip-us.apache.org/repos/asf/beam-site/repo Commit: http://git-wip-us.apache.org/repos/asf/beam-site/commit/e7547825 Tree: http://git-wip-us.apache.org/repos/asf/beam-site/tree/e7547825 Diff: http://git-wip-us.apache.org/repos/asf/beam-site/diff/e7547825 Branch: refs/heads/asf-site Commit: e75478257414633893bd6037176fde9cf38c772b Parents: 430a84c Author: Davor BonaciAuthored: Mon Jan 9 17:37:52 2017 -0800 Committer: Davor Bonaci Committed: Mon Jan 9 17:37:52 2017 -0800 -- content/blog/2016/01/08/added-apex-runner.html | 211 content/blog/index.html| 16 ++ content/index.html | 2 + 3 files changed, 229 insertions(+) -- http://git-wip-us.apache.org/repos/asf/beam-site/blob/e7547825/content/blog/2016/01/08/added-apex-runner.html -- diff --git a/content/blog/2016/01/08/added-apex-runner.html b/content/blog/2016/01/08/added-apex-runner.html new file mode 100644 index 000..abb22bb --- /dev/null +++ b/content/blog/2016/01/08/added-apex-runner.html @@ -0,0 +1,211 @@ + + + + + + + + + Release 0.4.0 adds a runner for Apache Apex + + + + + https://ajax.googleapis.com/ajax/libs/jquery/2.2.0/jquery.min.js"> + + + http://beam.apache.org/blog/2016/01/08/added-apex-runner.html; data-proofer-ignore> + http://beam.apache.org/feed.xml;> + + (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ +(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), + m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) + })(window,document,'script','//www.google-analytics.com/analytics.js','ga'); + +ga('create', 'UA-73650088-1', 'auto'); +ga('send', 'pageview'); + + + + + + + + + + + + + + + +Toggle navigation + + + + + + + + + Get Started + + Beam Overview + Quickstart + + Example Walkthroughs + WordCount + Mobile Gaming + + Resources + Downloads + Support + + + + Documentation + + Using the Documentation + + Beam Concepts + Programming Guide + Additional Resources + + Pipeline Fundamentals + Design Your Pipeline + Create Your Pipeline + Test Your Pipeline + + SDKs + Java SDK + Java SDK API Reference + +Python SDK + + Runners + Capability Matrix + Direct Runner + Apache Apex Runner + Apache Flink Runner + Apache Spark Runner + Cloud Dataflow Runner + + + + Contribute + + Get Started Contributing + +Guides + Contribution Guide +Testing Guide +Release Guide + +Technical References +Design Principles + Ongoing Projects +Source Repository + + Promotion +Presentation Materials +Logos and Design + +Maturity Model +Team + + + +Blog + + + + https://www.apache.org/foundation/press/kit/feather_small.png; alt="Apache Logo" style="height:24px;">Apache Software Foundation + +http://www.apache.org/;>ASF Homepage +http://www.apache.org/licenses/;>License +http://www.apache.org/security/;>Security +http://www.apache.org/foundation/thanks.html;>Thanks +http://www.apache.org/foundation/sponsorship.html;>Sponsorship +https://www.apache.org/foundation/policies/conduct;>Code of Conduct + + + + + + + + + + + + + + + + +http://schema.org/BlogPosting;> + + +Release 0.4.0 adds a runner for Apache Apex +Jan 8, 2016 ⢠Thomas Weise [https://twitter.com/thweise;>@thweise] + + + + +The latest
[jira] [Created] (BEAM-1252) BigQueryIO.Read: validate exported files with GCS glob.
Pei He created BEAM-1252: Summary: BigQueryIO.Read: validate exported files with GCS glob. Key: BEAM-1252 URL: https://issues.apache.org/jira/browse/BEAM-1252 Project: Beam Issue Type: Bug Components: sdk-java-gcp Reporter: Pei He Assignee: Pei He BigQuery has started creating user-visible temp files that we notice and start reading from, but then they get moved. It could cause job failures and data duplication. On Beam side, we can have stronger validation: 1. When listing files, validate that they match the expected URI. 2. When BQ has finished job, integrity check to verify that # files read from == # files BQ claims exist. 3. If possible, add a prefix to the filename of the glob (*.avro to step*.avro). Step name? Other? This might be as easy as dropping a '/' in the middle of the path. A la #7. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[2/2] beam git commit: This closes #1752: Make DataflowPipelineTranslator public
This closes #1752: Make DataflowPipelineTranslator public Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/b81bd25e Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/b81bd25e Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/b81bd25e Branch: refs/heads/master Commit: b81bd25e1c6fd7ecb930c821966fd1c143665946 Parents: 70dd546 0ca0bdb Author: Kenneth KnowlesAuthored: Mon Jan 9 14:18:57 2017 -0800 Committer: Kenneth Knowles Committed: Mon Jan 9 14:18:57 2017 -0800 -- .../apache/beam/runners/dataflow/DataflowPipelineTranslator.java | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) --
[33/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/ReduceFn.OnMergeContext.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/ReduceFn.OnMergeContext.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/ReduceFn.OnMergeContext.html new file mode 100644 index 000..0ddaec5 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/ReduceFn.OnMergeContext.html @@ -0,0 +1,297 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +ReduceFn.OnMergeContext (Apache Beam SDK for Java, version 0.4.0) + + + + + + +var methods = {"i0":6}; +var tabs = {65535:["t0","All Methods"],2:["t2","Instance Methods"],4:["t3","Abstract Methods"]}; +var altColor = "altColor"; +var rowColor = "rowColor"; +var tableTab = "tableTab"; +var activeTableTab = "activeTableTab"; + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +PrevClass +NextClass + + +Frames +NoFrames + + +AllClasses + + + + + + + +Summary: +Nested| +Field| +Constr| +Method + + +Detail: +Field| +Constr| +Method + + + + + + + + +org.apache.beam.runners.core +Class ReduceFn.OnMergeContext + + + +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">java.lang.Object + + +org.apache.beam.runners.core.ReduceFn.Context + + +org.apache.beam.runners.core.ReduceFn.OnMergeContext + + + + + + + + + +Enclosing class: +ReduceFnK,InputT,OutputT,W extends BoundedWindow + + + +public abstract class ReduceFn.OnMergeContext +extends ReduceFn.Context +Information accessible within ReduceFn.onMerge(org.apache.beam.runners.core.ReduceFnK, InputT, OutputT, W.OnMergeContext). + + + + + + + + + + + +Constructor Summary + +Constructors + +Constructor and Description + + +OnMergeContext() + + + + + + + + + +Method Summary + +All MethodsInstance MethodsAbstract Methods + +Modifier and Type +Method and Description + + +abstract MergingStateAccessorK,W +state() +Return the interface for accessing state. + + + + + + + +Methods inherited from classorg.apache.beam.runners.core.ReduceFn.Context +key, timers, window, windowingStrategy + + + + + +Methods inherited from classjava.lang.http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Object +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#clone--; title="class or interface in java.lang">clone, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#equals-java.lang.Object-; title="class or interface in java.lang">equals, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#finalize--; title="class or interface in java.lang">finalize, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#getClass--; title="class or interface in java.lang">getClass, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#hashCode--; title="class or interface in java.lang">hashCode, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#notify--; title="class or interface in java.lang">notify, http://docs.oracle.com/javase/7/docs/api/java/lang /Object.html?is-external=true#notifyAll--" title="class or interface in java.lang">notifyAll, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#toString--; title="class or interface in java.lang">toString, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#wait--; title="class or interface in java.lang">wait, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#wait-long-; title="class or interface in java.lang">wait, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#wait-long-int-; title="class or interface in java.lang">wait + + + + + + + + + + + + + + +Constructor Detail + + + + + +OnMergeContext +publicOnMergeContext() + + + + + + + + + +Method Detail + + + + + +state +public abstractMergingStateAccessorK,Wstate() +Return the interface for accessing state. + +Specified by: +statein classReduceFn.Context + + + + + + + + + + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index
[51/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
Publish Javadoc for version 0.4.0 Project: http://git-wip-us.apache.org/repos/asf/beam-site/repo Commit: http://git-wip-us.apache.org/repos/asf/beam-site/commit/de6a0417 Tree: http://git-wip-us.apache.org/repos/asf/beam-site/tree/de6a0417 Diff: http://git-wip-us.apache.org/repos/asf/beam-site/diff/de6a0417 Branch: refs/heads/asf-site Commit: de6a0417c84d12818656d11fc8f36b64c65e8e52 Parents: e8e45f9 Author: Davor BonaciAuthored: Mon Jan 9 14:28:45 2017 -0800 Committer: Davor Bonaci Committed: Mon Jan 9 14:28:45 2017 -0800 -- .../sdks/javadoc/0.4.0/allclasses-frame.html| 965 + .../sdks/javadoc/0.4.0/allclasses-noframe.html | 965 + .../sdks/javadoc/0.4.0/constant-values.html | 1318 + .../sdks/javadoc/0.4.0/deprecated-list.html | 603 + .../sdks/javadoc/0.4.0/help-doc.html| 231 + .../sdks/javadoc/0.4.0/index-all.html | 21330 + .../documentation/sdks/javadoc/0.4.0/index.html |76 + .../beam/runners/apex/ApexPipelineOptions.html | 379 + .../ApexRunner.CreateApexPCollectionView.html | 328 + .../apache/beam/runners/apex/ApexRunner.html| 395 + .../apex/ApexRunnerRegistrar.Options.html | 288 + .../apex/ApexRunnerRegistrar.Runner.html| 292 + .../beam/runners/apex/ApexRunnerRegistrar.html | 231 + .../beam/runners/apex/ApexRunnerResult.html | 450 + .../apex/ApexYarnLauncher.LaunchParams.html | 317 + .../apex/ApexYarnLauncher.ProcessWatcher.html | 300 + .../beam/runners/apex/ApexYarnLauncher.html | 405 + .../beam/runners/apex/TestApexRunner.html | 294 + .../apex/class-use/ApexPipelineOptions.html | 166 + .../ApexRunner.CreateApexPCollectionView.html | 168 + .../beam/runners/apex/class-use/ApexRunner.html | 168 + .../class-use/ApexRunnerRegistrar.Options.html | 126 + .../class-use/ApexRunnerRegistrar.Runner.html | 126 + .../apex/class-use/ApexRunnerRegistrar.html | 126 + .../apex/class-use/ApexRunnerResult.html| 172 + .../ApexYarnLauncher.LaunchParams.html | 168 + .../ApexYarnLauncher.ProcessWatcher.html| 126 + .../apex/class-use/ApexYarnLauncher.html| 126 + .../runners/apex/class-use/TestApexRunner.html | 168 + .../apache/beam/runners/apex/package-frame.html |34 + .../beam/runners/apex/package-summary.html | 229 + .../apache/beam/runners/apex/package-tree.html | 173 + .../apache/beam/runners/apex/package-use.html | 189 + .../beam/runners/core/AggregatorFactory.html| 237 + .../apache/beam/runners/core/AssignWindows.html | 331 + .../beam/runners/core/AssignWindowsDoFn.html| 341 + .../core/DoFnRunner.ReduceFnExecutor.html | 254 + .../apache/beam/runners/core/DoFnRunner.html| 288 + .../runners/core/DoFnRunners.OutputManager.html | 233 + .../apache/beam/runners/core/DoFnRunners.html | 453 + .../runners/core/ElementAndRestriction.html | 315 + .../core/ElementAndRestrictionCoder.html| 394 + .../runners/core/ElementByteSizeObservable.html | 267 + .../core/GroupAlsoByWindowViaWindowSetDoFn.html | 413 + .../runners/core/GroupAlsoByWindowsDoFn.html| 371 + .../GroupAlsoByWindowsViaOutputBufferDoFn.html | 359 + ...yKeyViaGroupByKeyOnly.GroupAlsoByWindow.html | 371 + ...upByKeyViaGroupByKeyOnly.GroupByKeyOnly.html | 356 + .../core/GroupByKeyViaGroupByKeyOnly.html | 383 + .../apache/beam/runners/core/KeyedWorkItem.html | 268 + .../beam/runners/core/KeyedWorkItemCoder.html | 473 + .../KeyedWorkItems.ComposedKeyedWorkItem.html | 351 + .../beam/runners/core/KeyedWorkItems.html | 362 + .../core/LateDataDroppingDoFnRunner.html| 366 + .../apache/beam/runners/core/NonEmptyPanes.html | 348 + .../beam/runners/core/OutputWindowedValue.html | 262 + .../beam/runners/core/PaneInfoTracker.html | 329 + .../beam/runners/core/PeekingReiterator.html| 383 + .../runners/core/PerKeyCombineFnRunners.html| 278 + .../core/PushbackSideInputDoFnRunner.html | 354 + .../beam/runners/core/ReduceFn.Context.html | 349 + .../runners/core/ReduceFn.OnMergeContext.html | 297 + .../runners/core/ReduceFn.OnTriggerContext.html | 311 + .../core/ReduceFn.ProcessValueContext.html | 309 + .../org/apache/beam/runners/core/ReduceFn.html | 471 + .../beam/runners/core/ReduceFnRunner.html | 378 + .../beam/runners/core/SideInputHandler.html | 437 + .../beam/runners/core/SimpleDoFnRunner.html | 373 + .../SplittableParDo.GBKIntoKeyedWorkItems.html | 333 + .../core/SplittableParDo.ProcessElements.html | 445 + .../runners/core/SplittableParDo.ProcessFn.html | 414 + .../beam/runners/core/SplittableParDo.html | 382 + .../beam/runners/core/SystemReduceFn.html | 463 +
[35/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/LateDataDroppingDoFnRunner.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/LateDataDroppingDoFnRunner.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/LateDataDroppingDoFnRunner.html new file mode 100644 index 000..c284882 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/LateDataDroppingDoFnRunner.html @@ -0,0 +1,366 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +LateDataDroppingDoFnRunner (Apache Beam SDK for Java, version 0.4.0) + + + + + + +var methods = {"i0":10,"i1":10,"i2":10}; +var tabs = {65535:["t0","All Methods"],2:["t2","Instance Methods"],8:["t4","Concrete Methods"]}; +var altColor = "altColor"; +var rowColor = "rowColor"; +var tableTab = "tableTab"; +var activeTableTab = "activeTableTab"; + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +PrevClass +NextClass + + +Frames +NoFrames + + +AllClasses + + + + + + + +Summary: +Nested| +Field| +Constr| +Method + + +Detail: +Field| +Constr| +Method + + + + + + + + +org.apache.beam.runners.core +Class LateDataDroppingDoFnRunnerK,InputT,OutputT,W extends BoundedWindow + + + +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">java.lang.Object + + +org.apache.beam.runners.core.LateDataDroppingDoFnRunnerK,InputT,OutputT,W + + + + + + + +Type Parameters: +K - key type +InputT - input value element type +OutputT - output value element type +W - window type + + +All Implemented Interfaces: +DoFnRunnerKeyedWorkItemK,InputT,KVK,OutputT + + + +public class LateDataDroppingDoFnRunnerK,InputT,OutputT,W extends BoundedWindow +extends http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Object +implements DoFnRunnerKeyedWorkItemK,InputT,KVK,OutputT +A customized DoFnRunner that handles late data dropping for + a KeyedWorkItem input OldDoFn. + + It expands windows before checking data lateness. + + KeyedWorkItems are always in empty windows. + + + + + + + + + + + +Nested Class Summary + + + + +Nested classes/interfaces inherited from interfaceorg.apache.beam.runners.core.DoFnRunner +DoFnRunner.ReduceFnExecutorK,InputT,OutputT,W + + + + + + + + +Constructor Summary + +Constructors + +Constructor and Description + + +LateDataDroppingDoFnRunner(DoFnRunnerKeyedWorkItemK,InputT,KVK,OutputTdoFnRunner, + WindowingStrategy?,?windowingStrategy, + TimerInternalstimerInternals, + Aggregatorhttp://docs.oracle.com/javase/7/docs/api/java/lang/Long.html?is-external=true; title="class or interface in java.lang">Long,http://docs.oracle.com/javase/7/docs/api/java/lang/Long.html?is-external=true; title="class or interface in java.lang">LongdroppedDueToLateness) + + + + + + + + + +Method Summary + +All MethodsInstance MethodsConcrete Methods + +Modifier and Type +Method and Description + + +void +finishBundle() +Calls a DoFn's @FinishBundle method and performs + additional tasks, such as flushing in-memory states. + + + +void +processElement(WindowedValueKeyedWorkItemK,InputTelem) +Calls a DoFn's @ProcessElement method with a + DoFn.ProcessContext containing the provided element. + + + +void +startBundle() +Prepares and calls a DoFn's @StartBundle method. + + + + + + + +Methods inherited from classjava.lang.http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Object +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#clone--; title="class or interface in java.lang">clone, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#equals-java.lang.Object-; title="class or interface in java.lang">equals, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#finalize--; title="class or interface in java.lang">finalize, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#getClass--; title="class or interface in java.lang">getClass, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#hashCode--;
[41/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/AssignWindows.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/AssignWindows.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/AssignWindows.html new file mode 100644 index 000..2085f42 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/AssignWindows.html @@ -0,0 +1,331 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +AssignWindows (Apache Beam SDK for Java, version 0.4.0) + + + + + + +var methods = {"i0":10}; +var tabs = {65535:["t0","All Methods"],2:["t2","Instance Methods"],8:["t4","Concrete Methods"]}; +var altColor = "altColor"; +var rowColor = "rowColor"; +var tableTab = "tableTab"; +var activeTableTab = "activeTableTab"; + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +PrevClass +NextClass + + +Frames +NoFrames + + +AllClasses + + + + + + + +Summary: +Nested| +Field| +Constr| +Method + + +Detail: +Field| +Constr| +Method + + + + + + + + +org.apache.beam.runners.core +Class AssignWindowsT,W extends BoundedWindow + + + +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">java.lang.Object + + +org.apache.beam.sdk.transforms.PTransformPCollectionT,PCollectionT + + +org.apache.beam.runners.core.AssignWindowsT,W + + + + + + + + + +Type Parameters: +T - Type of elements being windowed +W - Window type + + +All Implemented Interfaces: +http://docs.oracle.com/javase/7/docs/api/java/io/Serializable.html?is-external=true; title="class or interface in java.io">Serializable, HasDisplayData + + + +public class AssignWindowsT,W extends BoundedWindow +extends PTransformPCollectionT,PCollectionT +PTransform that uses privileged (non-user-facing) APIs to assign elements of a + PCollection to windows according to the provided WindowFn. + +See Also: +Serialized Form + + + + + + + + + + + + +Field Summary + + + + +Fields inherited from classorg.apache.beam.sdk.transforms.PTransform +name + + + + + + + + +Constructor Summary + +Constructors + +Constructor and Description + + +AssignWindows(WindowFn? super T,Wfn) + + + + + + + + + +Method Summary + +All MethodsInstance MethodsConcrete Methods + +Modifier and Type +Method and Description + + +PCollectionT +expand(PCollectionTinput) +Applies this PTransform on the given InputT, and returns its + Output. + + + + + + + +Methods inherited from classorg.apache.beam.sdk.transforms.PTransform +getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, populateDisplayData, toString, valida te + + + + + +Methods inherited from classjava.lang.http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Object +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#clone--; title="class or interface in java.lang">clone, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#equals-java.lang.Object-; title="class or interface in java.lang">equals, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#finalize--; title="class or interface in java.lang">finalize, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#getClass--; title="class or interface in java.lang">getClass, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#hashCode--; title="class or interface in java.lang">hashCode, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#notify--; title="class or interface in java.lang">notify, http://docs.oracle.com/javase/7/docs/api/java/lang /Object.html?is-external=true#notifyAll--" title="class or interface in java.lang">notifyAll, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#wait--; title="class or interface in java.lang">wait, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#wait-long-; title="class or interface in java.lang">wait, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#wait-long-int-; title="class or interface in java.lang">wait + + + + + + + + + + + + + + +Constructor Detail +
[47/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/deprecated-list.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/deprecated-list.html b/content/documentation/sdks/javadoc/0.4.0/deprecated-list.html new file mode 100644 index 000..d5eb2e8 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/deprecated-list.html @@ -0,0 +1,603 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +Deprecated List (Apache Beam SDK for Java, version 0.4.0) + + + + + + + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +Prev +Next + + +Frames +NoFrames + + +AllClasses + + + + + + + + + + +Deprecated API +Contents + +Deprecated Interfaces +Deprecated Classes +Deprecated Fields +Deprecated Methods +Deprecated Constructors +Deprecated Enum Constants + + + + + + + + +Deprecated Interfaces + +Interface and Description + + + +org.apache.beam.sdk.transforms.Aggregator.AggregatorFactory +this is for use only by runners and exists only for a migration period. Please + use the identical interface in org.apache.beam.runners.core + + + +org.apache.beam.sdk.util.IOChannelFactory +This is under redesign, see: https://issues.apache.org/jira/browse/BEAM-59. + + + +org.apache.beam.sdk.util.state.StateTag.StateBinder +for migration only; runners should reference the top level StateBinder + and move towards StateSpec rather than StateTag. + + + +org.apache.beam.sdk.testing.StreamingIT + + +org.apache.beam.sdk.util.state.TimerCallback +Use InMemoryTimerInternals advance and remove methods instead of callback. + + + + + + + + + + + + +Deprecated Classes + +Class and Description + + + +org.apache.beam.sdk.util.AttemptAndTimeBoundedExponentialBackOff + + +org.apache.beam.sdk.util.AttemptBoundedExponentialBackOff + + +org.apache.beam.sdk.transforms.Combine.SimpleCombineFn + + +org.apache.beam.runners.dataflow.DataflowRunner.StreamingPCollectionViewWriterFn + + +org.apache.beam.sdk.transforms.DoFnAdapters +This class will go away when we start running DoFn's directly (using DoFnInvoker) rather than via OldDoFn. + + + +org.apache.beam.sdk.util.IntervalBoundedExponentialBackOff + + +org.apache.beam.sdk.transforms.OldDoFn +Uses of OldDoFn should be replaced by the new DoFn. + + + +org.apache.beam.sdk.util.PCollectionViews.IterablePCollectionView +Runners should not inspect the PCollectionView subclass, as it is an + implementation detail. To specialize a side input, a runner should inspect the + language-independent metadata of the ViewFn. + + + +org.apache.beam.sdk.util.PCollectionViews.ListPCollectionView +Runners should not inspect the PCollectionView subclass, as it is an + implementation detail. To specialize a side input, a runner should inspect the + language-independent metadata of the ViewFn. + + + +org.apache.beam.sdk.util.PCollectionViews.MapPCollectionView +Runners should not inspect the PCollectionView subclass, as it is an + implementation detail. To specialize a side input, a runner should inspect the + language-independent metadata of the ViewFn. + + + +org.apache.beam.sdk.util.PCollectionViews.MultimapPCollectionView +Runners should not inspect the PCollectionView subclass, as it is an + implementation detail. To specialize a side input, a runner should inspect the + language-independent metadata of the ViewFn. + + + +org.apache.beam.sdk.util.PCollectionViews.SingletonPCollectionView +Runners should not inspect the PCollectionView subclass, as it is an + implementation detail. To specialize a side input, a runner should inspect the + language-independent metadata of the ViewFn. + + + + + + + + + + + + +Deprecated Fields + +Field and Description + + + +org.apache.beam.sdk.transforms.reflect.DoFnInvokers.INSTANCE +use DoFnInvokers.invokerFor(DoFn). + + + +org.apache.beam.sdk.util.AppEngineEnvironment.IS_APP_ENGINE + + + + + + + + + + + +Deprecated Methods + +Method and Description + + + +org.apache.beam.sdk.util.state.StateSpecs.adaptTagBinder(StateTag.StateBinderK) +for migration purposes only + + + +org.apache.beam.sdk.util.state.InMemoryTimerInternals.advanceInputWatermark(TimerCallback, Instant) +Use advanceInputWatermark without callback and fireEventTimers. + + + +org.apache.beam.sdk.util.state.InMemoryTimerInternals.advanceProcessingTime(TimerCallback, Instant) +Use advanceProcessingTime without callback and fireProcessingTimers.
[20/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/AfterEachStateMachine.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/AfterEachStateMachine.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/AfterEachStateMachine.html new file mode 100644 index 000..2dc83bc --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/AfterEachStateMachine.html @@ -0,0 +1,450 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +AfterEachStateMachine (Apache Beam SDK for Java, version 0.4.0) + + + + + + +var methods = {"i0":9,"i1":9,"i2":10,"i3":10,"i4":10,"i5":10,"i6":10}; +var tabs = {65535:["t0","All Methods"],1:["t1","Static Methods"],2:["t2","Instance Methods"],8:["t4","Concrete Methods"]}; +var altColor = "altColor"; +var rowColor = "rowColor"; +var tableTab = "tableTab"; +var activeTableTab = "activeTableTab"; + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +PrevClass +NextClass + + +Frames +NoFrames + + +AllClasses + + + + + + + +Summary: +Nested| +Field| +Constr| +Method + + +Detail: +Field| +Constr| +Method + + + + + + + + +org.apache.beam.runners.core.triggers +Class AfterEachStateMachine + + + +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">java.lang.Object + + +org.apache.beam.runners.core.triggers.TriggerStateMachine + + +org.apache.beam.runners.core.triggers.AfterEachStateMachine + + + + + + + + + +All Implemented Interfaces: +http://docs.oracle.com/javase/7/docs/api/java/io/Serializable.html?is-external=true; title="class or interface in java.io">Serializable + + + +public class AfterEachStateMachine +extends TriggerStateMachine +A composite TriggerStateMachine that executes its sub-triggers in order. + Only one sub-trigger is executing at a time, + and any time it fires the AfterEach fires. When the currently executing + sub-trigger finishes, the AfterEach starts executing the next sub-trigger. + + AfterEach.inOrder(t1, t2, ...) finishes when all of the sub-triggers have finished. + + The following properties hold: + +AfterEach.inOrder(AfterEach.inOrder(a, b), c) behaves the same as + AfterEach.inOrder(a, b, c) and AfterEach.inOrder(a, AfterEach.inOrder(b, c). +AfterEach.inOrder(Repeatedly.forever(a), b) behaves the same as + Repeatedly.forever(a), since the repeated trigger never finishes. + + +See Also: +Serialized Form + + + + + + + + + + + + +Nested Class Summary + + + + +Nested classes/interfaces inherited from classorg.apache.beam.runners.core.triggers.TriggerStateMachine +TriggerStateMachine.MergingTriggerInfo, TriggerStateMachine.OnceTriggerStateMachine, TriggerStateMachine.OnElementContext, TriggerStateMachine.OnMergeContext, TriggerStateMachine.TriggerContext, TriggerStateMachine.TriggerInfo + + + + + + + + +Field Summary + + + + +Fields inherited from classorg.apache.beam.runners.core.triggers.TriggerStateMachine +subTriggers + + + + + + + + +Method Summary + +All MethodsStatic MethodsInstance MethodsConcrete Methods + +Modifier and Type +Method and Description + + +static TriggerStateMachine +inOrder(http://docs.oracle.com/javase/7/docs/api/java/lang/Iterable.html?is-external=true; title="class or interface in java.lang">Iterable? extends TriggerStateMachinetriggers) + + +static TriggerStateMachine +inOrder(TriggerStateMachine...triggers) +Returns an AfterEach Trigger with the given subtriggers. + + + +void +onElement(TriggerStateMachine.OnElementContextc) +Called every time an element is incorporated into a window. + + + +void +onFire(TriggerStateMachine.TriggerContextcontext) +Adjusts the state of the trigger to be ready for the next pane. + + + +void +onMerge(TriggerStateMachine.OnMergeContextcontext) +Called immediately after windows have been merged. + + + +boolean +shouldFire(TriggerStateMachine.TriggerContextcontext) +Returns true if the current state of the trigger indicates that its condition + is satisfied and it is ready to fire. + + + +http://docs.oracle.com/javase/7/docs/api/java/lang/String.html?is-external=true; title="class or interface in java.lang">String +toString() + + + + + + +Methods inherited from
[17/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/DefaultTriggerStateMachine.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/DefaultTriggerStateMachine.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/DefaultTriggerStateMachine.html new file mode 100644 index 000..f80e08d --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/DefaultTriggerStateMachine.html @@ -0,0 +1,458 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +DefaultTriggerStateMachine (Apache Beam SDK for Java, version 0.4.0) + + + + + + +var methods = {"i0":10,"i1":10,"i2":9,"i3":10,"i4":10,"i5":10,"i6":10}; +var tabs = {65535:["t0","All Methods"],1:["t1","Static Methods"],2:["t2","Instance Methods"],8:["t4","Concrete Methods"]}; +var altColor = "altColor"; +var rowColor = "rowColor"; +var tableTab = "tableTab"; +var activeTableTab = "activeTableTab"; + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +PrevClass +NextClass + + +Frames +NoFrames + + +AllClasses + + + + + + + +Summary: +Nested| +Field| +Constr| +Method + + +Detail: +Field| +Constr| +Method + + + + + + + + +org.apache.beam.runners.core.triggers +Class DefaultTriggerStateMachine + + + +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">java.lang.Object + + +org.apache.beam.runners.core.triggers.TriggerStateMachine + + +org.apache.beam.runners.core.triggers.DefaultTriggerStateMachine + + + + + + + + + +All Implemented Interfaces: +http://docs.oracle.com/javase/7/docs/api/java/io/Serializable.html?is-external=true; title="class or interface in java.io">Serializable + + + +@Experimental(value=TRIGGER) +public class DefaultTriggerStateMachine +extends TriggerStateMachine +A trigger that is equivalent to Repeatedly.forever(AfterWatermark.pastEndOfWindow()). See + RepeatedlyStateMachine.forever(org.apache.beam.runners.core.triggers.TriggerStateMachine) and AfterWatermarkStateMachine.pastEndOfWindow() for + more details. + +See Also: +Serialized Form + + + + + + + + + + + + +Nested Class Summary + + + + +Nested classes/interfaces inherited from classorg.apache.beam.runners.core.triggers.TriggerStateMachine +TriggerStateMachine.MergingTriggerInfo, TriggerStateMachine.OnceTriggerStateMachine, TriggerStateMachine.OnElementContext, TriggerStateMachine.OnMergeContext, TriggerStateMachine.TriggerContext, TriggerStateMachine.TriggerInfo + + + + + + + + +Field Summary + + + + +Fields inherited from classorg.apache.beam.runners.core.triggers.TriggerStateMachine +subTriggers + + + + + + + + +Method Summary + +All MethodsStatic MethodsInstance MethodsConcrete Methods + +Modifier and Type +Method and Description + + +void +clear(TriggerStateMachine.TriggerContextc) +Clear any state associated with this trigger in the given window. + + + +boolean +isCompatible(TriggerStateMachineother) +Returns whether this performs the same triggering as the given Trigger. + + + +static DefaultTriggerStateMachine +of() +Returns the default trigger. + + + +void +onElement(TriggerStateMachine.OnElementContextc) +Called every time an element is incorporated into a window. + + + +void +onFire(TriggerStateMachine.TriggerContextcontext) +Adjusts the state of the trigger to be ready for the next pane. + + + +void +onMerge(TriggerStateMachine.OnMergeContextc) +Called immediately after windows have been merged. + + + +boolean +shouldFire(TriggerStateMachine.TriggerContextcontext) +Returns true if the current state of the trigger indicates that its condition + is satisfied and it is ready to fire. + + + + + + + +Methods inherited from classorg.apache.beam.runners.core.triggers.TriggerStateMachine +equals, hashCode, orFinally, prefetchOnElement, prefetchOnFire, prefetchOnMerge, prefetchShouldFire, subTriggers, toString + + + + + +Methods inherited from classjava.lang.http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Object +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#clone--; title="class or interface in java.lang">clone,
[08/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/class-use/RepeatedlyStateMachine.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/class-use/RepeatedlyStateMachine.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/class-use/RepeatedlyStateMachine.html new file mode 100644 index 000..5099b44 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/class-use/RepeatedlyStateMachine.html @@ -0,0 +1,172 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +Uses of Class org.apache.beam.runners.core.triggers.RepeatedlyStateMachine (Apache Beam SDK for Java, version 0.4.0) + + + + + + + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +Prev +Next + + +Frames +NoFrames + + +AllClasses + + + + + + + + + + +Uses of Classorg.apache.beam.runners.core.triggers.RepeatedlyStateMachine + + + + + +Packages that use RepeatedlyStateMachine + +Package +Description + + + +org.apache.beam.runners.core.triggers + +State machine implementations for triggers, called "triggers" because + they react to events. + + + + + + + + + + +Uses of RepeatedlyStateMachine in org.apache.beam.runners.core.triggers + +Methods in org.apache.beam.runners.core.triggers that return RepeatedlyStateMachine + +Modifier and Type +Method and Description + + + +static RepeatedlyStateMachine +RepeatedlyStateMachine.forever(TriggerStateMachinerepeated) +Create a composite trigger that repeatedly executes the trigger repeated, firing each + time it fires and ignoring any indications to finish. + + + + + + + + + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +Prev +Next + + +Frames +NoFrames + + +AllClasses + + + + + + + + + +Copyright 2016 https://www.apache.org/;>The Apache Software Foundation. All rights reserved. + + http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/class-use/ReshuffleTriggerStateMachine.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/class-use/ReshuffleTriggerStateMachine.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/class-use/ReshuffleTriggerStateMachine.html new file mode 100644 index 000..ee20cf8 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/class-use/ReshuffleTriggerStateMachine.html @@ -0,0 +1,126 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +Uses of Class org.apache.beam.runners.core.triggers.ReshuffleTriggerStateMachine (Apache Beam SDK for Java, version 0.4.0) + + + + + + + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +Prev +Next + + +Frames +NoFrames + + +AllClasses + + + + + + + + + + +Uses of Classorg.apache.beam.runners.core.triggers.ReshuffleTriggerStateMachine + +No usage of org.apache.beam.runners.core.triggers.ReshuffleTriggerStateMachine + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +Prev +Next + + +Frames +NoFrames + + +AllClasses + + + +var methods = {"i0":10,"i1":9,"i2":10,"i3":10,"i4":10,"i5":10,"i6":10,"i7":10,"i8":10,"i9":10,"i10":10,"i11":10,"i12":10}; +var tabs = {65535:["t0","All Methods"],1:["t1","Static Methods"],2:["t2","Instance Methods"],8:["t4","Concrete Methods"]}; +var altColor = "altColor"; +var rowColor = "rowColor"; +var tableTab = "tableTab"; +var activeTableTab = "activeTableTab"; + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +PrevClass +NextClass + + +Frames +NoFrames + + +AllClasses + + + + + + + +Summary: +Nested| +Field| +Constr| +Method + + +Detail: +Field| +Constr| +Method + + + + + + + + +org.apache.beam.runners.core.triggers +Class AfterPaneStateMachine + + + +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">java.lang.Object + + +org.apache.beam.runners.core.triggers.TriggerStateMachine + + +org.apache.beam.runners.core.triggers.TriggerStateMachine.OnceTriggerStateMachine + + +org.apache.beam.runners.core.triggers.AfterPaneStateMachine + + + + + + + + + + + +All Implemented Interfaces: +http://docs.oracle.com/javase/7/docs/api/java/io/Serializable.html?is-external=true; title="class or interface in java.io">Serializable + + + +@Experimental(value=TRIGGER) +public class AfterPaneStateMachine +extends TriggerStateMachine.OnceTriggerStateMachine +TriggerStateMachines that fire based on properties of the elements in the current pane. + +See Also: +Serialized Form + + + + + + + + + + + + +Nested Class Summary + + + + +Nested classes/interfaces inherited from classorg.apache.beam.runners.core.triggers.TriggerStateMachine +TriggerStateMachine.MergingTriggerInfo, TriggerStateMachine.OnceTriggerStateMachine, TriggerStateMachine.OnElementContext, TriggerStateMachine.OnMergeContext, TriggerStateMachine.TriggerContext, TriggerStateMachine.TriggerInfo + + + + + + + + +Field Summary + + + + +Fields inherited from classorg.apache.beam.runners.core.triggers.TriggerStateMachine +subTriggers + + + + + + + + +Method Summary + +All MethodsStatic MethodsInstance MethodsConcrete Methods + +Modifier and Type +Method and Description + + +void +clear(TriggerStateMachine.TriggerContextc) +Clear any state associated with this trigger in the given window. + + + +static AfterPaneStateMachine +elementCountAtLeast(intcountElems) +Creates a trigger that fires when the pane contains at least countElems elements. + + + +boolean +equals(http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Objectobj) + + +int +getElementCount() +The number of elements after which this trigger may fire. + + + +int +hashCode() + + +boolean +isCompatible(TriggerStateMachineother) +Returns whether this performs the same triggering as the given Trigger. + + + +void +onElement(TriggerStateMachine.OnElementContextc) +Called every time an element is incorporated into a window. + + + +void +onMerge(TriggerStateMachine.OnMergeContextcontext) +Called immediately after windows have been merged. + + + +protected void +onOnlyFiring(TriggerStateMachine.TriggerContextcontext) +Called exactly once by TriggerStateMachine.OnceTriggerStateMachine.onFire(org.apache.beam.runners.core.triggers.TriggerStateMachine.TriggerContext) when the trigger is fired. + + + +void +prefetchOnMerge(MergingStateAccessor?,?state) +Called to allow the trigger to prefetch any state it will likely need to read from during + an TriggerStateMachine.onMerge(org.apache.beam.runners.core.triggers.TriggerStateMachine.OnMergeContext) call. + + + +void +prefetchShouldFire(StateAccessor?state) +Called to allow the trigger to prefetch any
[52/52] beam-site git commit: Update to reflect the 0.4.0 release
Update to reflect the 0.4.0 release Project: http://git-wip-us.apache.org/repos/asf/beam-site/repo Commit: http://git-wip-us.apache.org/repos/asf/beam-site/commit/fe57db99 Tree: http://git-wip-us.apache.org/repos/asf/beam-site/tree/fe57db99 Diff: http://git-wip-us.apache.org/repos/asf/beam-site/diff/fe57db99 Branch: refs/heads/asf-site Commit: fe57db99ac7c284aed8d827ab1eb82acaa27df76 Parents: de6a041 Author: Davor BonaciAuthored: Mon Jan 9 14:30:58 2017 -0800 Committer: Davor Bonaci Committed: Mon Jan 9 14:30:58 2017 -0800 -- .../beam/capability/2016/03/17/capability-matrix.html | 2 +- .../capability/2016/04/03/presentation-materials.html | 2 +- .../python/sdk/2016/02/25/python-sdk-now-public.html | 2 +- content/beam/release/2016/06/15/first-release.html | 2 +- .../2016/10/11/strata-hadoop-world-and-beam.html | 2 +- .../update/website/2016/02/22/beam-has-a-logo.html | 2 +- content/blog/2016/05/18/splitAtFraction-method.html| 2 +- .../2016/05/27/where-is-my-pcollection-dot-map.html| 2 +- .../blog/2016/06/13/flink-batch-runner-milestone.html | 2 +- content/blog/2016/08/03/six-months.html| 2 +- content/blog/2016/10/20/test-stream.html | 4 ++-- content/blog/index.html| 2 +- content/coming-soon.html | 2 +- content/contribute/contribution-guide/index.html | 2 +- content/contribute/design-principles/index.html| 2 +- content/contribute/index.html | 2 +- content/contribute/logos/index.html| 2 +- content/contribute/maturity-model/index.html | 2 +- content/contribute/presentation-materials/index.html | 2 +- content/contribute/release-guide/index.html| 2 +- content/contribute/source-repository/index.html| 2 +- content/contribute/team/index.html | 2 +- content/contribute/testing/index.html | 2 +- content/contribute/work-in-progress/index.html | 2 +- content/documentation/index.html | 2 +- .../pipelines/create-your-pipeline/index.html | 2 +- .../pipelines/design-your-pipeline/index.html | 2 +- .../pipelines/test-your-pipeline/index.html| 7 +++ content/documentation/programming-guide/index.html | 12 ++-- content/documentation/resources/index.html | 2 +- content/documentation/runners/apex/index.html | 2 +- .../documentation/runners/capability-matrix/index.html | 2 +- content/documentation/runners/dataflow/index.html | 6 +++--- content/documentation/runners/direct/index.html| 12 ++-- content/documentation/runners/flink/index.html | 6 +++--- content/documentation/runners/spark/index.html | 4 ++-- content/documentation/sdks/java/index.html | 2 +- content/documentation/sdks/javadoc/current/index.html | 8 content/documentation/sdks/python/index.html | 2 +- content/feed.xml | 2 +- content/get-started/beam-overview/index.html | 2 +- content/get-started/downloads/index.html | 13 + content/get-started/index.html | 2 +- content/get-started/mobile-gaming-example/index.html | 2 +- content/get-started/quickstart/index.html | 2 +- content/get-started/support/index.html | 2 +- content/get-started/wordcount-example/index.html | 2 +- content/index.html | 2 +- content/privacy_policy/index.html | 2 +- 49 files changed, 78 insertions(+), 74 deletions(-) -- http://git-wip-us.apache.org/repos/asf/beam-site/blob/fe57db99/content/beam/capability/2016/03/17/capability-matrix.html -- diff --git a/content/beam/capability/2016/03/17/capability-matrix.html b/content/beam/capability/2016/03/17/capability-matrix.html index 15b2db1..7c19748 100644 --- a/content/beam/capability/2016/03/17/capability-matrix.html +++ b/content/beam/capability/2016/03/17/capability-matrix.html @@ -78,7 +78,7 @@ SDKs Java SDK - Java SDK API Reference Java SDK API Reference http://git-wip-us.apache.org/repos/asf/beam-site/blob/fe57db99/content/beam/capability/2016/04/03/presentation-materials.html -- diff --git a/content/beam/capability/2016/04/03/presentation-materials.html b/content/beam/capability/2016/04/03/presentation-materials.html index 17451c9..0ada66f 100644
[29/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/SystemReduceFn.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/SystemReduceFn.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/SystemReduceFn.html new file mode 100644 index 000..d7d43e5 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/SystemReduceFn.html @@ -0,0 +1,463 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +SystemReduceFn (Apache Beam SDK for Java, version 0.4.0) + + + + + + +var methods = {"i0":9,"i1":10,"i2":9,"i3":10,"i4":10,"i5":10,"i6":10}; +var tabs = {65535:["t0","All Methods"],1:["t1","Static Methods"],2:["t2","Instance Methods"],8:["t4","Concrete Methods"]}; +var altColor = "altColor"; +var rowColor = "rowColor"; +var tableTab = "tableTab"; +var activeTableTab = "activeTableTab"; + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +PrevClass +NextClass + + +Frames +NoFrames + + +AllClasses + + + + + + + +Summary: +Nested| +Field| +Constr| +Method + + +Detail: +Field| +Constr| +Method + + + + + + + + +org.apache.beam.runners.core +Class SystemReduceFnK,InputT,AccumT,OutputT,W extends BoundedWindow + + + +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">java.lang.Object + + +org.apache.beam.runners.core.ReduceFnK,InputT,OutputT,W + + +org.apache.beam.runners.core.SystemReduceFnK,InputT,AccumT,OutputT,W + + + + + + + + + +Type Parameters: +K - The type of key being processed. +InputT - The type of values associated with the key. +OutputT - The output type that will be produced for each key. +W - The type of windows this operates on. + + +All Implemented Interfaces: +http://docs.oracle.com/javase/7/docs/api/java/io/Serializable.html?is-external=true; title="class or interface in java.io">Serializable + + + +public abstract class SystemReduceFnK,InputT,AccumT,OutputT,W extends BoundedWindow +extends ReduceFnK,InputT,OutputT,W +ReduceFn implementing the default reduction behaviors of GroupByKey. + +See Also: +Serialized Form + + + + + + + + + + + + +Nested Class Summary + + + + +Nested classes/interfaces inherited from classorg.apache.beam.runners.core.ReduceFn +ReduceFn.Context, ReduceFn.OnMergeContext, ReduceFn.OnTriggerContext, ReduceFn.ProcessValueContext + + + + + + + + +Constructor Summary + +Constructors + +Constructor and Description + + +SystemReduceFn(StateTag? super K,? extends CombiningStateInputT,OutputTbufferTag) + + + + + + + + + +Method Summary + +All MethodsStatic MethodsInstance MethodsConcrete Methods + +Modifier and Type +Method and Description + + +static K,T,W extends BoundedWindowSystemReduceFnK,T,http://docs.oracle.com/javase/7/docs/api/java/lang/Iterable.html?is-external=true; title="class or interface in java.lang">IterableT,http://docs.oracle.com/javase/7/docs/api/java/lang/Iterable.html?is-external=true; title="class or interface in java.lang">IterableT,W +buffering(CoderTinputCoder) +Create a factory that produces SystemReduceFn instances that that buffer all of the + input values in persistent state and produces an IterableT. + + + +void +clearState(ReduceFn.Contextc) +Called to clear any persisted state that the ReduceFn may be holding. + + + +static K,InputT,AccumT,OutputT,W extends BoundedWindowSystemReduceFnK,InputT,AccumT,OutputT,W +combining(CoderKkeyCoder, + AppliedCombineFnK,InputT,AccumT,OutputTcombineFn) +Create a factory that produces SystemReduceFn instances that combine all of the input + values using a Combine.CombineFn. + + + +ReadableStatehttp://docs.oracle.com/javase/7/docs/api/java/lang/Boolean.html?is-external=true; title="class or interface in java.lang">Boolean +isEmpty(StateAccessorKstate) +Returns true if the there is no buffered state. + + + +void +onTrigger(ReduceFn.OnTriggerContextc) +Called when triggers fire. + + + +void +prefetchOnTrigger(StateAccessorKstate) +Called before ReduceFn.onTrigger(org.apache.beam.runners.core.ReduceFnK, InputT, OutputT, W.OnTriggerContext) is invoked to provide an opportunity to prefetch any needed + state. + + + +void +processValue(ReduceFn.ProcessValueContextc) +Called for each value of type InputT associated with the current key. + + + + + + + +Methods
[21/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/AfterDelayFromFirstElementStateMachine.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/AfterDelayFromFirstElementStateMachine.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/AfterDelayFromFirstElementStateMachine.html new file mode 100644 index 000..dcacb25 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/AfterDelayFromFirstElementStateMachine.html @@ -0,0 +1,779 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +AfterDelayFromFirstElementStateMachine (Apache Beam SDK for Java, version 0.4.0) + + + + + + +var methods = {"i0":10,"i1":10,"i2":10,"i3":10,"i4":6,"i5":10,"i6":10,"i7":10,"i8":6,"i9":10,"i10":10,"i11":10,"i12":10,"i13":10,"i14":10,"i15":10,"i16":10}; +var tabs = {65535:["t0","All Methods"],2:["t2","Instance Methods"],4:["t3","Abstract Methods"],8:["t4","Concrete Methods"]}; +var altColor = "altColor"; +var rowColor = "rowColor"; +var tableTab = "tableTab"; +var activeTableTab = "activeTableTab"; + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +PrevClass +NextClass + + +Frames +NoFrames + + +AllClasses + + + + + + + +Summary: +Nested| +Field| +Constr| +Method + + +Detail: +Field| +Constr| +Method + + + + + + + + +org.apache.beam.runners.core.triggers +Class AfterDelayFromFirstElementStateMachine + + + +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">java.lang.Object + + +org.apache.beam.runners.core.triggers.TriggerStateMachine + + +org.apache.beam.runners.core.triggers.TriggerStateMachine.OnceTriggerStateMachine + + +org.apache.beam.runners.core.triggers.AfterDelayFromFirstElementStateMachine + + + + + + + + + + + +All Implemented Interfaces: +http://docs.oracle.com/javase/7/docs/api/java/io/Serializable.html?is-external=true; title="class or interface in java.io">Serializable + + +Direct Known Subclasses: +AfterProcessingTimeStateMachine + + + +@Experimental(value=TRIGGER) +public abstract class AfterDelayFromFirstElementStateMachine +extends TriggerStateMachine.OnceTriggerStateMachine +A base class for triggers that happen after a processing time delay from the arrival + of the first element in a pane. + + This class is for internal use only and may change at any time. + +See Also: +Serialized Form + + + + + + + + + + + + +Nested Class Summary + + + + +Nested classes/interfaces inherited from classorg.apache.beam.runners.core.triggers.TriggerStateMachine +TriggerStateMachine.MergingTriggerInfo, TriggerStateMachine.OnceTriggerStateMachine, TriggerStateMachine.OnElementContext, TriggerStateMachine.OnMergeContext, TriggerStateMachine.TriggerContext, TriggerStateMachine.TriggerInfo + + + + + + + + +Field Summary + +Fields + +Modifier and Type +Field and Description + + +protected static StateTaghttp://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Object,AccumulatorCombiningStateorg.joda.time.Instant,Combine.Holderorg.joda.time.Instant,org.joda.time.Instant +DELAYED_UNTIL_TAG + + +protected static http://docs.oracle.com/javase/7/docs/api/java/util/List.html?is-external=true; title="class or interface in java.util">ListSerializableFunctionorg.joda.time.Instant,org.joda.time.Instant +IDENTITY + + +protected TimeDomain +timeDomain + + +protected http://docs.oracle.com/javase/7/docs/api/java/util/List.html?is-external=true; title="class or interface in java.util">ListSerializableFunctionorg.joda.time.Instant,org.joda.time.Instant +timestampMappers +A list of timestampMappers m1, m2, m3, ... + + + + + + + +Fields inherited from classorg.apache.beam.runners.core.triggers.TriggerStateMachine +subTriggers + + + + + + + + +Constructor Summary + +Constructors + +Constructor and Description + + +AfterDelayFromFirstElementStateMachine(TimeDomaintimeDomain, + http://docs.oracle.com/javase/7/docs/api/java/util/List.html?is-external=true; title="class or interface in java.util">ListSerializableFunctionorg.joda.time.Instant,org.joda.time.InstanttimestampMappers) + + + + + + + + + +Method Summary + +All
[39/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/ElementAndRestrictionCoder.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/ElementAndRestrictionCoder.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/ElementAndRestrictionCoder.html new file mode 100644 index 000..0beb6ea --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/ElementAndRestrictionCoder.html @@ -0,0 +1,394 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +ElementAndRestrictionCoder (Apache Beam SDK for Java, version 0.4.0) + + + + + + +var methods = {"i0":10,"i1":10,"i2":10,"i3":10,"i4":9}; +var tabs = {65535:["t0","All Methods"],1:["t1","Static Methods"],2:["t2","Instance Methods"],8:["t4","Concrete Methods"]}; +var altColor = "altColor"; +var rowColor = "rowColor"; +var tableTab = "tableTab"; +var activeTableTab = "activeTableTab"; + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +PrevClass +NextClass + + +Frames +NoFrames + + +AllClasses + + + + + + + +Summary: +Nested| +Field| +Constr| +Method + + +Detail: +Field| +Constr| +Method + + + + + + + + +org.apache.beam.runners.core +Class ElementAndRestrictionCoderElementT,RestrictionT + + + +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">java.lang.Object + + +org.apache.beam.sdk.coders.StandardCoderT + + +org.apache.beam.sdk.coders.DeterministicStandardCoderT + + +org.apache.beam.sdk.coders.AtomicCoderT + + +org.apache.beam.sdk.coders.CustomCoderElementAndRestrictionElementT,RestrictionT + + +org.apache.beam.runners.core.ElementAndRestrictionCoderElementT,RestrictionT + + + + + + + + + + + + + + + +All Implemented Interfaces: +http://docs.oracle.com/javase/7/docs/api/java/io/Serializable.html?is-external=true; title="class or interface in java.io">Serializable, CoderElementAndRestrictionElementT,RestrictionT + + + +@Experimental(value=SPLITTABLE_DO_FN) +public class ElementAndRestrictionCoderElementT,RestrictionT +extends CustomCoderElementAndRestrictionElementT,RestrictionT +A Coder for ElementAndRestriction. + +See Also: +Serialized Form + + + + + + + + + + + + +Nested Class Summary + + + + +Nested classes/interfaces inherited from interfaceorg.apache.beam.sdk.coders.Coder +Coder.Context, Coder.NonDeterministicException + + + + + + + + +Method Summary + +All MethodsStatic MethodsInstance MethodsConcrete Methods + +Modifier and Type +Method and Description + + +ElementAndRestrictionElementT,RestrictionT +decode(http://docs.oracle.com/javase/7/docs/api/java/io/InputStream.html?is-external=true; title="class or interface in java.io">InputStreaminStream, + Coder.Contextcontext) +Decodes a value of type T from the given input stream in + the given context. + + + +void +encode(ElementAndRestrictionElementT,RestrictionTvalue, + http://docs.oracle.com/javase/7/docs/api/java/io/OutputStream.html?is-external=true; title="class or interface in java.io">OutputStreamoutStream, + Coder.Contextcontext) +Encodes the given value of type T onto the given output stream + in the given context. + + + +CoderElementT +getElementCoder() + + +CoderRestrictionT +getRestrictionCoder() + + +static ElementT,RestrictionTElementAndRestrictionCoderElementT,RestrictionT +of(CoderElementTelementCoder, + CoderRestrictionTrestrictionCoder) +Creates an ElementAndRestrictionCoder from an element coder and a restriction coder. + + + + + + + +Methods inherited from classorg.apache.beam.sdk.coders.CustomCoder +asCloudObject, getEncodingId, of, verifyDeterministic + + + + + +Methods inherited from classorg.apache.beam.sdk.coders.AtomicCoder +getCoderArguments, getInstanceComponents + + + + + +Methods inherited from classorg.apache.beam.sdk.coders.StandardCoder +consistentWithEquals, equals, getAllowedEncodings, getComponents, getEncodedElementByteSize, hashCode, isRegisterByteSizeObserverCheap, registerByteSizeObserver, structuralValue, toString, verifyDeterministic, verifyDeterministic + + + + + +Methods inherited from classjava.lang.http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Object
[46/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/index-all.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/index-all.html b/content/documentation/sdks/javadoc/0.4.0/index-all.html new file mode 100644 index 000..c3ed2ab --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/index-all.html @@ -0,0 +1,21330 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +Index (Apache Beam SDK for Java, version 0.4.0) + + + + + + + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +Prev +Next + + +Frames +NoFrames + + +AllClasses + + + + + + + + + +ABCDEFGHIJKLMNOPQRSTUVWXYZ + + +A + +absolute(String, String...) - Static method in class org.apache.beam.sdk.transforms.display.DisplayData.Path + +Construct a path from an absolute component path hierarchy. + +access(StateTag? super K, StateT) - Method in interface org.apache.beam.sdk.util.state.StateAccessor + +Access the storage for the given address in the current window. + +accessInEachMergingWindow(StateTag? super K, StateT) - Method in interface org.apache.beam.sdk.util.state.MergingStateAccessor + +Analogous to StateAccessor.access(org.apache.beam.sdk.util.state.StateTag? super K, StateT), but returned as a map from each window which is + about to be merged to the corresponding state. + +AccumulatingCombineFn() - Constructor for class org.apache.beam.sdk.transforms.Combine.AccumulatingCombineFn + +accumulatingFiredPanes() - Static method in class org.apache.beam.sdk.transforms.windowing.Window + +Returns a new Window PTransform that uses the registered WindowFn and + Triggering behavior, and that accumulates elements in a pane after they are triggered. + +accumulatingFiredPanes() - Method in class org.apache.beam.sdk.transforms.windowing.Window.Bound + +Returns a new Window PTransform that uses the registered WindowFn and + Triggering behavior, and that accumulates elements in a pane after they are triggered. + +AccumulatorCombiningStateInputT,AccumT,OutputT - Interface in org.apache.beam.sdk.util.state + +State for a single value that is managed by a Combine.CombineFn. + +ackDeadlineSeconds(PubsubClient.SubscriptionPath) - Method in class org.apache.beam.sdk.util.PubsubClient + +Return the ack deadline, in seconds, for subscription. + +ackDeadlineSeconds(PubsubClient.SubscriptionPath) - Method in class org.apache.beam.sdk.util.PubsubGrpcClient + +ackDeadlineSeconds(PubsubClient.SubscriptionPath) - Method in class org.apache.beam.sdk.util.PubsubJsonClient + +ackDeadlineSeconds(PubsubClient.SubscriptionPath) - Method in class org.apache.beam.sdk.util.PubsubTestClient + +ackId - Variable in class org.apache.beam.sdk.util.PubsubClient.IncomingMessage + +Id to pass back to Pubsub to acknowledge receipt of this message. + +acknowledge(PubsubClient.SubscriptionPath, ListString) - Method in class org.apache.beam.sdk.util.PubsubClient + +Acknowldege messages from subscription with ackIds. + +acknowledge(PubsubClient.SubscriptionPath, ListString) - Method in class org.apache.beam.sdk.util.PubsubGrpcClient + +acknowledge(PubsubClient.SubscriptionPath, ListString) - Method in class org.apache.beam.sdk.util.PubsubJsonClient + +acknowledge(PubsubClient.SubscriptionPath, ListString) - Method in class org.apache.beam.sdk.util.PubsubTestClient + +ActiveWindowSetW extends BoundedWindow - Interface in org.apache.beam.sdk.util + +Track which windows are active, and the state address window(s) under which their + state is stored. + +ActiveWindowSet.MergeCallbackW extends BoundedWindow - Interface in org.apache.beam.sdk.util + +Callback for ActiveWindowSet.merge(org.apache.beam.sdk.util.ActiveWindowSet.MergeCallbackW). + +adaptContext(OldDoFnInputT, OutputT, DoFnInputT, OutputT.Context) - Static method in class org.apache.beam.sdk.transforms.DoFnAdapters + +Deprecated. +Creates a OldDoFn.ProcessContext from a DoFn.ProcessContext. + +adaptProcessContext(OldDoFnInputT, OutputT, DoFnInputT, OutputT.ProcessContext, DoFnInvoker.ArgumentProviderInputT, OutputT) - Static method in class org.apache.beam.sdk.transforms.DoFnAdapters + +Deprecated. +Creates a OldDoFn.ProcessContext from a DoFn.ProcessContext. + +adaptTagBinder(StateTag.StateBinderK) - Static method in class org.apache.beam.sdk.util.state.StateSpecs + +Deprecated. +for migration purposes only + + +add(KVbyte[], byte[]) -
[11/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/TriggerStateMachineRunner.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/TriggerStateMachineRunner.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/TriggerStateMachineRunner.html new file mode 100644 index 000..19ca594 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/TriggerStateMachineRunner.html @@ -0,0 +1,567 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +TriggerStateMachineRunner (Apache Beam SDK for Java, version 0.4.0) + + + + + + +var methods = {"i0":10,"i1":10,"i2":10,"i3":10,"i4":10,"i5":10,"i6":10,"i7":10,"i8":10,"i9":10,"i10":10,"i11":10}; +var tabs = {65535:["t0","All Methods"],2:["t2","Instance Methods"],8:["t4","Concrete Methods"]}; +var altColor = "altColor"; +var rowColor = "rowColor"; +var tableTab = "tableTab"; +var activeTableTab = "activeTableTab"; + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +PrevClass +NextClass + + +Frames +NoFrames + + +AllClasses + + + + + + + +Summary: +Nested| +Field| +Constr| +Method + + +Detail: +Field| +Constr| +Method + + + + + + + + +org.apache.beam.runners.core.triggers +Class TriggerStateMachineRunnerW extends BoundedWindow + + + +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">java.lang.Object + + +org.apache.beam.runners.core.triggers.TriggerStateMachineRunnerW + + + + + + + +Type Parameters: +W - The kind of windows being processed. + + + +public class TriggerStateMachineRunnerW extends BoundedWindow +extends http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Object +Executes a trigger while managing persistence of information about which subtriggers are + finished. Subtriggers include all recursive trigger expressions as well as the entire trigger. + + Specifically, the responsibilities are: + + + Invoking the trigger's methods via its ExecutableTriggerStateMachine wrapper by + constructing the appropriate trigger contexts. + Committing a record of which subtriggers are finished to persistent state. + Restoring the record of which subtriggers are finished from persistent state. + Clearing out the persisted finished set when a caller indicates + (via {#link #clearFinished}) that it is no longer needed. + + + These responsibilities are intertwined: trigger contexts include mutable information about + which subtriggers are finished. This class provides the information when building the contexts + and commits the information when the method of the ExecutableTriggerStateMachine returns. + + + + + + + + + + + +Field Summary + +Fields + +Modifier and Type +Field and Description + + +static StateTaghttp://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Object,ValueStatehttp://docs.oracle.com/javase/7/docs/api/java/util/BitSet.html?is-external=true; title="class or interface in java.util">BitSet +FINISHED_BITS_TAG + + + + + + + + + +Constructor Summary + +Constructors + +Constructor and Description + + +TriggerStateMachineRunner(ExecutableTriggerStateMachinerootTrigger, + TriggerStateMachineContextFactoryWcontextFactory) + + + + + + + + + +Method Summary + +All MethodsInstance MethodsConcrete Methods + +Modifier and Type +Method and Description + + +void +clearFinished(StateAccessor?state) +Clear the finished bits. + + + +void +clearState(Wwindow, + Timerstimers, + StateAccessor?state) +Clear the state used for executing triggers, but leave the finished set to indicate + the window is closed. + + + +boolean +isClosed(StateAccessor?state) +Return true if the trigger is closed in the window corresponding to the specified state. + + + +void +onFire(Wwindow, + Timerstimers, + StateAccessor?state) + + +void +onMerge(Wwindow, + Timerstimers, + MergingStateAccessor?,Wstate) +Run the trigger merging logic as part of executing the specified merge. + + + +void +prefetchForMerge(Wwindow, +
[10/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/class-use/AfterPaneStateMachine.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/class-use/AfterPaneStateMachine.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/class-use/AfterPaneStateMachine.html new file mode 100644 index 000..ce9ce50 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/class-use/AfterPaneStateMachine.html @@ -0,0 +1,171 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +Uses of Class org.apache.beam.runners.core.triggers.AfterPaneStateMachine (Apache Beam SDK for Java, version 0.4.0) + + + + + + + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +Prev +Next + + +Frames +NoFrames + + +AllClasses + + + + + + + + + + +Uses of Classorg.apache.beam.runners.core.triggers.AfterPaneStateMachine + + + + + +Packages that use AfterPaneStateMachine + +Package +Description + + + +org.apache.beam.runners.core.triggers + +State machine implementations for triggers, called "triggers" because + they react to events. + + + + + + + + + + +Uses of AfterPaneStateMachine in org.apache.beam.runners.core.triggers + +Methods in org.apache.beam.runners.core.triggers that return AfterPaneStateMachine + +Modifier and Type +Method and Description + + + +static AfterPaneStateMachine +AfterPaneStateMachine.elementCountAtLeast(intcountElems) +Creates a trigger that fires when the pane contains at least countElems elements. + + + + + + + + + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +Prev +Next + + +Frames +NoFrames + + +AllClasses + + + + + + + + + +Copyright 2016 https://www.apache.org/;>The Apache Software Foundation. All rights reserved. + + http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/class-use/AfterProcessingTimeStateMachine.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/class-use/AfterProcessingTimeStateMachine.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/class-use/AfterProcessingTimeStateMachine.html new file mode 100644 index 000..cbbbff7 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/class-use/AfterProcessingTimeStateMachine.html @@ -0,0 +1,176 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +Uses of Class org.apache.beam.runners.core.triggers.AfterProcessingTimeStateMachine (Apache Beam SDK for Java, version 0.4.0) + + + + + + + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +Prev +Next + + +Frames +NoFrames + + +AllClasses + + + + + + + + + + +Uses of Classorg.apache.beam.runners.core.triggers.AfterProcessingTimeStateMachine + + + + + +Packages that use AfterProcessingTimeStateMachine + +Package +Description + + + +org.apache.beam.runners.core.triggers + +State machine implementations for triggers, called "triggers" because + they react to events. + + + + + + + + + + +Uses of AfterProcessingTimeStateMachine in org.apache.beam.runners.core.triggers + +Methods in org.apache.beam.runners.core.triggers that return AfterProcessingTimeStateMachine +
[38/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/GroupAlsoByWindowsDoFn.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/GroupAlsoByWindowsDoFn.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/GroupAlsoByWindowsDoFn.html new file mode 100644 index 000..eec7d61 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/GroupAlsoByWindowsDoFn.html @@ -0,0 +1,371 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +GroupAlsoByWindowsDoFn (Apache Beam SDK for Java, version 0.4.0) + + + + + + + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +PrevClass +NextClass + + +Frames +NoFrames + + +AllClasses + + + + + + + +Summary: +Nested| +Field| +Constr| +Method + + +Detail: +Field| +Constr| +Method + + + + + + + + +org.apache.beam.runners.core +Class GroupAlsoByWindowsDoFnK,InputT,OutputT,W extends BoundedWindow + + + +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">java.lang.Object + + +org.apache.beam.sdk.transforms.OldDoFnKVK,http://docs.oracle.com/javase/7/docs/api/java/lang/Iterable.html?is-external=true; title="class or interface in java.lang">IterableWindowedValueInputT,KVK,OutputT + + +org.apache.beam.runners.core.GroupAlsoByWindowsDoFnK,InputT,OutputT,W + + + + + + + + + +Type Parameters: +K - key type +InputT - input value element type +OutputT - output value element type +W - window type + + +All Implemented Interfaces: +http://docs.oracle.com/javase/7/docs/api/java/io/Serializable.html?is-external=true; title="class or interface in java.io">Serializable, HasDisplayData + + +Direct Known Subclasses: +GroupAlsoByWindowsViaOutputBufferDoFn + + + +@SystemDoFnInternal +public abstract class GroupAlsoByWindowsDoFnK,InputT,OutputT,W extends BoundedWindow +extends OldDoFnKVK,http://docs.oracle.com/javase/7/docs/api/java/lang/Iterable.html?is-external=true; title="class or interface in java.lang">IterableWindowedValueInputT,KVK,OutputT +OldDoFn that merges windows and groups elements in those windows, optionally + combining values. + +See Also: +Serialized Form + + + + + + + + + + + + +Nested Class Summary + + + + +Nested classes/interfaces inherited from classorg.apache.beam.sdk.transforms.OldDoFn +OldDoFn.Context, OldDoFn.ProcessContext, OldDoFn.RequiresWindowAccess + + + + + + + + +Field Summary + +Fields + +Modifier and Type +Field and Description + + +static http://docs.oracle.com/javase/7/docs/api/java/lang/String.html?is-external=true; title="class or interface in java.lang">String +DROPPED_DUE_TO_CLOSED_WINDOW_COUNTER + + +static http://docs.oracle.com/javase/7/docs/api/java/lang/String.html?is-external=true; title="class or interface in java.lang">String +DROPPED_DUE_TO_LATENESS_COUNTER + + +protected Aggregatorhttp://docs.oracle.com/javase/7/docs/api/java/lang/Long.html?is-external=true; title="class or interface in java.lang">Long,http://docs.oracle.com/javase/7/docs/api/java/lang/Long.html?is-external=true; title="class or interface in java.lang">Long +droppedDueToClosedWindow + + +protected Aggregatorhttp://docs.oracle.com/javase/7/docs/api/java/lang/Long.html?is-external=true; title="class or interface in java.lang">Long,http://docs.oracle.com/javase/7/docs/api/java/lang/Long.html?is-external=true; title="class or interface in java.lang">Long +droppedDueToLateness + + + + + + + + + +Constructor Summary + +Constructors + +Constructor and Description + + +GroupAlsoByWindowsDoFn() + + + + + + + + + +Method Summary + + + + +Methods inherited from classorg.apache.beam.sdk.transforms.OldDoFn +createAggregator, createAggregator, finishBundle, getAllowedTimestampSkew, getInputTypeDescriptor, getOutputTypeDescriptor, populateDisplayData, processElement, setup, startBundle, teardown, toDoFn + + + + + +Methods inherited from classjava.lang.http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Object +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#clone--; title="class or interface in java.lang">clone, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#equals-java.lang.Object-;
[06/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/class-use/TriggerStateMachine.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/class-use/TriggerStateMachine.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/class-use/TriggerStateMachine.html new file mode 100644 index 000..cb547dd --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/class-use/TriggerStateMachine.html @@ -0,0 +1,386 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +Uses of Class org.apache.beam.runners.core.triggers.TriggerStateMachine (Apache Beam SDK for Java, version 0.4.0) + + + + + + + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +Prev +Next + + +Frames +NoFrames + + +AllClasses + + + + + + + + + + +Uses of Classorg.apache.beam.runners.core.triggers.TriggerStateMachine + + + + + +Packages that use TriggerStateMachine + +Package +Description + + + +org.apache.beam.runners.core.triggers + +State machine implementations for triggers, called "triggers" because + they react to events. + + + + + + + + + + +Uses of TriggerStateMachine in org.apache.beam.runners.core.triggers + +Subclasses of TriggerStateMachine in org.apache.beam.runners.core.triggers + +Modifier and Type +Class and Description + + + +class +AfterAllStateMachine +A TriggerStateMachine that fires and finishes once after all of its sub-triggers + have fired. + + + +class +AfterDelayFromFirstElementStateMachine +A base class for triggers that happen after a processing time delay from the arrival + of the first element in a pane. + + + +class +AfterEachStateMachine +A composite TriggerStateMachine that executes its sub-triggers in order. + + + +class +AfterFirstStateMachine +Create a composite TriggerStateMachine that fires once after at least one of its + sub-triggers have fired. + + + +class +AfterPaneStateMachine +TriggerStateMachines that fire based on properties of the elements in the current pane. + + + +class +AfterProcessingTimeStateMachine +AfterProcessingTime triggers fire based on the current processing time. + + + +static class +AfterWatermarkStateMachine.AfterWatermarkEarlyAndLate + + +static class +AfterWatermarkStateMachine.FromEndOfWindow +A watermark trigger targeted relative to the end of the window. + + + +class +DefaultTriggerStateMachine +A trigger that is equivalent to Repeatedly.forever(AfterWatermark.pastEndOfWindow()). + + + +class +NeverStateMachine +A TriggerStateMachine which never fires. + + + +class +RepeatedlyStateMachine +Repeat a trigger, either until some condition is met or forever. + + + +class +ReshuffleTriggerStateMachine +The trigger used with Reshuffle which triggers on every element + and never buffers state. + + + +static class +TriggerStateMachine.OnceTriggerStateMachine +TriggerStateMachines that are guaranteed to fire at most once should extend from this, + rather than the general TriggerStateMachine class to indicate that behavior. + + + + + +Fields in org.apache.beam.runners.core.triggers with type parameters of type TriggerStateMachine + +Modifier and Type +Field and Description + + + +protected http://docs.oracle.com/javase/7/docs/api/java/util/List.html?is-external=true; title="class or interface in java.util">ListTriggerStateMachine +TriggerStateMachine.subTriggers + + + + +Methods in org.apache.beam.runners.core.triggers that return TriggerStateMachine + +Modifier and Type +Method and Description + + + +TriggerStateMachine +ExecutableTriggerStateMachine.getSpec() +Return the underlying trigger specification corresponding to this ExecutableTrigger. + + + +static TriggerStateMachine +AfterEachStateMachine.inOrder(http://docs.oracle.com/javase/7/docs/api/java/lang/Iterable.html?is-external=true; title="class or interface in java.lang">Iterable? extends TriggerStateMachinetriggers) + + +static TriggerStateMachine +AfterEachStateMachine.inOrder(TriggerStateMachine...triggers) +Returns an AfterEach Trigger with the given subtriggers. + + + +TriggerStateMachine +TriggerStateMachine.orFinally(TriggerStateMachine.OnceTriggerStateMachineuntil) +Specify an ending condition for this trigger. + + + +static
[1/2] beam git commit: Make DataflowPipelineTranslator public
Repository: beam Updated Branches: refs/heads/master 70dd54625 -> b81bd25e1 Make DataflowPipelineTranslator public There are a couple places outside the Beam codebase that still refer to this, and no great reason to break them. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/0ca0bdbe Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/0ca0bdbe Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/0ca0bdbe Branch: refs/heads/master Commit: 0ca0bdbe00381ae6f1b1ddf2c31b4e7b2cbb342f Parents: 70dd546 Author: Kenneth KnowlesAuthored: Mon Jan 9 13:37:55 2017 -0800 Committer: Kenneth Knowles Committed: Mon Jan 9 14:18:17 2017 -0800 -- .../apache/beam/runners/dataflow/DataflowPipelineTranslator.java | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/beam/blob/0ca0bdbe/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowPipelineTranslator.java -- diff --git a/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowPipelineTranslator.java b/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowPipelineTranslator.java index 524c30b..bd0d5ba 100644 --- a/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowPipelineTranslator.java +++ b/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowPipelineTranslator.java @@ -41,6 +41,7 @@ import com.google.api.services.dataflow.model.Environment; import com.google.api.services.dataflow.model.Job; import com.google.api.services.dataflow.model.Step; import com.google.api.services.dataflow.model.WorkerPool; +import com.google.common.annotations.VisibleForTesting; import com.google.common.base.Supplier; import com.google.common.collect.BiMap; import com.google.common.collect.ImmutableBiMap; @@ -106,7 +107,8 @@ import org.slf4j.LoggerFactory; * into Cloud Dataflow Service API {@link Job}s. */ @SuppressWarnings({"rawtypes", "unchecked"}) -class DataflowPipelineTranslator { +@VisibleForTesting +public class DataflowPipelineTranslator { // Must be kept in sync with their internal counterparts. private static final Logger LOG = LoggerFactory.getLogger(DataflowPipelineTranslator.class); private static final ObjectMapper MAPPER = new ObjectMapper();
[27/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/class-use/DoFnRunner.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/class-use/DoFnRunner.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/class-use/DoFnRunner.html new file mode 100644 index 000..2033d83 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/class-use/DoFnRunner.html @@ -0,0 +1,301 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +Uses of Interface org.apache.beam.runners.core.DoFnRunner (Apache Beam SDK for Java, version 0.4.0) + + + + + + + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +Prev +Next + + +Frames +NoFrames + + +AllClasses + + + + + + + + + + +Uses of Interfaceorg.apache.beam.runners.core.DoFnRunner + + + + + +Packages that use DoFnRunner + +Package +Description + + + +org.apache.beam.runners.core + +Provides utilities for Beam runner authors. + + + + + + + + + + +Uses of DoFnRunner in org.apache.beam.runners.core + +Classes in org.apache.beam.runners.core that implement DoFnRunner + +Modifier and Type +Class and Description + + + +class +LateDataDroppingDoFnRunnerK,InputT,OutputT,W extends BoundedWindow +A customized DoFnRunner that handles late data dropping for + a KeyedWorkItem input OldDoFn. + + + +class +PushbackSideInputDoFnRunnerInputT,OutputT +A DoFnRunner that can refuse to process elements that are not ready, instead returning + them via the PushbackSideInputDoFnRunner.processElementInReadyWindows(WindowedValue). + + + +class +SimpleDoFnRunnerInputT,OutputT +Runs a DoFn by constructing the appropriate contexts and passing them in. + + + + + +Methods in org.apache.beam.runners.core that return DoFnRunner + +Modifier and Type +Method and Description + + + +static InputT,OutputTDoFnRunnerInputT,OutputT +DoFnRunners.createDefault(PipelineOptionsoptions, + DoFnInputT,OutputTdoFn, + SideInputReadersideInputReader, + DoFnRunners.OutputManageroutputManager, + TupleTagOutputTmainOutputTag, + http://docs.oracle.com/javase/7/docs/api/java/util/List.html?is-external=true; title="class or interface in java.util">ListTupleTag?sideOutputTags, + ExecutionContext.StepContextstepContext, + Aggregator.AggregatorFactoryaggregatorFactory, + WindowingStrategy?,?windowingStrategy) +Creates a DoFnRunner for the provided DoFn. + + + +static InputT,OutputTDoFnRunnerInputT,OutputT +DoFnRunners.createDefault(PipelineOptionsoptions, + http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">ObjectdeserializedFn, + SideInputReadersideInputReader, + DoFnRunners.OutputManageroutputManager, + TupleTagOutputTmainOutputTag, + http://docs.oracle.com/javase/7/docs/api/java/util/List.html?is-external=true; title="class or interface in java.util">ListTupleTag?sideOutputTags, + ExecutionContext.StepContextstepContext, + Aggregator.AggregatorFactoryaggregatorFactory, + WindowingStrategy?,?windowingStrategy) +Deprecated. +for migration purposes only for services where users may still submit either OldDoFn or DoFn. If you know that you have a DoFn then you should use the + variant for that instead. + + + + +static InputT,OutputTDoFnRunnerInputT,OutputT +DoFnRunners.createDefault(PipelineOptionsoptions, + OldDoFnInputT,OutputTdoFn, + SideInputReadersideInputReader, + DoFnRunners.OutputManageroutputManager, + TupleTagOutputTmainOutputTag, + http://docs.oracle.com/javase/7/docs/api/java/util/List.html?is-external=true; title="class or interface in java.util">ListTupleTag?sideOutputTags, + ExecutionContext.StepContextstepContext, + Aggregator.AggregatorFactoryaggregatorFactory, + WindowingStrategy?,?windowingStrategy) +Deprecated. +please port uses of OldDoFn to use DoFn + + + + +static K,InputT,OutputT,W extends BoundedWindowDoFnRunnerKeyedWorkItemK,InputT,KVK,OutputT
[42/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/apex/class-use/ApexYarnLauncher.LaunchParams.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/apex/class-use/ApexYarnLauncher.LaunchParams.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/apex/class-use/ApexYarnLauncher.LaunchParams.html new file mode 100644 index 000..bca02dc --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/apex/class-use/ApexYarnLauncher.LaunchParams.html @@ -0,0 +1,168 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +Uses of Class org.apache.beam.runners.apex.ApexYarnLauncher.LaunchParams (Apache Beam SDK for Java, version 0.4.0) + + + + + + + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +Prev +Next + + +Frames +NoFrames + + +AllClasses + + + + + + + + + + +Uses of Classorg.apache.beam.runners.apex.ApexYarnLauncher.LaunchParams + + + + + +Packages that use ApexYarnLauncher.LaunchParams + +Package +Description + + + +org.apache.beam.runners.apex + +Implementation of the Beam runner for Apache Apex. + + + + + + + + + + +Uses of ApexYarnLauncher.LaunchParams in org.apache.beam.runners.apex + +Methods in org.apache.beam.runners.apex with parameters of type ApexYarnLauncher.LaunchParams + +Modifier and Type +Method and Description + + + +protected org.apache.apex.api.Launcher.AppHandle +ApexYarnLauncher.launchApp(ApexYarnLauncher.LaunchParamsparams) + + + + + + + + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +Prev +Next + + +Frames +NoFrames + + +AllClasses + + + + + + + + + +Copyright 2016 https://www.apache.org/;>The Apache Software Foundation. All rights reserved. + + http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/apex/class-use/ApexYarnLauncher.ProcessWatcher.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/apex/class-use/ApexYarnLauncher.ProcessWatcher.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/apex/class-use/ApexYarnLauncher.ProcessWatcher.html new file mode 100644 index 000..8c10439 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/apex/class-use/ApexYarnLauncher.ProcessWatcher.html @@ -0,0 +1,126 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +Uses of Class org.apache.beam.runners.apex.ApexYarnLauncher.ProcessWatcher (Apache Beam SDK for Java, version 0.4.0) + + + + + + + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +Prev +Next + + +Frames +NoFrames + + +AllClasses + + + + + + + + + + +Uses of Classorg.apache.beam.runners.apex.ApexYarnLauncher.ProcessWatcher + +No usage of org.apache.beam.runners.apex.ApexYarnLauncher.ProcessWatcher + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +Prev +Next + + +Frames +NoFrames + + +AllClasses + + + + + + + + + +Copyright 2016 https://www.apache.org/;>The Apache Software Foundation. All rights reserved. + +
[43/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/apex/ApexYarnLauncher.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/apex/ApexYarnLauncher.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/apex/ApexYarnLauncher.html new file mode 100644 index 000..1e510f6 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/apex/ApexYarnLauncher.html @@ -0,0 +1,405 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +ApexYarnLauncher (Apache Beam SDK for Java, version 0.4.0) + + + + + + +var methods = {"i0":9,"i1":9,"i2":10,"i3":10,"i4":9}; +var tabs = {65535:["t0","All Methods"],1:["t1","Static Methods"],2:["t2","Instance Methods"],8:["t4","Concrete Methods"]}; +var altColor = "altColor"; +var rowColor = "rowColor"; +var tableTab = "tableTab"; +var activeTableTab = "activeTableTab"; + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +PrevClass +NextClass + + +Frames +NoFrames + + +AllClasses + + + + + + + +Summary: +Nested| +Field| +Constr| +Method + + +Detail: +Field| +Constr| +Method + + + + + + + + +org.apache.beam.runners.apex +Class ApexYarnLauncher + + + +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">java.lang.Object + + +org.apache.beam.runners.apex.ApexYarnLauncher + + + + + + + + +public class ApexYarnLauncher +extends http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Object +Proxy to launch the YARN application through the hadoop script to run in the + pre-configured environment (class path, configuration, native libraries etc.). + + The proxy takes the DAG and communicates with the Hadoop services to launch + it on the cluster. + + + + + + + + + + + +Nested Class Summary + +Nested Classes + +Modifier and Type +Class and Description + + +protected static class +ApexYarnLauncher.LaunchParams +Launch parameters that will be serialized and passed to the child process. + + + +static class +ApexYarnLauncher.ProcessWatcher +Starts a command and waits for it to complete. + + + + + + + + + + +Constructor Summary + +Constructors + +Constructor and Description + + +ApexYarnLauncher() + + + + + + + + + +Method Summary + +All MethodsStatic MethodsInstance MethodsConcrete Methods + +Modifier and Type +Method and Description + + +static void +createJar(http://docs.oracle.com/javase/7/docs/api/java/io/File.html?is-external=true; title="class or interface in java.io">Filedir, + http://docs.oracle.com/javase/7/docs/api/java/io/File.html?is-external=true; title="class or interface in java.io">FilejarFile) +Create a jar file from the given directory. + + + +static http://docs.oracle.com/javase/7/docs/api/java/util/List.html?is-external=true; title="class or interface in java.util">Listhttp://docs.oracle.com/javase/7/docs/api/java/io/File.html?is-external=true; title="class or interface in java.io">File +getYarnDeployDependencies() +From the current classpath, find the jar files that need to be deployed + with the application to run on YARN. + + + +protected org.apache.apex.api.Launcher.AppHandle +launchApp(ApexYarnLauncher.LaunchParamsparams) + + +org.apache.apex.api.Launcher.AppHandle +launchApp(com.datatorrent.api.StreamingApplicationapp) + + +static void +main(http://docs.oracle.com/javase/7/docs/api/java/lang/String.html?is-external=true; title="class or interface in java.lang">String[]args) +The main method expects the serialized DAG and will launch the YARN application. + + + + + + + +Methods inherited from classjava.lang.http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Object +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#clone--; title="class or interface in java.lang">clone, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#equals-java.lang.Object-; title="class or interface in java.lang">equals, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#finalize--; title="class or interface in java.lang">finalize, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#getClass--; title="class
[12/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/TriggerStateMachine.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/TriggerStateMachine.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/TriggerStateMachine.html new file mode 100644 index 000..a86eb1f --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/TriggerStateMachine.html @@ -0,0 +1,739 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +TriggerStateMachine (Apache Beam SDK for Java, version 0.4.0) + + + + + + +var methods = {"i0":10,"i1":10,"i2":10,"i3":10,"i4":6,"i5":6,"i6":6,"i7":10,"i8":10,"i9":10,"i10":10,"i11":10,"i12":6,"i13":10,"i14":10}; +var tabs = {65535:["t0","All Methods"],2:["t2","Instance Methods"],4:["t3","Abstract Methods"],8:["t4","Concrete Methods"]}; +var altColor = "altColor"; +var rowColor = "rowColor"; +var tableTab = "tableTab"; +var activeTableTab = "activeTableTab"; + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +PrevClass +NextClass + + +Frames +NoFrames + + +AllClasses + + + + + + + +Summary: +Nested| +Field| +Constr| +Method + + +Detail: +Field| +Constr| +Method + + + + + + + + +org.apache.beam.runners.core.triggers +Class TriggerStateMachine + + + +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">java.lang.Object + + +org.apache.beam.runners.core.triggers.TriggerStateMachine + + + + + + + +All Implemented Interfaces: +http://docs.oracle.com/javase/7/docs/api/java/io/Serializable.html?is-external=true; title="class or interface in java.io">Serializable + + +Direct Known Subclasses: +AfterEachStateMachine, AfterWatermarkStateMachine.AfterWatermarkEarlyAndLate, DefaultTriggerStateMachine, RepeatedlyStateMachine, ReshuffleTriggerStateMachine, TriggerStateMachine.OnceTriggerStateMachine + + + +public abstract class TriggerStateMachine +extends http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Object +implements http://docs.oracle.com/javase/7/docs/api/java/io/Serializable.html?is-external=true; title="class or interface in java.io">Serializable +Triggers control when the elements for a specific key and window are output. As elements + arrive, they are put into one or more windows by a Window transform and its associated + WindowFn, and then passed to the associated Trigger to determine if the Windows contents should be output. + + See GroupByKey and Window for more information + about how grouping with windows works. + + The elements that are assigned to a window since the last time it was fired (or since the + window was created) are placed into the current window pane. Triggers are evaluated against the + elements as they are added. When the root trigger fires, the elements in the current pane will be + output. When the root trigger finishes (indicating it will never fire again), the window is + closed and any new elements assigned to that window are discarded. + + Several predefined Triggers are provided: + + + AfterWatermarkStateMachine for firing when the watermark passes a timestamp + determined from either the end of the window or the arrival of the first element in a pane. + AfterProcessingTimeStateMachine for firing after some amount of processing time has + elapsed (typically since the first element in a pane). + AfterPaneStateMachine for firing off a property of the elements in the current pane, + such as the number of elements that have been assigned to the current pane. + + + In addition, Triggers can be combined in a variety of ways: + + + RepeatedlyStateMachine.forever(org.apache.beam.runners.core.triggers.TriggerStateMachine) to create a trigger that executes forever. Any time + its argument finishes it gets reset and starts over. Can be combined with orFinally(org.apache.beam.runners.core.triggers.TriggerStateMachine.OnceTriggerStateMachine) to specify a condition that causes the repetition to stop. + AfterEachStateMachine.inOrder(org.apache.beam.runners.core.triggers.TriggerStateMachine...) to execute each trigger in sequence, firing each (and + every) time that a trigger
[13/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/TriggerStateMachine.TriggerContext.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/TriggerStateMachine.TriggerContext.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/TriggerStateMachine.TriggerContext.html new file mode 100644 index 000..f1fc252 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/TriggerStateMachine.TriggerContext.html @@ -0,0 +1,405 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +TriggerStateMachine.TriggerContext (Apache Beam SDK for Java, version 0.4.0) + + + + + + +var methods = {"i0":6,"i1":6,"i2":6,"i3":6,"i4":6,"i5":6,"i6":6,"i7":6}; +var tabs = {65535:["t0","All Methods"],2:["t2","Instance Methods"],4:["t3","Abstract Methods"]}; +var altColor = "altColor"; +var rowColor = "rowColor"; +var tableTab = "tableTab"; +var activeTableTab = "activeTableTab"; + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +PrevClass +NextClass + + +Frames +NoFrames + + +AllClasses + + + + + + + +Summary: +Nested| +Field| +Constr| +Method + + +Detail: +Field| +Constr| +Method + + + + + + + + +org.apache.beam.runners.core.triggers +Class TriggerStateMachine.TriggerContext + + + +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">java.lang.Object + + +org.apache.beam.runners.core.triggers.TriggerStateMachine.TriggerContext + + + + + + + +Direct Known Subclasses: +TriggerStateMachine.OnElementContext, TriggerStateMachine.OnMergeContext + + +Enclosing class: +TriggerStateMachine + + + +public abstract class TriggerStateMachine.TriggerContext +extends http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Object +Information accessible to all operational hooks in this Trigger. + + Used directly in TriggerStateMachine.shouldFire(org.apache.beam.runners.core.triggers.TriggerStateMachine.TriggerContext) and TriggerStateMachine.clear(org.apache.beam.runners.core.triggers.TriggerStateMachine.TriggerContext), and extended with additional information in other methods. + + + + + + + + + + + +Constructor Summary + +Constructors + +Constructor and Description + + +TriggerContext() + + + + + + + + + +Method Summary + +All MethodsInstance MethodsAbstract Methods + +Modifier and Type +Method and Description + + +abstract org.joda.time.Instant +currentEventTime() +The current event time for the input or null if unknown. + + + +abstract org.joda.time.Instant +currentProcessingTime() +The current processing time. + + + +abstract org.joda.time.Instant +currentSynchronizedProcessingTime() +The current synchronized upstream processing time or null if unknown. + + + +abstract void +deleteTimer(org.joda.time.Instanttimestamp, + TimeDomaindomain) +Removes the timer set in this trigger context for the given Instant + and TimeDomain. + + + +abstract TriggerStateMachine.TriggerContext +forTrigger(ExecutableTriggerStateMachinetrigger) +Create a sub-context for the given sub-trigger. + + + +abstract StateAccessor? +state() +Returns the interface for accessing persistent state. + + + +abstract TriggerStateMachine.TriggerInfo +trigger() +Returns the interface for accessing trigger info. + + + +abstract BoundedWindow +window() +The window that the current context is executing in. + + + + + + + +Methods inherited from classjava.lang.http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Object +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#clone--; title="class or interface in java.lang">clone, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#equals-java.lang.Object-; title="class or interface in java.lang">equals, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#finalize--; title="class or interface in java.lang">finalize, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#getClass--; title="class or interface in java.lang">getClass,
[40/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/DoFnRunners.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/DoFnRunners.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/DoFnRunners.html new file mode 100644 index 000..4362657 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/DoFnRunners.html @@ -0,0 +1,453 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +DoFnRunners (Apache Beam SDK for Java, version 0.4.0) + + + + + + +var methods = {"i0":9,"i1":41,"i2":41,"i3":9,"i4":9}; +var tabs = {65535:["t0","All Methods"],1:["t1","Static Methods"],8:["t4","Concrete Methods"],32:["t6","Deprecated Methods"]}; +var altColor = "altColor"; +var rowColor = "rowColor"; +var tableTab = "tableTab"; +var activeTableTab = "activeTableTab"; + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +PrevClass +NextClass + + +Frames +NoFrames + + +AllClasses + + + + + + + +Summary: +Nested| +Field| +Constr| +Method + + +Detail: +Field| +Constr| +Method + + + + + + + + +org.apache.beam.runners.core +Class DoFnRunners + + + +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">java.lang.Object + + +org.apache.beam.runners.core.DoFnRunners + + + + + + + + +public class DoFnRunners +extends http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Object +Static utility methods that provide DoFnRunner implementations. + + + + + + + + + + + +Nested Class Summary + +Nested Classes + +Modifier and Type +Class and Description + + +static interface +DoFnRunners.OutputManager +Information about how to create output receivers and output to them. + + + + + + + + + + +Constructor Summary + +Constructors + +Constructor and Description + + +DoFnRunners() + + + + + + + + + +Method Summary + +All MethodsStatic MethodsConcrete MethodsDeprecated Methods + +Modifier and Type +Method and Description + + +static InputT,OutputTDoFnRunnerInputT,OutputT +createDefault(PipelineOptionsoptions, + DoFnInputT,OutputTdoFn, + SideInputReadersideInputReader, + DoFnRunners.OutputManageroutputManager, + TupleTagOutputTmainOutputTag, + http://docs.oracle.com/javase/7/docs/api/java/util/List.html?is-external=true; title="class or interface in java.util">ListTupleTag?sideOutputTags, + ExecutionContext.StepContextstepContext, + Aggregator.AggregatorFactoryaggregatorFactory, + WindowingStrategy?,?windowingStrategy) +Creates a DoFnRunner for the provided DoFn. + + + +static InputT,OutputTDoFnRunnerInputT,OutputT +createDefault(PipelineOptionsoptions, + http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">ObjectdeserializedFn, + SideInputReadersideInputReader, + DoFnRunners.OutputManageroutputManager, + TupleTagOutputTmainOutputTag, + http://docs.oracle.com/javase/7/docs/api/java/util/List.html?is-external=true; title="class or interface in java.util">ListTupleTag?sideOutputTags, + ExecutionContext.StepContextstepContext, + Aggregator.AggregatorFactoryaggregatorFactory, + WindowingStrategy?,?windowingStrategy) +Deprecated. +for migration purposes only for services where users may still submit either OldDoFn or DoFn. If you know that you have a DoFn then you should use the + variant for that instead. + + + + +static InputT,OutputTDoFnRunnerInputT,OutputT +createDefault(PipelineOptionsoptions, + OldDoFnInputT,OutputTdoFn, + SideInputReadersideInputReader, + DoFnRunners.OutputManageroutputManager, + TupleTagOutputTmainOutputTag, + http://docs.oracle.com/javase/7/docs/api/java/util/List.html?is-external=true; title="class or interface in java.util">ListTupleTag?sideOutputTags, + ExecutionContext.StepContextstepContext, + Aggregator.AggregatorFactoryaggregatorFactory, + WindowingStrategy?,?windowingStrategy) +Deprecated. +please port uses of OldDoFn to use DoFn + + + + +static
[15/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/RepeatedlyStateMachine.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/RepeatedlyStateMachine.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/RepeatedlyStateMachine.html new file mode 100644 index 000..275841e --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/RepeatedlyStateMachine.html @@ -0,0 +1,440 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +RepeatedlyStateMachine (Apache Beam SDK for Java, version 0.4.0) + + + + + + +var methods = {"i0":9,"i1":10,"i2":10,"i3":10,"i4":10,"i5":10}; +var tabs = {65535:["t0","All Methods"],1:["t1","Static Methods"],2:["t2","Instance Methods"],8:["t4","Concrete Methods"]}; +var altColor = "altColor"; +var rowColor = "rowColor"; +var tableTab = "tableTab"; +var activeTableTab = "activeTableTab"; + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +PrevClass +NextClass + + +Frames +NoFrames + + +AllClasses + + + + + + + +Summary: +Nested| +Field| +Constr| +Method + + +Detail: +Field| +Constr| +Method + + + + + + + + +org.apache.beam.runners.core.triggers +Class RepeatedlyStateMachine + + + +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">java.lang.Object + + +org.apache.beam.runners.core.triggers.TriggerStateMachine + + +org.apache.beam.runners.core.triggers.RepeatedlyStateMachine + + + + + + + + + +All Implemented Interfaces: +http://docs.oracle.com/javase/7/docs/api/java/io/Serializable.html?is-external=true; title="class or interface in java.io">Serializable + + + +public class RepeatedlyStateMachine +extends TriggerStateMachine +Repeat a trigger, either until some condition is met or forever. + + For example, to fire after the end of the window, and every time late data arrives: + + Repeatedly.forever(AfterWatermark.isPastEndOfWindow()); + + + Repeatedly.forever(someTrigger) behaves like an infinite + AfterEach.inOrder(someTrigger, someTrigger, someTrigger, ...). + +See Also: +Serialized Form + + + + + + + + + + + + +Nested Class Summary + + + + +Nested classes/interfaces inherited from classorg.apache.beam.runners.core.triggers.TriggerStateMachine +TriggerStateMachine.MergingTriggerInfo, TriggerStateMachine.OnceTriggerStateMachine, TriggerStateMachine.OnElementContext, TriggerStateMachine.OnMergeContext, TriggerStateMachine.TriggerContext, TriggerStateMachine.TriggerInfo + + + + + + + + +Field Summary + + + + +Fields inherited from classorg.apache.beam.runners.core.triggers.TriggerStateMachine +subTriggers + + + + + + + + +Method Summary + +All MethodsStatic MethodsInstance MethodsConcrete Methods + +Modifier and Type +Method and Description + + +static RepeatedlyStateMachine +forever(TriggerStateMachinerepeated) +Create a composite trigger that repeatedly executes the trigger repeated, firing each + time it fires and ignoring any indications to finish. + + + +void +onElement(TriggerStateMachine.OnElementContextc) +Called every time an element is incorporated into a window. + + + +void +onFire(TriggerStateMachine.TriggerContextcontext) +Adjusts the state of the trigger to be ready for the next pane. + + + +void +onMerge(TriggerStateMachine.OnMergeContextc) +Called immediately after windows have been merged. + + + +boolean +shouldFire(TriggerStateMachine.TriggerContextcontext) +Returns true if the current state of the trigger indicates that its condition + is satisfied and it is ready to fire. + + + +http://docs.oracle.com/javase/7/docs/api/java/lang/String.html?is-external=true; title="class or interface in java.lang">String +toString() + + + + + + +Methods inherited from classorg.apache.beam.runners.core.triggers.TriggerStateMachine +clear, equals, hashCode, isCompatible, orFinally, prefetchOnElement, prefetchOnFire, prefetchOnMerge, prefetchShouldFire, subTriggers + + + + + +Methods inherited from classjava.lang.http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Object +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#clone--; title="class or interface in
[45/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/index.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/index.html b/content/documentation/sdks/javadoc/0.4.0/index.html new file mode 100644 index 000..8760b98 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/index.html @@ -0,0 +1,76 @@ +http://www.w3.org/TR/html4/frameset.dtd;> + + + + + +Apache Beam SDK for Java, version 0.4.0 + +tmpTargetPage = "" + window.location.search; +if (tmpTargetPage != "" && tmpTargetPage != "undefined") +tmpTargetPage = tmpTargetPage.substring(1); +if (tmpTargetPage.indexOf(":") != -1 || (tmpTargetPage != "" && !validURL(tmpTargetPage))) +tmpTargetPage = "undefined"; +targetPage = tmpTargetPage; +function validURL(url) { +try { +url = decodeURIComponent(url); +} +catch (error) { +return false; +} +var pos = url.indexOf(".html"); +if (pos == -1 || pos != url.length - 5) +return false; +var allowNumber = false; +var allowSep = false; +var seenDot = false; +for (var i = 0; i < url.length - 5; i++) { +var ch = url.charAt(i); +if ('a' <= ch && ch <= 'z' || +'A' <= ch && ch <= 'Z' || +ch == '$' || +ch == '_' || +ch.charCodeAt(0) > 127) { +allowNumber = true; +allowSep = true; +} else if ('0' <= ch && ch <= '9' +|| ch == '-') { +if (!allowNumber) + return false; +} else if (ch == '/' || ch == '.') { +if (!allowSep) +return false; +allowNumber = false; +allowSep = false; +if (ch == '.') + seenDot = true; +if (ch == '/' && seenDot) + return false; +} else { +return false; +} +} +return true; +} +function loadFrames() { +if (targetPage != "" && targetPage != "undefined") + top.classFrame.location = top.targetPage; +} + + + + + + + + + + +JavaScript is disabled on your browser. + +Frame Alert +This document is designed to be viewed using the frames feature. If you see this message, you are using a non-frame-capable web client. Link to Non-frame version. + + + http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/apex/ApexPipelineOptions.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/apex/ApexPipelineOptions.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/apex/ApexPipelineOptions.html new file mode 100644 index 000..d45f4a9 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/apex/ApexPipelineOptions.html @@ -0,0 +1,379 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +ApexPipelineOptions (Apache Beam SDK for Java, version 0.4.0) + + + + + + +var methods = {"i0":6,"i1":6,"i2":6,"i3":6,"i4":6,"i5":6,"i6":6,"i7":6,"i8":6,"i9":6}; +var tabs = {65535:["t0","All Methods"],2:["t2","Instance Methods"],4:["t3","Abstract Methods"]}; +var altColor = "altColor"; +var rowColor = "rowColor"; +var tableTab = "tableTab"; +var activeTableTab = "activeTableTab"; + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +PrevClass +NextClass + + +Frames +NoFrames + + +AllClasses + + + + + + + +Summary: +Nested| +Field| +Constr| +Method + + +Detail: +Field| +Constr| +Method + + + + + + + + +org.apache.beam.runners.apex +Interface ApexPipelineOptions + + + + + + +All Superinterfaces: +HasDisplayData, PipelineOptions, http://docs.oracle.com/javase/7/docs/api/java/io/Serializable.html?is-external=true; title="class or interface in java.io">Serializable + + + +public interface ApexPipelineOptions +extends PipelineOptions, http://docs.oracle.com/javase/7/docs/api/java/io/Serializable.html?is-external=true; title="class or interface in java.io">Serializable +Options that configure the Apex pipeline. + + + + + + + + + + + +Nested Class Summary + +
[32/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/ReduceFnRunner.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/ReduceFnRunner.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/ReduceFnRunner.html new file mode 100644 index 000..f96c50a --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/ReduceFnRunner.html @@ -0,0 +1,378 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +ReduceFnRunner (Apache Beam SDK for Java, version 0.4.0) + + + + + + +var methods = {"i0":10,"i1":10,"i2":10}; +var tabs = {65535:["t0","All Methods"],2:["t2","Instance Methods"],8:["t4","Concrete Methods"]}; +var altColor = "altColor"; +var rowColor = "rowColor"; +var tableTab = "tableTab"; +var activeTableTab = "activeTableTab"; + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +PrevClass +NextClass + + +Frames +NoFrames + + +AllClasses + + + + + + + +Summary: +Nested| +Field| +Constr| +Method + + +Detail: +Field| +Constr| +Method + + + + + + + + +org.apache.beam.runners.core +Class ReduceFnRunnerK,InputT,OutputT,W extends BoundedWindow + + + +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">java.lang.Object + + +org.apache.beam.runners.core.ReduceFnRunnerK,InputT,OutputT,W + + + + + + + +Type Parameters: +K - The type of key being processed. +InputT - The type of values associated with the key. +OutputT - The output type that will be produced for each key. +W - The type of windows this operates on. + + + +public class ReduceFnRunnerK,InputT,OutputT,W extends BoundedWindow +extends http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Object +Manages the execution of a ReduceFn after a GroupByKeyViaGroupByKeyOnly.GroupByKeyOnly has partitioned the + PCollection by key. + + The onTrigger(org.apache.beam.runners.core.ReduceFnK, InputT, OutputT, W.Context, org.apache.beam.runners.core.ReduceFnK, InputT, OutputT, W.Context, boolean, boolean) relies on a TriggerStateMachineRunner to manage the execution + of the triggering logic. The ReduceFnRunners responsibilities are: + + + Tracking the windows that are active (have buffered data) as elements arrive and triggers are + fired. + Holding the watermark based on the timestamps of elements in a pane and releasing it when the + trigger fires. + Calling the appropriate callbacks on ReduceFn based on trigger execution, timer + firings, etc, and providing appropriate contexts to the ReduceFn for actions such as + output. + Scheduling garbage collection of state associated with a specific window, and making that + happen when the appropriate timer fires. + + + + + + + + + + + + +Constructor Summary + +Constructors + +Constructor and Description + + +ReduceFnRunner(Kkey, + WindowingStrategy?,WwindowingStrategy, + ExecutableTriggerStateMachinetriggerStateMachine, + StateInternalsKstateInternals, + TimerInternalstimerInternals, + OutputWindowedValueKVK,OutputToutputter, + SideInputReadersideInputReader, + Aggregatorhttp://docs.oracle.com/javase/7/docs/api/java/lang/Long.html?is-external=true; title="class or interface in java.lang">Long,http://docs.oracle.com/javase/7/docs/api/java/lang/Long.html?is-external=true; title="class or interface in java.lang">LongdroppedDueToClosedWindow, + ReduceFnK,InputT,OutputT,WreduceFn, + PipelineOptionsoptions) + + + + + + + + + +Method Summary + +All MethodsInstance MethodsConcrete Methods + +Modifier and Type +Method and Description + + +void +onTimers(http://docs.oracle.com/javase/7/docs/api/java/lang/Iterable.html?is-external=true; title="class or interface in java.lang">IterableTimerInternals.TimerDatatimers) + + +void +persist() + + +void +processElements(http://docs.oracle.com/javase/7/docs/api/java/lang/Iterable.html?is-external=true; title="class or interface in java.lang">IterableWindowedValueInputTvalues) +Incorporate values into the underlying reduce function, and manage holds, timers, + triggers, and window merging. + + + + + + + +Methods inherited from
[07/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/class-use/TriggerStateMachine.OnceTriggerStateMachine.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/class-use/TriggerStateMachine.OnceTriggerStateMachine.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/class-use/TriggerStateMachine.OnceTriggerStateMachine.html new file mode 100644 index 000..b58c186 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/class-use/TriggerStateMachine.OnceTriggerStateMachine.html @@ -0,0 +1,297 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +Uses of Class org.apache.beam.runners.core.triggers.TriggerStateMachine.OnceTriggerStateMachine (Apache Beam SDK for Java, version 0.4.0) + + + + + + + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +Prev +Next + + +Frames +NoFrames + + +AllClasses + + + + + + + + + + +Uses of Classorg.apache.beam.runners.core.triggers.TriggerStateMachine.OnceTriggerStateMachine + + + + + +Packages that use TriggerStateMachine.OnceTriggerStateMachine + +Package +Description + + + +org.apache.beam.runners.core.triggers + +State machine implementations for triggers, called "triggers" because + they react to events. + + + + + + + + + + +Uses of TriggerStateMachine.OnceTriggerStateMachine in org.apache.beam.runners.core.triggers + +Subclasses of TriggerStateMachine.OnceTriggerStateMachine in org.apache.beam.runners.core.triggers + +Modifier and Type +Class and Description + + + +class +AfterAllStateMachine +A TriggerStateMachine that fires and finishes once after all of its sub-triggers + have fired. + + + +class +AfterDelayFromFirstElementStateMachine +A base class for triggers that happen after a processing time delay from the arrival + of the first element in a pane. + + + +class +AfterFirstStateMachine +Create a composite TriggerStateMachine that fires once after at least one of its + sub-triggers have fired. + + + +class +AfterPaneStateMachine +TriggerStateMachines that fire based on properties of the elements in the current pane. + + + +class +AfterProcessingTimeStateMachine +AfterProcessingTime triggers fire based on the current processing time. + + + +static class +AfterWatermarkStateMachine.FromEndOfWindow +A watermark trigger targeted relative to the end of the window. + + + +class +NeverStateMachine +A TriggerStateMachine which never fires. + + + + + +Methods in org.apache.beam.runners.core.triggers that return TriggerStateMachine.OnceTriggerStateMachine + +Modifier and Type +Method and Description + + + +static TriggerStateMachine.OnceTriggerStateMachine +AfterAllStateMachine.of(http://docs.oracle.com/javase/7/docs/api/java/lang/Iterable.html?is-external=true; title="class or interface in java.lang">Iterable? extends TriggerStateMachinetriggers) + + +static TriggerStateMachine.OnceTriggerStateMachine +AfterFirstStateMachine.of(http://docs.oracle.com/javase/7/docs/api/java/lang/Iterable.html?is-external=true; title="class or interface in java.lang">Iterable? extends TriggerStateMachinetriggers) + + +static TriggerStateMachine.OnceTriggerStateMachine +AfterAllStateMachine.of(TriggerStateMachine.OnceTriggerStateMachine...triggers) +Returns an AfterAll Trigger with the given subtriggers. + + + +static TriggerStateMachine.OnceTriggerStateMachine +AfterFirstStateMachine.of(TriggerStateMachine.OnceTriggerStateMachine...triggers) +Returns an AfterFirst Trigger with the given subtriggers. + + + +static TriggerStateMachine.OnceTriggerStateMachine +TriggerStateMachines.stateMachineForOnceTrigger(Trigger.OnceTriggertrigger) + + + + +Methods in org.apache.beam.runners.core.triggers with parameters of type TriggerStateMachine.OnceTriggerStateMachine + +Modifier and Type +Method and Description + + + +static W extends BoundedWindowExecutableTriggerStateMachine +ExecutableTriggerStateMachine.createForOnceTrigger(TriggerStateMachine.OnceTriggerStateMachinetrigger, +intnextUnusedIndex) + + +static TriggerStateMachine.OnceTriggerStateMachine +AfterAllStateMachine.of(TriggerStateMachine.OnceTriggerStateMachine...triggers) +Returns an AfterAll Trigger with
[05/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/package-tree.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/package-tree.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/package-tree.html new file mode 100644 index 000..23bbf96 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/package-tree.html @@ -0,0 +1,183 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +org.apache.beam.runners.core.triggers Class Hierarchy (Apache Beam SDK for Java, version 0.4.0) + + + + + + + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +Prev +Next + + +Frames +NoFrames + + +AllClasses + + + + + + + + + + +Hierarchy For Package org.apache.beam.runners.core.triggers +Package Hierarchies: + +All Packages + + + +Class Hierarchy + +java.lang.http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Object + +org.apache.beam.runners.core.triggers.AfterWatermarkStateMachine +org.apache.beam.runners.core.triggers.ExecutableTriggerStateMachine (implements java.io.http://docs.oracle.com/javase/7/docs/api/java/io/Serializable.html?is-external=true; title="class or interface in java.io">Serializable) +org.apache.beam.runners.core.triggers.FinishedTriggersBitSet (implements org.apache.beam.runners.core.triggers.FinishedTriggers) +org.apache.beam.runners.core.triggers.FinishedTriggersSet (implements org.apache.beam.runners.core.triggers.FinishedTriggers) +org.apache.beam.runners.core.triggers.TriggerStateMachine (implements java.io.http://docs.oracle.com/javase/7/docs/api/java/io/Serializable.html?is-external=true; title="class or interface in java.io">Serializable) + +org.apache.beam.runners.core.triggers.AfterEachStateMachine +org.apache.beam.runners.core.triggers.AfterWatermarkStateMachine.AfterWatermarkEarlyAndLate +org.apache.beam.runners.core.triggers.DefaultTriggerStateMachine +org.apache.beam.runners.core.triggers.RepeatedlyStateMachine +org.apache.beam.runners.core.triggers.ReshuffleTriggerStateMachine +org.apache.beam.runners.core.triggers.TriggerStateMachine.OnceTriggerStateMachine + +org.apache.beam.runners.core.triggers.AfterAllStateMachine +org.apache.beam.runners.core.triggers.AfterDelayFromFirstElementStateMachine + +org.apache.beam.runners.core.triggers.AfterProcessingTimeStateMachine + + +org.apache.beam.runners.core.triggers.AfterFirstStateMachine +org.apache.beam.runners.core.triggers.AfterPaneStateMachine +org.apache.beam.runners.core.triggers.AfterWatermarkStateMachine.FromEndOfWindow +org.apache.beam.runners.core.triggers.NeverStateMachine + + + + +org.apache.beam.runners.core.triggers.TriggerStateMachine.TriggerContext + +org.apache.beam.runners.core.triggers.TriggerStateMachine.OnElementContext +org.apache.beam.runners.core.triggers.TriggerStateMachine.OnMergeContext + + +org.apache.beam.runners.core.triggers.TriggerStateMachineContextFactoryW +org.apache.beam.runners.core.triggers.TriggerStateMachineRunnerW +org.apache.beam.runners.core.triggers.TriggerStateMachines + + + +Interface Hierarchy + +org.apache.beam.runners.core.triggers.FinishedTriggers +org.apache.beam.runners.core.triggers.TriggerStateMachine.TriggerInfo + +org.apache.beam.runners.core.triggers.TriggerStateMachine.MergingTriggerInfo + + + + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +Prev +Next + + +Frames +NoFrames + + +AllClasses + + + + + + + + + +Copyright 2016 https://www.apache.org/;>The Apache Software Foundation. All rights reserved. + + http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/package-use.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/package-use.html
[09/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/class-use/ExecutableTriggerStateMachine.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/class-use/ExecutableTriggerStateMachine.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/class-use/ExecutableTriggerStateMachine.html new file mode 100644 index 000..cd3ac04 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/class-use/ExecutableTriggerStateMachine.html @@ -0,0 +1,387 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +Uses of Class org.apache.beam.runners.core.triggers.ExecutableTriggerStateMachine (Apache Beam SDK for Java, version 0.4.0) + + + + + + + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +Prev +Next + + +Frames +NoFrames + + +AllClasses + + + + + + + + + + +Uses of Classorg.apache.beam.runners.core.triggers.ExecutableTriggerStateMachine + + + + + +Packages that use ExecutableTriggerStateMachine + +Package +Description + + + +org.apache.beam.runners.core + +Provides utilities for Beam runner authors. + + + +org.apache.beam.runners.core.triggers + +State machine implementations for triggers, called "triggers" because + they react to events. + + + + + + + + + + +Uses of ExecutableTriggerStateMachine in org.apache.beam.runners.core + +Constructors in org.apache.beam.runners.core with parameters of type ExecutableTriggerStateMachine + +Constructor and Description + + + +ReduceFnRunner(Kkey, + WindowingStrategy?,WwindowingStrategy, + ExecutableTriggerStateMachinetriggerStateMachine, + StateInternalsKstateInternals, + TimerInternalstimerInternals, + OutputWindowedValueKVK,OutputToutputter, + SideInputReadersideInputReader, + Aggregatorhttp://docs.oracle.com/javase/7/docs/api/java/lang/Long.html?is-external=true; title="class or interface in java.lang">Long,http://docs.oracle.com/javase/7/docs/api/java/lang/Long.html?is-external=true; title="class or interface in java.lang">LongdroppedDueToClosedWindow, + ReduceFnK,InputT,OutputT,WreduceFn, + PipelineOptionsoptions) + + + + + + + +Uses of ExecutableTriggerStateMachine in org.apache.beam.runners.core.triggers + +Methods in org.apache.beam.runners.core.triggers that return ExecutableTriggerStateMachine + +Modifier and Type +Method and Description + + + +static W extends BoundedWindowExecutableTriggerStateMachine +ExecutableTriggerStateMachine.create(TriggerStateMachinetrigger) + + +static W extends BoundedWindowExecutableTriggerStateMachine +ExecutableTriggerStateMachine.createForOnceTrigger(TriggerStateMachine.OnceTriggerStateMachinetrigger, +intnextUnusedIndex) + + +ExecutableTriggerStateMachine +TriggerStateMachine.TriggerInfo.firstUnfinishedSubTrigger() +Returns the first unfinished sub-trigger. + + + +ExecutableTriggerStateMachine +ExecutableTriggerStateMachine.getSubTriggerContaining(intindex) + + +ExecutableTriggerStateMachine +TriggerStateMachine.TriggerInfo.subTrigger(intsubtriggerIndex) +Access the executable version of the specified sub-trigger. + + + + + +Methods in org.apache.beam.runners.core.triggers that return types with arguments of type ExecutableTriggerStateMachine + +Modifier and Type +Method and Description + + + +http://docs.oracle.com/javase/7/docs/api/java/util/Set.html?is-external=true; title="class or interface in java.util">SetExecutableTriggerStateMachine +FinishedTriggersSet.getFinishedTriggers() +Returns a mutable http://docs.oracle.com/javase/7/docs/api/java/util/Set.html?is-external=true; title="class or interface in java.util">Set of the underlying triggers that are finished. + + + +http://docs.oracle.com/javase/7/docs/api/java/lang/Iterable.html?is-external=true; title="class or interface in java.lang">IterableExecutableTriggerStateMachine +TriggerStateMachine.TriggerInfo.subTriggers() +Access the executable versions of the sub-triggers of the current trigger. + + + +http://docs.oracle.com/javase/7/docs/api/java/util/List.html?is-external=true; title="class or interface in
[04/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/dataflow/DataflowJobAlreadyUpdatedException.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/dataflow/DataflowJobAlreadyUpdatedException.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/dataflow/DataflowJobAlreadyUpdatedException.html new file mode 100644 index 000..1daa241 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/dataflow/DataflowJobAlreadyUpdatedException.html @@ -0,0 +1,289 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +DataflowJobAlreadyUpdatedException (Apache Beam SDK for Java, version 0.4.0) + + + + + + + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +PrevClass +NextClass + + +Frames +NoFrames + + +AllClasses + + + + + + + +Summary: +Nested| +Field| +Constr| +Method + + +Detail: +Field| +Constr| +Method + + + + + + + + +org.apache.beam.runners.dataflow +Class DataflowJobAlreadyUpdatedException + + + +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">java.lang.Object + + +http://docs.oracle.com/javase/7/docs/api/java/lang/Throwable.html?is-external=true; title="class or interface in java.lang">java.lang.Throwable + + +http://docs.oracle.com/javase/7/docs/api/java/lang/Exception.html?is-external=true; title="class or interface in java.lang">java.lang.Exception + + +http://docs.oracle.com/javase/7/docs/api/java/lang/RuntimeException.html?is-external=true; title="class or interface in java.lang">java.lang.RuntimeException + + +org.apache.beam.runners.dataflow.DataflowJobException + + +org.apache.beam.runners.dataflow.DataflowJobAlreadyUpdatedException + + + + + + + + + + + + + + + +All Implemented Interfaces: +http://docs.oracle.com/javase/7/docs/api/java/io/Serializable.html?is-external=true; title="class or interface in java.io">Serializable + + + +public class DataflowJobAlreadyUpdatedException +extends DataflowJobException +An exception that is thrown if the existing job has already been updated within the Dataflow + service and is no longer able to be updated. The DataflowPipelineJob contained within + this exception contains information about the pre-existing updated job. + +See Also: +Serialized Form + + + + + + + + + + + + +Constructor Summary + +Constructors + +Constructor and Description + + +DataflowJobAlreadyUpdatedException(DataflowPipelineJobjob, + http://docs.oracle.com/javase/7/docs/api/java/lang/String.html?is-external=true; title="class or interface in java.lang">Stringmessage) +Create a new DataflowJobAlreadyUpdatedException with the specified DataflowPipelineJob and message. + + + + + + + + + + +Method Summary + + + + +Methods inherited from classorg.apache.beam.runners.dataflow.DataflowJobException +getJob + + + + + +Methods inherited from classjava.lang.http://docs.oracle.com/javase/7/docs/api/java/lang/Throwable.html?is-external=true; title="class or interface in java.lang">Throwable +http://docs.oracle.com/javase/7/docs/api/java/lang/Throwable.html?is-external=true#addSuppressed-java.lang.Throwable-; title="class or interface in java.lang">addSuppressed, http://docs.oracle.com/javase/7/docs/api/java/lang/Throwable.html?is-external=true#fillInStackTrace--; title="class or interface in java.lang">fillInStackTrace, http://docs.oracle.com/javase/7/docs/api/java/lang/Throwable.html?is-external=true#getCause--; title="class or interface in java.lang">getCause, http://docs.oracle.com/javase/7/docs/api/java/lang/Throwable.html?is-external=true#getLocalizedMessage--; title="class or interface in java.lang">getLocalizedMessage, http://docs.oracle.com/javase/7/docs/api/java/lang/Throwable.html?is-external=true#getMessage--; title="class or interface in java.lang">getMessage, http://docs.oracle.com/javase/7/docs/api/java/lang/Throwable.html?is-external=true#getStackTrace--; title="class or inter face in java.lang">getStackTrace, http://docs.oracle.com/javase/7/docs/api/java/lang/Throwable.html?is-external=true#getSuppressed--; title="class or interface in java.lang">getSuppressed, http://docs.oracle.com/javase/7/docs/api/java/lang/Throwable.html?is-external=true#initCause-java.lang.Throwable-;
[22/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/package-tree.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/package-tree.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/package-tree.html new file mode 100644 index 000..9676c45 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/package-tree.html @@ -0,0 +1,229 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +org.apache.beam.runners.core Class Hierarchy (Apache Beam SDK for Java, version 0.4.0) + + + + + + + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +Prev +Next + + +Frames +NoFrames + + +AllClasses + + + + + + + + + + +Hierarchy For Package org.apache.beam.runners.core +Package Hierarchies: + +All Packages + + + +Class Hierarchy + +java.lang.http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Object + +org.apache.beam.sdk.transforms.DoFnInputT,OutputT (implements org.apache.beam.sdk.transforms.display.HasDisplayData, java.io.http://docs.oracle.com/javase/7/docs/api/java/io/Serializable.html?is-external=true; title="class or interface in java.io">Serializable) + +org.apache.beam.runners.core.SplittableParDo.ProcessFnInputT,OutputT,RestrictionT,TrackerT + + +org.apache.beam.runners.core.DoFnRunners +org.apache.beam.runners.core.ElementAndRestrictionElementT,RestrictionT +org.apache.beam.runners.core.KeyedWorkItems +org.apache.beam.runners.core.KeyedWorkItems.ComposedKeyedWorkItemK,ElemT (implements org.apache.beam.runners.core.KeyedWorkItemK,ElemT) +org.apache.beam.runners.core.LateDataDroppingDoFnRunnerK,InputT,OutputT,W (implements org.apache.beam.runners.core.DoFnRunnerInputT,OutputT) +org.apache.beam.runners.core.NonEmptyPanesK,W +org.apache.beam.sdk.transforms.OldDoFnInputT,OutputT (implements org.apache.beam.sdk.transforms.display.HasDisplayData, java.io.http://docs.oracle.com/javase/7/docs/api/java/io/Serializable.html?is-external=true; title="class or interface in java.io">Serializable) + +org.apache.beam.runners.core.AssignWindowsDoFnT,W (implements org.apache.beam.sdk.transforms.OldDoFn.RequiresWindowAccess) +org.apache.beam.runners.core.GroupAlsoByWindowsDoFnK,InputT,OutputT,W + +org.apache.beam.runners.core.GroupAlsoByWindowsViaOutputBufferDoFnK,InputT,OutputT,W + + +org.apache.beam.runners.core.GroupAlsoByWindowViaWindowSetDoFnK,InputT,OutputT,W,RinT (implements org.apache.beam.runners.core.DoFnRunner.ReduceFnExecutorK,InputT,OutputT,W) + + +org.apache.beam.runners.core.PaneInfoTracker +org.apache.beam.runners.core.PeekingReiteratorT (implements org.apache.beam.sdk.util.common.ReiteratorT) +org.apache.beam.runners.core.PerKeyCombineFnRunners +org.apache.beam.sdk.transforms.PTransformInputT,OutputT (implements org.apache.beam.sdk.transforms.display.HasDisplayData, java.io.http://docs.oracle.com/javase/7/docs/api/java/io/Serializable.html?is-external=true; title="class or interface in java.io">Serializable) + +org.apache.beam.runners.core.AssignWindowsT,W +org.apache.beam.runners.core.GroupByKeyViaGroupByKeyOnlyK,V +org.apache.beam.runners.core.GroupByKeyViaGroupByKeyOnly.GroupAlsoByWindowK,V +org.apache.beam.runners.core.GroupByKeyViaGroupByKeyOnly.GroupByKeyOnlyK,V +org.apache.beam.runners.core.SplittableParDoInputT,OutputT,RestrictionT +org.apache.beam.runners.core.SplittableParDo.GBKIntoKeyedWorkItemsKeyT,InputT +org.apache.beam.runners.core.SplittableParDo.ProcessElementsInputT,OutputT,RestrictionT +org.apache.beam.runners.core.UnboundedReadFromBoundedSourceT + + +org.apache.beam.runners.core.PushbackSideInputDoFnRunnerInputT,OutputT (implements org.apache.beam.runners.core.DoFnRunnerInputT,OutputT) +org.apache.beam.runners.core.ReduceFnK,InputT,OutputT,W (implements java.io.http://docs.oracle.com/javase/7/docs/api/java/io/Serializable.html?is-external=true; title="class or interface in java.io">Serializable) + +org.apache.beam.runners.core.SystemReduceFnK,InputT,AccumT,OutputT,W + + +org.apache.beam.runners.core.ReduceFn.Context + +org.apache.beam.runners.core.ReduceFn.OnMergeContext +org.apache.beam.runners.core.ReduceFn.OnTriggerContext +org.apache.beam.runners.core.ReduceFn.ProcessValueContext + +
[25/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/class-use/KeyedWorkItemCoder.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/class-use/KeyedWorkItemCoder.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/class-use/KeyedWorkItemCoder.html new file mode 100644 index 000..ee4e4e2 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/class-use/KeyedWorkItemCoder.html @@ -0,0 +1,177 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +Uses of Class org.apache.beam.runners.core.KeyedWorkItemCoder (Apache Beam SDK for Java, version 0.4.0) + + + + + + + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +Prev +Next + + +Frames +NoFrames + + +AllClasses + + + + + + + + + + +Uses of Classorg.apache.beam.runners.core.KeyedWorkItemCoder + + + + + +Packages that use KeyedWorkItemCoder + +Package +Description + + + +org.apache.beam.runners.core + +Provides utilities for Beam runner authors. + + + + + + + + + + +Uses of KeyedWorkItemCoder in org.apache.beam.runners.core + +Methods in org.apache.beam.runners.core that return KeyedWorkItemCoder + +Modifier and Type +Method and Description + + + +static K,ElemTKeyedWorkItemCoderK,ElemT +KeyedWorkItemCoder.of(CoderKkeyCoder, + CoderElemTelemCoder, + Coder? extends BoundedWindowwindowCoder) +Create a new KeyedWorkItemCoder with the provided key coder, element coder, and window + coder. + + + +static K,ElemTKeyedWorkItemCoderK,ElemT +KeyedWorkItemCoder.of(http://docs.oracle.com/javase/7/docs/api/java/util/List.html?is-external=true; title="class or interface in java.util">ListCoder?components) + + + + + + + + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +Prev +Next + + +Frames +NoFrames + + +AllClasses + + + + + + + + + +Copyright 2016 https://www.apache.org/;>The Apache Software Foundation. All rights reserved. + + http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/class-use/KeyedWorkItems.ComposedKeyedWorkItem.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/class-use/KeyedWorkItems.ComposedKeyedWorkItem.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/class-use/KeyedWorkItems.ComposedKeyedWorkItem.html new file mode 100644 index 000..bd98ece --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/class-use/KeyedWorkItems.ComposedKeyedWorkItem.html @@ -0,0 +1,126 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +Uses of Class org.apache.beam.runners.core.KeyedWorkItems.ComposedKeyedWorkItem (Apache Beam SDK for Java, version 0.4.0) + + + + + + + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +Prev +Next + + +Frames +NoFrames + + +AllClasses + + + + + + + + + + +Uses of Classorg.apache.beam.runners.core.KeyedWorkItems.ComposedKeyedWorkItem + +No usage of org.apache.beam.runners.core.KeyedWorkItems.ComposedKeyedWorkItem + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +Prev +Next + + +Frames +NoFrames + + +AllClasses + + + +var methods = {"i0":10,"i1":10,"i2":9,"i3":9,"i4":10,"i5":10,"i6":10}; +var tabs = {65535:["t0","All Methods"],1:["t1","Static Methods"],2:["t2","Instance Methods"],8:["t4","Concrete Methods"]}; +var altColor = "altColor"; +var rowColor = "rowColor"; +var tableTab = "tableTab"; +var activeTableTab = "activeTableTab"; + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +PrevClass +NextClass + + +Frames +NoFrames + + +AllClasses + + + + + + + +Summary: +Nested| +Field| +Constr| +Method + + +Detail: +Field| +Constr| +Method + + + + + + + + +org.apache.beam.runners.core.triggers +Class FinishedTriggersBitSet + + + +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">java.lang.Object + + +org.apache.beam.runners.core.triggers.FinishedTriggersBitSet + + + + + + + +All Implemented Interfaces: +FinishedTriggers + + + +public class FinishedTriggersBitSet +extends http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Object +implements FinishedTriggers +A FinishedTriggers implementation based on an underlying http://docs.oracle.com/javase/7/docs/api/java/util/BitSet.html?is-external=true; title="class or interface in java.util">BitSet. + + + + + + + + + + + +Method Summary + +All MethodsStatic MethodsInstance MethodsConcrete Methods + +Modifier and Type +Method and Description + + +void +clearRecursively(ExecutableTriggerStateMachinetrigger) +Sets the trigger and all of its subtriggers to unfinished. + + + +FinishedTriggersBitSet +copy() +Create an independent copy of this mutable FinishedTriggers. + + + +static FinishedTriggersBitSet +emptyWithCapacity(intcapacity) + + +static FinishedTriggersBitSet +fromBitSet(http://docs.oracle.com/javase/7/docs/api/java/util/BitSet.html?is-external=true; title="class or interface in java.util">BitSetbitSet) + + +http://docs.oracle.com/javase/7/docs/api/java/util/BitSet.html?is-external=true; title="class or interface in java.util">BitSet +getBitSet() +Returns the underlying http://docs.oracle.com/javase/7/docs/api/java/util/BitSet.html?is-external=true; title="class or interface in java.util">BitSet for this FinishedTriggersBitSet. + + + +boolean +isFinished(ExecutableTriggerStateMachinetrigger) +Returns true if the trigger is finished. + + + +void +setFinished(ExecutableTriggerStateMachinetrigger, + booleanvalue) +Sets the fact that the trigger is finished. + + + + + + + +Methods inherited from classjava.lang.http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Object +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#clone--; title="class or interface in java.lang">clone, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#equals-java.lang.Object-; title="class or interface in java.lang">equals, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#finalize--; title="class or interface in java.lang">finalize, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#getClass--; title="class or interface in java.lang">getClass, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#hashCode--; title="class or interface in java.lang">hashCode, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#notify--; title="class or interface in java.lang">notify, http://docs.oracle.com/javase/7/docs/api/java/lang /Object.html?is-external=true#notifyAll--" title="class or interface in
[36/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/KeyedWorkItemCoder.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/KeyedWorkItemCoder.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/KeyedWorkItemCoder.html new file mode 100644 index 000..46210fb --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/KeyedWorkItemCoder.html @@ -0,0 +1,473 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +KeyedWorkItemCoder (Apache Beam SDK for Java, version 0.4.0) + + + + + + +var methods = {"i0":10,"i1":10,"i2":10,"i3":10,"i4":10,"i5":10,"i6":9,"i7":9,"i8":10}; +var tabs = {65535:["t0","All Methods"],1:["t1","Static Methods"],2:["t2","Instance Methods"],8:["t4","Concrete Methods"]}; +var altColor = "altColor"; +var rowColor = "rowColor"; +var tableTab = "tableTab"; +var activeTableTab = "activeTableTab"; + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +PrevClass +NextClass + + +Frames +NoFrames + + +AllClasses + + + + + + + +Summary: +Nested| +Field| +Constr| +Method + + +Detail: +Field| +Constr| +Method + + + + + + + + +org.apache.beam.runners.core +Class KeyedWorkItemCoderK,ElemT + + + +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">java.lang.Object + + +org.apache.beam.sdk.coders.StandardCoderKeyedWorkItemK,ElemT + + +org.apache.beam.runners.core.KeyedWorkItemCoderK,ElemT + + + + + + + + + +All Implemented Interfaces: +http://docs.oracle.com/javase/7/docs/api/java/io/Serializable.html?is-external=true; title="class or interface in java.io">Serializable, CoderKeyedWorkItemK,ElemT + + + +public class KeyedWorkItemCoderK,ElemT +extends StandardCoderKeyedWorkItemK,ElemT +A Coder for KeyedWorkItems. + +See Also: +Serialized Form + + + + + + + + + + + + +Nested Class Summary + + + + +Nested classes/interfaces inherited from interfaceorg.apache.beam.sdk.coders.Coder +Coder.Context, Coder.NonDeterministicException + + + + + + + + +Method Summary + +All MethodsStatic MethodsInstance MethodsConcrete Methods + +Modifier and Type +Method and Description + + +boolean +consistentWithEquals() +Returns true if this Coder is injective with respect to http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#equals-java.lang.Object-; title="class or interface in java.lang">Object.equals(java.lang.Object). + + + +KeyedWorkItemK,ElemT +decode(http://docs.oracle.com/javase/7/docs/api/java/io/InputStream.html?is-external=true; title="class or interface in java.io">InputStreaminStream, + Coder.Contextcontext) +Decodes a value of type T from the given input stream in + the given context. + + + +void +encode(KeyedWorkItemK,ElemTvalue, + http://docs.oracle.com/javase/7/docs/api/java/io/OutputStream.html?is-external=true; title="class or interface in java.io">OutputStreamoutStream, + Coder.Contextcontext) +Encodes the given value of type T onto the given output stream + in the given context. + + + +http://docs.oracle.com/javase/7/docs/api/java/util/List.html?is-external=true; title="class or interface in java.util">List? extends Coder? +getCoderArguments() +If this is a Coder for a parameterized type, returns the + list of Coders being used for each of the parameters, or + returns null if this cannot be done or this is not a + parameterized type. + + + +CoderElemT +getElementCoder() + + +CoderK +getKeyCoder() + + +static K,ElemTKeyedWorkItemCoderK,ElemT +of(CoderKkeyCoder, + CoderElemTelemCoder, + Coder? extends BoundedWindowwindowCoder) +Create a new KeyedWorkItemCoder with the provided key coder, element coder, and window + coder. + + + +static K,ElemTKeyedWorkItemCoderK,ElemT +of(http://docs.oracle.com/javase/7/docs/api/java/util/List.html?is-external=true; title="class or interface in java.util">ListCoder?components) + + +void +verifyDeterministic() +Throw Coder.NonDeterministicException if the coding is not deterministic. + + + + + + + +Methods inherited from classorg.apache.beam.sdk.coders.StandardCoder +asCloudObject, equals, getAllowedEncodings, getComponents, getEncodedElementByteSize, getEncodingId, hashCode, isRegisterByteSizeObserverCheap< /a>, registerByteSizeObserver, structuralValue,
[24/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/class-use/ReduceFn.OnMergeContext.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/class-use/ReduceFn.OnMergeContext.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/class-use/ReduceFn.OnMergeContext.html new file mode 100644 index 000..ecf0758 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/class-use/ReduceFn.OnMergeContext.html @@ -0,0 +1,170 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +Uses of Class org.apache.beam.runners.core.ReduceFn.OnMergeContext (Apache Beam SDK for Java, version 0.4.0) + + + + + + + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +Prev +Next + + +Frames +NoFrames + + +AllClasses + + + + + + + + + + +Uses of Classorg.apache.beam.runners.core.ReduceFn.OnMergeContext + + + + + +Packages that use ReduceFn.OnMergeContext + +Package +Description + + + +org.apache.beam.runners.core + +Provides utilities for Beam runner authors. + + + + + + + + + + +Uses of ReduceFn.OnMergeContext in org.apache.beam.runners.core + +Methods in org.apache.beam.runners.core with parameters of type ReduceFn.OnMergeContext + +Modifier and Type +Method and Description + + + +abstract void +ReduceFn.onMerge(ReduceFn.OnMergeContextcontext) +Called when windows are merged. + + + + + + + + + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +Prev +Next + + +Frames +NoFrames + + +AllClasses + + + + + + + + + +Copyright 2016 https://www.apache.org/;>The Apache Software Foundation. All rights reserved. + + http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/class-use/ReduceFn.OnTriggerContext.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/class-use/ReduceFn.OnTriggerContext.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/class-use/ReduceFn.OnTriggerContext.html new file mode 100644 index 000..fbeabb6 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/class-use/ReduceFn.OnTriggerContext.html @@ -0,0 +1,174 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +Uses of Class org.apache.beam.runners.core.ReduceFn.OnTriggerContext (Apache Beam SDK for Java, version 0.4.0) + + + + + + + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +Prev +Next + + +Frames +NoFrames + + +AllClasses + + + + + + + + + + +Uses of Classorg.apache.beam.runners.core.ReduceFn.OnTriggerContext + + + + + +Packages that use ReduceFn.OnTriggerContext + +Package +Description + + + +org.apache.beam.runners.core + +Provides utilities for Beam runner authors. + + + + + + + + + + +Uses of ReduceFn.OnTriggerContext in org.apache.beam.runners.core + +Methods in org.apache.beam.runners.core with parameters of type ReduceFn.OnTriggerContext + +Modifier and Type +Method and Description + + + +abstract void +ReduceFn.onTrigger(ReduceFn.OnTriggerContextcontext) +Called when triggers fire. + + + +void +SystemReduceFn.onTrigger(ReduceFn.OnTriggerContextc) + + + + + + + + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +Prev +Next + + +Frames +NoFrames + +
[48/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/constant-values.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/constant-values.html b/content/documentation/sdks/javadoc/0.4.0/constant-values.html new file mode 100644 index 000..ed8d527 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/constant-values.html @@ -0,0 +1,1318 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +Constant Field Values (Apache Beam SDK for Java, version 0.4.0) + + + + + + + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +Prev +Next + + +Frames +NoFrames + + +AllClasses + + + + + + + + + + +Constant Field Values +Contents + +org.apache.* + + + + + +org.apache.* + + + +org.apache.beam.runners.core.GroupAlsoByWindowsDoFnK,InputT,OutputT,W extends BoundedWindow + +Modifier and Type +Constant Field +Value + + + + + +publicstaticfinalhttp://docs.oracle.com/javase/7/docs/api/java/lang/String.html?is-external=true; title="class or interface in java.lang">String +DROPPED_DUE_TO_CLOSED_WINDOW_COUNTER +"DroppedDueToClosedWindow" + + + + +publicstaticfinalhttp://docs.oracle.com/javase/7/docs/api/java/lang/String.html?is-external=true; title="class or interface in java.lang">String +DROPPED_DUE_TO_LATENESS_COUNTER +"DroppedDueToLateness" + + + + + + + + +org.apache.beam.runners.dataflow.DataflowRunner + +Modifier and Type +Constant Field +Value + + + + + +publicstaticfinalhttp://docs.oracle.com/javase/7/docs/api/java/lang/String.html?is-external=true; title="class or interface in java.lang">String +PROJECT_ID_REGEXP +"[a-z][-a-z0-9:.]+[a-z0-9]" + + + + + + + + +org.apache.beam.runners.dataflow.util.OutputReference + +Modifier and Type +Constant Field +Value + + + + + +publicfinalhttp://docs.oracle.com/javase/7/docs/api/java/lang/String.html?is-external=true; title="class or interface in java.lang">String +type +"OutputReference" + + + + + + +org.apache.beam.runners.dataflow.util.PackageUtil + +Modifier and Type +Constant Field +Value + + + + + +publicstaticfinalint +SANE_CLASSPATH_SIZE +1000 + + + + + + + + +org.apache.beam.runners.spark.io.hadoop.ShardNameTemplateHelper + +Modifier and Type +Constant Field +Value + + + + + +publicstaticfinalhttp://docs.oracle.com/javase/7/docs/api/java/lang/String.html?is-external=true; title="class or interface in java.lang">String +OUTPUT_FILE_PREFIX +"spark.beam.fileoutputformat.prefix" + + + + +publicstaticfinalhttp://docs.oracle.com/javase/7/docs/api/java/lang/String.html?is-external=true; title="class or interface in java.lang">String +OUTPUT_FILE_SUFFIX +"spark.beam.fileoutputformat.suffix" + + + + +publicstaticfinalhttp://docs.oracle.com/javase/7/docs/api/java/lang/String.html?is-external=true; title="class or interface in java.lang">String +OUTPUT_FILE_TEMPLATE +"spark.beam.fileoutputformat.template" + + + + + + + + +org.apache.beam.runners.spark.util.BroadcastHelperT + +Modifier and Type +Constant Field +Value + + + + + +publicstaticfinalhttp://docs.oracle.com/javase/7/docs/api/java/lang/String.html?is-external=true; title="class or interface in java.lang">String +DIRECT_BROADCAST +"beam.spark.directBroadcast" + + + + + + + + +org.apache.beam.sdk.io.BoundedSource.BoundedReaderT + +Modifier and Type +Constant Field +Value + + + + + +publicstaticfinallong +SPLIT_POINTS_UNKNOWN +-1L + + + + + + +org.apache.beam.sdk.io.ShardNameTemplate + +Modifier and Type +Constant Field +Value + + + + + +publicstaticfinalhttp://docs.oracle.com/javase/7/docs/api/java/lang/String.html?is-external=true; title="class or interface in java.lang">String +DIRECTORY_CONTAINER +"/part-S" + + + + +publicstaticfinalhttp://docs.oracle.com/javase/7/docs/api/java/lang/String.html?is-external=true; title="class or interface in java.lang">String +INDEX_OF_MAX +"-S-of-N" + + + + + + +org.apache.beam.sdk.io.UnboundedSource.UnboundedReaderOutputT + +Modifier and Type +Constant Field +Value + + + + + +publicstaticfinallong +BACKLOG_UNKNOWN +-1L + + + + + + +org.apache.beam.sdk.io.XmlSink + +Modifier and Type +Constant Field +Value + + + + + +protectedstaticfinalhttp://docs.oracle.com/javase/7/docs/api/java/lang/String.html?is-external=true; title="class or interface in java.lang">String +XML_EXTENSION +"xml" + + + + + + + + +org.apache.beam.sdk.io.gcp.datastore.DatastoreV1.Read + +Modifier and Type +Constant Field
[18/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/AfterWatermarkStateMachine.AfterWatermarkEarlyAndLate.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/AfterWatermarkStateMachine.AfterWatermarkEarlyAndLate.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/AfterWatermarkStateMachine.AfterWatermarkEarlyAndLate.html new file mode 100644 index 000..88e6f58 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/triggers/AfterWatermarkStateMachine.AfterWatermarkEarlyAndLate.html @@ -0,0 +1,437 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +AfterWatermarkStateMachine.AfterWatermarkEarlyAndLate (Apache Beam SDK for Java, version 0.4.0) + + + + + + +var methods = {"i0":10,"i1":10,"i2":10,"i3":10,"i4":10,"i5":10,"i6":10}; +var tabs = {65535:["t0","All Methods"],2:["t2","Instance Methods"],8:["t4","Concrete Methods"]}; +var altColor = "altColor"; +var rowColor = "rowColor"; +var tableTab = "tableTab"; +var activeTableTab = "activeTableTab"; + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +PrevClass +NextClass + + +Frames +NoFrames + + +AllClasses + + + + + + + +Summary: +Nested| +Field| +Constr| +Method + + +Detail: +Field| +Constr| +Method + + + + + + + + +org.apache.beam.runners.core.triggers +Class AfterWatermarkStateMachine.AfterWatermarkEarlyAndLate + + + +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">java.lang.Object + + +org.apache.beam.runners.core.triggers.TriggerStateMachine + + +org.apache.beam.runners.core.triggers.AfterWatermarkStateMachine.AfterWatermarkEarlyAndLate + + + + + + + + + +All Implemented Interfaces: +http://docs.oracle.com/javase/7/docs/api/java/io/Serializable.html?is-external=true; title="class or interface in java.io">Serializable + + +Enclosing class: +AfterWatermarkStateMachine + + + +public static class AfterWatermarkStateMachine.AfterWatermarkEarlyAndLate +extends TriggerStateMachine + +See Also: +AfterWatermarkStateMachine, +Serialized Form + + + + + + + + + + + + +Nested Class Summary + + + + +Nested classes/interfaces inherited from classorg.apache.beam.runners.core.triggers.TriggerStateMachine +TriggerStateMachine.MergingTriggerInfo, TriggerStateMachine.OnceTriggerStateMachine, TriggerStateMachine.OnElementContext, TriggerStateMachine.OnMergeContext, TriggerStateMachine.TriggerContext, TriggerStateMachine.TriggerInfo + + + + + + + + +Field Summary + + + + +Fields inherited from classorg.apache.beam.runners.core.triggers.TriggerStateMachine +subTriggers + + + + + + + + +Method Summary + +All MethodsInstance MethodsConcrete Methods + +Modifier and Type +Method and Description + + +void +onElement(TriggerStateMachine.OnElementContextc) +Called every time an element is incorporated into a window. + + + +void +onFire(TriggerStateMachine.TriggerContextcontext) +Adjusts the state of the trigger to be ready for the next pane. + + + +void +onMerge(TriggerStateMachine.OnMergeContextc) +Called immediately after windows have been merged. + + + +boolean +shouldFire(TriggerStateMachine.TriggerContextcontext) +Returns true if the current state of the trigger indicates that its condition + is satisfied and it is ready to fire. + + + +http://docs.oracle.com/javase/7/docs/api/java/lang/String.html?is-external=true; title="class or interface in java.lang">String +toString() + + +AfterWatermarkStateMachine.AfterWatermarkEarlyAndLate +withEarlyFirings(TriggerStateMachine.OnceTriggerStateMachineearlyTrigger) + + +AfterWatermarkStateMachine.AfterWatermarkEarlyAndLate +withLateFirings(TriggerStateMachine.OnceTriggerStateMachinelateTrigger) + + + + + + +Methods inherited from classorg.apache.beam.runners.core.triggers.TriggerStateMachine +clear, equals, hashCode, isCompatible, orFinally, prefetchOnElement, prefetchOnFire, prefetchOnMerge, prefetchShouldFire, subTriggers + + + + + +Methods inherited from classjava.lang.http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Object +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#clone--;
[49/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/allclasses-noframe.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/allclasses-noframe.html b/content/documentation/sdks/javadoc/0.4.0/allclasses-noframe.html new file mode 100644 index 000..a2e6008 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/allclasses-noframe.html @@ -0,0 +1,965 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +All Classes (Apache Beam SDK for Java, version 0.4.0) + + + + + +AllClasses + + +AccumulatorCombiningState +ActiveWindowSet +ActiveWindowSet.MergeCallback +AfterAll +AfterAllStateMachine +AfterDelayFromFirstElement +AfterDelayFromFirstElementStateMachine +AfterEach +AfterEachStateMachine +AfterFirst +AfterFirstStateMachine +AfterPane +AfterPaneStateMachine +AfterProcessingTime +AfterProcessingTimeStateMachine +AfterSynchronizedProcessingTime +AfterWatermark +AfterWatermark.AfterWatermarkEarlyAndLate +AfterWatermark.FromEndOfWindow +AfterWatermarkStateMachine +AfterWatermarkStateMachine.AfterWatermarkEarlyAndLate +AfterWatermarkStateMachine.FromEndOfWindow +AggAccumParam +Aggregator +Aggregator.AggregatorFactory +AggregatorContainer +AggregatorContainer.AggregatorKey +AggregatorContainer.Mutator +AggregatorFactory +AggregatorMetric +AggregatorMetricSource +AggregatorRetrievalException +AggregatorRetriever +AggregatorValues +ApexPipelineOptions +ApexRunner +ApexRunner.CreateApexPCollectionView +ApexRunnerRegistrar +ApexRunnerRegistrar.Options +ApexRunnerRegistrar.Runner +ApexRunnerResult +ApexYarnLauncher +ApexYarnLauncher.LaunchParams +ApexYarnLauncher.ProcessWatcher +ApiSurface +AppEngineEnvironment +ApplicationNameOptions +AppliedCombineFn +AppliedPTransform +ApproximateQuantiles +ApproximateQuantiles.ApproximateQuantilesCombineFn +ApproximateUnique +ApproximateUnique.ApproximateUniqueCombineFn +ApproximateUnique.ApproximateUniqueCombineFn.LargestUnique +AssignWindows +AssignWindowsDoFn +AtomicCoder +AttemptAndTimeBoundedExponentialBackOff +AttemptAndTimeBoundedExponentialBackOff.ResetPolicy +AttemptBoundedExponentialBackOff +AvroCoder +AvroHDFSFileSource +AvroIO +AvroIO.Read +AvroIO.Read.Bound +AvroIO.Write +AvroIO.Write.Bound +AvroSource +AvroSource.AvroReader +AvroUtils +AvroUtils.AvroMetadata +AvroWrapperCoder +BagState +BaseExecutionContext +BaseExecutionContext.CreateStepContextFunction +BaseExecutionContext.StepContext +BeamSparkRunnerRegistrator +BigDecimalCoder +BigEndianIntegerCoder +BigEndianLongCoder +BigIntegerCoder +BigQueryIO +BigQueryIO.Read +BigQueryIO.Read.Bound +BigQueryIO.Write +BigQueryIO.Write.Bound +BigQueryIO.Write.CreateDisposition +BigQueryIO.Write.WriteDisposition +BigqueryMatcher +BigQueryOptions +BigtableIO +BigtableIO.Read +BigtableIO.Write +BigtableTestOptions +BitSetCoder +BlockBasedSource +BlockBasedSource.Block +BlockBasedSource.BlockBasedReader +BoundedReadFromUnboundedSource +BoundedSource +BoundedSource.BoundedReader +BoundedWindow +BroadcastHelper +BucketingFunction +BufferedElementCountingOutputStream +BufferedExternalSorter +BufferedExternalSorter.Options +BundleFactory +ByteArray +ByteArrayCoder +ByteBuddyDoFnInvokerFactory +ByteBuddyDoFnInvokerFactory.DefaultRestrictionCoder +ByteBuddyDoFnInvokerFactory.DefaultSplitRestriction +ByteBuddyDoFnInvokerFactory.DoFnInvokerBase +ByteCoder +ByteKey +ByteKeyRange +ByteKeyRangeTracker +ByteStringCoder +CalendarWindows +CalendarWindows.DaysWindows +CalendarWindows.MonthsWindows +CalendarWindows.YearsWindows +CannotProvideCoderException +CannotProvideCoderException.ReasonCode +Clock +CloudDebuggerOptions +CloudObject +CloudResourceManagerOptions +Coder +Coder.Context +Coder.NonDeterministicException +CoderException +CoderFactories +CoderFactory +CoderHelpers +CoderProperties +CoderProperties.TestElementByteSizeObserver +CoderProvider +CoderProviders +CoderRegistry +CoderUtils +CoGbkResult +CoGbkResult.CoGbkResultCoder +CoGbkResultSchema +CoGroupByKey +CollectionCoder +Combine +Combine.AccumulatingCombineFn +Combine.AccumulatingCombineFn.Accumulator +Combine.BinaryCombineDoubleFn +Combine.BinaryCombineFn +Combine.BinaryCombineIntegerFn +Combine.BinaryCombineLongFn +Combine.CombineFn +Combine.Globally +Combine.GloballyAsSingletonView +Combine.GroupedValues +Combine.Holder +Combine.IterableCombineFn +Combine.KeyedCombineFn +Combine.PerKey +Combine.PerKeyWithHotKeyFanout +Combine.SimpleCombineFn +CombineContextFactory +CombineFnBase +CombineFnBase.GlobalCombineFn +CombineFnBase.PerKeyCombineFn +CombineFns +CombineFns.CoCombineResult +CombineFns.ComposeCombineFnBuilder +CombineFns.ComposedCombineFn +CombineFns.ComposedCombineFnWithContext +CombineFns.ComposedKeyedCombineFn +CombineFns.ComposedKeyedCombineFnWithContext +CombineFns.ComposeKeyedCombineFnBuilder +CombineFnUtil +CombineWithContext +CombineWithContext.CombineFnWithContext +CombineWithContext.Context
[37/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/GroupByKeyViaGroupByKeyOnly.GroupByKeyOnly.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/GroupByKeyViaGroupByKeyOnly.GroupByKeyOnly.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/GroupByKeyViaGroupByKeyOnly.GroupByKeyOnly.html new file mode 100644 index 000..75643f2 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/GroupByKeyViaGroupByKeyOnly.GroupByKeyOnly.html @@ -0,0 +1,356 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +GroupByKeyViaGroupByKeyOnly.GroupByKeyOnly (Apache Beam SDK for Java, version 0.4.0) + + + + + + +var methods = {"i0":10,"i1":10}; +var tabs = {65535:["t0","All Methods"],2:["t2","Instance Methods"],8:["t4","Concrete Methods"]}; +var altColor = "altColor"; +var rowColor = "rowColor"; +var tableTab = "tableTab"; +var activeTableTab = "activeTableTab"; + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +PrevClass +NextClass + + +Frames +NoFrames + + +AllClasses + + + + + + + +Summary: +Nested| +Field| +Constr| +Method + + +Detail: +Field| +Constr| +Method + + + + + + + + +org.apache.beam.runners.core +Class GroupByKeyViaGroupByKeyOnly.GroupByKeyOnlyK,V + + + +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">java.lang.Object + + +org.apache.beam.sdk.transforms.PTransformPCollectionKVK,V,PCollectionKVK,http://docs.oracle.com/javase/7/docs/api/java/lang/Iterable.html?is-external=true; title="class or interface in java.lang">IterableWindowedValueV + + +org.apache.beam.runners.core.GroupByKeyViaGroupByKeyOnly.GroupByKeyOnlyK,V + + + + + + + + + +All Implemented Interfaces: +http://docs.oracle.com/javase/7/docs/api/java/io/Serializable.html?is-external=true; title="class or interface in java.io">Serializable, HasDisplayData + + +Enclosing class: +GroupByKeyViaGroupByKeyOnlyK,V + + + +public static class GroupByKeyViaGroupByKeyOnly.GroupByKeyOnlyK,V +extends PTransformPCollectionKVK,V,PCollectionKVK,http://docs.oracle.com/javase/7/docs/api/java/lang/Iterable.html?is-external=true; title="class or interface in java.lang">IterableWindowedValueV +Runner-specific primitive that groups by key only, ignoring any window assignments. A + runner that uses GroupByKeyViaGroupByKeyOnly should have a primitive way to translate + or evaluate this class. + +See Also: +Serialized Form + + + + + + + + + + + + +Field Summary + + + + +Fields inherited from classorg.apache.beam.sdk.transforms.PTransform +name + + + + + + + + +Constructor Summary + +Constructors + +Constructor and Description + + +GroupByKeyOnly() + + + + + + + + + +Method Summary + +All MethodsInstance MethodsConcrete Methods + +Modifier and Type +Method and Description + + +PCollectionKVK,http://docs.oracle.com/javase/7/docs/api/java/lang/Iterable.html?is-external=true; title="class or interface in java.lang">IterableWindowedValueV +expand(PCollectionKVK,Vinput) +Applies this PTransform on the given InputT, and returns its + Output. + + + +CoderKVK,http://docs.oracle.com/javase/7/docs/api/java/lang/Iterable.html?is-external=true; title="class or interface in java.lang">IterableV +getDefaultOutputCoder(PCollectionKVK,Vinput) +Returns the default Coder to use for the output of this + single-output PTransform when applied to the given input. + + + + + + + +Methods inherited from classorg.apache.beam.sdk.transforms.PTransform +getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, populateDisplayData, toString, validate + + + + + +Methods inherited from classjava.lang.http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Object +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#clone--; title="class or interface in java.lang">clone, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#equals-java.lang.Object-; title="class or interface in java.lang">equals, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#finalize--; title="class or interface in java.lang">finalize,
[02/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/dataflow/DataflowPipelineTranslator.TranslationContext.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/dataflow/DataflowPipelineTranslator.TranslationContext.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/dataflow/DataflowPipelineTranslator.TranslationContext.html new file mode 100644 index 000..9ab89fe --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/dataflow/DataflowPipelineTranslator.TranslationContext.html @@ -0,0 +1,540 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +DataflowPipelineTranslator.TranslationContext (Apache Beam SDK for Java, version 0.4.0) + + + + + + +var methods = {"i0":6,"i1":6,"i2":6,"i3":6,"i4":6,"i5":6,"i6":6,"i7":6,"i8":6,"i9":6,"i10":6,"i11":6,"i12":6,"i13":6,"i14":6,"i15":6,"i16":6}; +var tabs = {65535:["t0","All Methods"],2:["t2","Instance Methods"],4:["t3","Abstract Methods"]}; +var altColor = "altColor"; +var rowColor = "rowColor"; +var tableTab = "tableTab"; +var activeTableTab = "activeTableTab"; + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +PrevClass +NextClass + + +Frames +NoFrames + + +AllClasses + + + + + + + +Summary: +Nested| +Field| +Constr| +Method + + +Detail: +Field| +Constr| +Method + + + + + + + + +org.apache.beam.runners.dataflow +Interface DataflowPipelineTranslator.TranslationContext + + + + + + +Enclosing class: +DataflowPipelineTranslator + + + +public static interface DataflowPipelineTranslator.TranslationContext +The interface provided to registered callbacks for interacting + with the DataflowRunner, including reading and writing the + values of PCollections and side inputs (PCollectionViews). + + + + + + + + + + + +Method Summary + +All MethodsInstance MethodsAbstract Methods + +Modifier and Type +Method and Description + + +long +addCollectionToSingletonOutput(PValueinputValue, + PValueoutputValue) +Adds an output to the previously added CollectionToSingleton Dataflow step, + consuming the specified input PValue and producing the specified output + PValue. + + + +void +addEncodingInput(Coder?value) +Sets the encoding for the current Dataflow step. + + + +void +addInput(http://docs.oracle.com/javase/7/docs/api/java/lang/String.html?is-external=true; title="class or interface in java.lang">Stringname, +http://docs.oracle.com/javase/7/docs/api/java/lang/Boolean.html?is-external=true; title="class or interface in java.lang">Booleanvalue) +Adds an input with the given name and value to the current + Dataflow step. + + + +void +addInput(http://docs.oracle.com/javase/7/docs/api/java/lang/String.html?is-external=true; title="class or interface in java.lang">Stringname, +http://docs.oracle.com/javase/7/docs/api/java/util/List.html?is-external=true; title="class or interface in java.util">List? extends http://docs.oracle.com/javase/7/docs/api/java/util/Map.html?is-external=true; title="class or interface in java.util">Maphttp://docs.oracle.com/javase/7/docs/api/java/lang/String.html?is-external=true; title="class or interface in java.lang">String,http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Objectelements) +Adds an input that is a list of objects. + + + +void +addInput(http://docs.oracle.com/javase/7/docs/api/java/lang/String.html?is-external=true; title="class or interface in java.lang">Stringname, +http://docs.oracle.com/javase/7/docs/api/java/lang/Long.html?is-external=true; title="class or interface in java.lang">Longvalue) +Adds an input with the given name and value to the current + Dataflow step. + + + +void +addInput(http://docs.oracle.com/javase/7/docs/api/java/lang/String.html?is-external=true; title="class or interface in java.lang">Stringname, +http://docs.oracle.com/javase/7/docs/api/java/util/Map.html?is-external=true; title="class or interface in java.util">Maphttp://docs.oracle.com/javase/7/docs/api/java/lang/String.html?is-external=true; title="class or interface in java.lang">String,http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in
[50/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/allclasses-frame.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/allclasses-frame.html b/content/documentation/sdks/javadoc/0.4.0/allclasses-frame.html new file mode 100644 index 000..95e9fee --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/allclasses-frame.html @@ -0,0 +1,965 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +All Classes (Apache Beam SDK for Java, version 0.4.0) + + + + + +AllClasses + + +AccumulatorCombiningState +ActiveWindowSet +ActiveWindowSet.MergeCallback +AfterAll +AfterAllStateMachine +AfterDelayFromFirstElement +AfterDelayFromFirstElementStateMachine +AfterEach +AfterEachStateMachine +AfterFirst +AfterFirstStateMachine +AfterPane +AfterPaneStateMachine +AfterProcessingTime +AfterProcessingTimeStateMachine +AfterSynchronizedProcessingTime +AfterWatermark +AfterWatermark.AfterWatermarkEarlyAndLate +AfterWatermark.FromEndOfWindow +AfterWatermarkStateMachine +AfterWatermarkStateMachine.AfterWatermarkEarlyAndLate +AfterWatermarkStateMachine.FromEndOfWindow +AggAccumParam +Aggregator +Aggregator.AggregatorFactory +AggregatorContainer +AggregatorContainer.AggregatorKey +AggregatorContainer.Mutator +AggregatorFactory +AggregatorMetric +AggregatorMetricSource +AggregatorRetrievalException +AggregatorRetriever +AggregatorValues +ApexPipelineOptions +ApexRunner +ApexRunner.CreateApexPCollectionView +ApexRunnerRegistrar +ApexRunnerRegistrar.Options +ApexRunnerRegistrar.Runner +ApexRunnerResult +ApexYarnLauncher +ApexYarnLauncher.LaunchParams +ApexYarnLauncher.ProcessWatcher +ApiSurface +AppEngineEnvironment +ApplicationNameOptions +AppliedCombineFn +AppliedPTransform +ApproximateQuantiles +ApproximateQuantiles.ApproximateQuantilesCombineFn +ApproximateUnique +ApproximateUnique.ApproximateUniqueCombineFn +ApproximateUnique.ApproximateUniqueCombineFn.LargestUnique +AssignWindows +AssignWindowsDoFn +AtomicCoder +AttemptAndTimeBoundedExponentialBackOff +AttemptAndTimeBoundedExponentialBackOff.ResetPolicy +AttemptBoundedExponentialBackOff +AvroCoder +AvroHDFSFileSource +AvroIO +AvroIO.Read +AvroIO.Read.Bound +AvroIO.Write +AvroIO.Write.Bound +AvroSource +AvroSource.AvroReader +AvroUtils +AvroUtils.AvroMetadata +AvroWrapperCoder +BagState +BaseExecutionContext +BaseExecutionContext.CreateStepContextFunction +BaseExecutionContext.StepContext +BeamSparkRunnerRegistrator +BigDecimalCoder +BigEndianIntegerCoder +BigEndianLongCoder +BigIntegerCoder +BigQueryIO +BigQueryIO.Read +BigQueryIO.Read.Bound +BigQueryIO.Write +BigQueryIO.Write.Bound +BigQueryIO.Write.CreateDisposition +BigQueryIO.Write.WriteDisposition +BigqueryMatcher +BigQueryOptions +BigtableIO +BigtableIO.Read +BigtableIO.Write +BigtableTestOptions +BitSetCoder +BlockBasedSource +BlockBasedSource.Block +BlockBasedSource.BlockBasedReader +BoundedReadFromUnboundedSource +BoundedSource +BoundedSource.BoundedReader +BoundedWindow +BroadcastHelper +BucketingFunction +BufferedElementCountingOutputStream +BufferedExternalSorter +BufferedExternalSorter.Options +BundleFactory +ByteArray +ByteArrayCoder +ByteBuddyDoFnInvokerFactory +ByteBuddyDoFnInvokerFactory.DefaultRestrictionCoder +ByteBuddyDoFnInvokerFactory.DefaultSplitRestriction +ByteBuddyDoFnInvokerFactory.DoFnInvokerBase +ByteCoder +ByteKey +ByteKeyRange +ByteKeyRangeTracker +ByteStringCoder +CalendarWindows +CalendarWindows.DaysWindows +CalendarWindows.MonthsWindows +CalendarWindows.YearsWindows +CannotProvideCoderException +CannotProvideCoderException.ReasonCode +Clock +CloudDebuggerOptions +CloudObject +CloudResourceManagerOptions +Coder +Coder.Context +Coder.NonDeterministicException +CoderException +CoderFactories +CoderFactory +CoderHelpers +CoderProperties +CoderProperties.TestElementByteSizeObserver +CoderProvider +CoderProviders +CoderRegistry +CoderUtils +CoGbkResult +CoGbkResult.CoGbkResultCoder +CoGbkResultSchema +CoGroupByKey +CollectionCoder +Combine +Combine.AccumulatingCombineFn +Combine.AccumulatingCombineFn.Accumulator +Combine.BinaryCombineDoubleFn +Combine.BinaryCombineFn +Combine.BinaryCombineIntegerFn +Combine.BinaryCombineLongFn +Combine.CombineFn +Combine.Globally +Combine.GloballyAsSingletonView +Combine.GroupedValues +Combine.Holder +Combine.IterableCombineFn +Combine.KeyedCombineFn +Combine.PerKey +Combine.PerKeyWithHotKeyFanout +Combine.SimpleCombineFn +CombineContextFactory +CombineFnBase +CombineFnBase.GlobalCombineFn +CombineFnBase.PerKeyCombineFn +CombineFns +CombineFns.CoCombineResult +CombineFns.ComposeCombineFnBuilder +CombineFns.ComposedCombineFn +CombineFns.ComposedCombineFnWithContext +CombineFns.ComposedKeyedCombineFn +CombineFns.ComposedKeyedCombineFnWithContext +CombineFns.ComposeKeyedCombineFnBuilder +CombineFnUtil +CombineWithContext +CombineWithContext.CombineFnWithContext +CombineWithContext.Context +CombineWithContext.KeyedCombineFnWithContext
[44/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/apex/ApexRunnerRegistrar.Runner.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/apex/ApexRunnerRegistrar.Runner.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/apex/ApexRunnerRegistrar.Runner.html new file mode 100644 index 000..02ea533 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/apex/ApexRunnerRegistrar.Runner.html @@ -0,0 +1,292 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +ApexRunnerRegistrar.Runner (Apache Beam SDK for Java, version 0.4.0) + + + + + + +var methods = {"i0":10}; +var tabs = {65535:["t0","All Methods"],2:["t2","Instance Methods"],8:["t4","Concrete Methods"]}; +var altColor = "altColor"; +var rowColor = "rowColor"; +var tableTab = "tableTab"; +var activeTableTab = "activeTableTab"; + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +PrevClass +NextClass + + +Frames +NoFrames + + +AllClasses + + + + + + + +Summary: +Nested| +Field| +Constr| +Method + + +Detail: +Field| +Constr| +Method + + + + + + + + +org.apache.beam.runners.apex +Class ApexRunnerRegistrar.Runner + + + +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">java.lang.Object + + +org.apache.beam.runners.apex.ApexRunnerRegistrar.Runner + + + + + + + +All Implemented Interfaces: +PipelineRunnerRegistrar + + +Enclosing class: +ApexRunnerRegistrar + + + +@AutoService(value=PipelineRunnerRegistrar.class) +public static class ApexRunnerRegistrar.Runner +extends http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Object +implements PipelineRunnerRegistrar +Registers the ApexRunner. + + + + + + + + + + + +Constructor Summary + +Constructors + +Constructor and Description + + +Runner() + + + + + + + + + +Method Summary + +All MethodsInstance MethodsConcrete Methods + +Modifier and Type +Method and Description + + +http://docs.oracle.com/javase/7/docs/api/java/lang/Iterable.html?is-external=true; title="class or interface in java.lang">Iterablehttp://docs.oracle.com/javase/7/docs/api/java/lang/Class.html?is-external=true; title="class or interface in java.lang">Class? extends PipelineRunner? +getPipelineRunners() +Get the set of PipelineRunners to register. + + + + + + + +Methods inherited from classjava.lang.http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Object +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#clone--; title="class or interface in java.lang">clone, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#equals-java.lang.Object-; title="class or interface in java.lang">equals, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#finalize--; title="class or interface in java.lang">finalize, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#getClass--; title="class or interface in java.lang">getClass, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#hashCode--; title="class or interface in java.lang">hashCode, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#notify--; title="class or interface in java.lang">notify, http://docs.oracle.com/javase/7/docs/api/java/lang /Object.html?is-external=true#notifyAll--" title="class or interface in java.lang">notifyAll, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#toString--; title="class or interface in java.lang">toString, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#wait--; title="class or interface in java.lang">wait, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#wait-long-; title="class or interface in java.lang">wait, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#wait-long-int-; title="class or interface in java.lang">wait + + + + + + + + + + + + + + +Constructor Detail + + + + + +Runner +publicRunner() + + + + + + + + + +Method Detail + + + + + +getPipelineRunners
[34/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/PeekingReiterator.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/PeekingReiterator.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/PeekingReiterator.html new file mode 100644 index 000..e427d31 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/core/PeekingReiterator.html @@ -0,0 +1,383 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +PeekingReiterator (Apache Beam SDK for Java, version 0.4.0) + + + + + + +var methods = {"i0":10,"i1":10,"i2":10,"i3":10,"i4":10}; +var tabs = {65535:["t0","All Methods"],2:["t2","Instance Methods"],8:["t4","Concrete Methods"]}; +var altColor = "altColor"; +var rowColor = "rowColor"; +var tableTab = "tableTab"; +var activeTableTab = "activeTableTab"; + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +PrevClass +NextClass + + +Frames +NoFrames + + +AllClasses + + + + + + + +Summary: +Nested| +Field| +Constr| +Method + + +Detail: +Field| +Constr| +Method + + + + + + + + +org.apache.beam.runners.core +Class PeekingReiteratorT + + + +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">java.lang.Object + + +org.apache.beam.runners.core.PeekingReiteratorT + + + + + + + +Type Parameters: +T - the type of elements returned by this iterator + + +All Implemented Interfaces: +http://docs.oracle.com/javase/7/docs/api/java/util/Iterator.html?is-external=true; title="class or interface in java.util">IteratorT, ReiteratorT + + + +public final class PeekingReiteratorT +extends http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Object +implements ReiteratorT +A Reiterator that supports one-element lookahead during iteration. + + + + + + + + + + + +Constructor Summary + +Constructors + +Constructor and Description + + +PeekingReiterator(ReiteratorTiterator) + + + + + + + + + +Method Summary + +All MethodsInstance MethodsConcrete Methods + +Modifier and Type +Method and Description + + +PeekingReiteratorT +copy() +Returns a copy of the current Reiterator. + + + +boolean +hasNext() + + +T +next() + + +T +peek() +Returns the element that would be returned by next(), without + actually consuming the element. + + + +void +remove() + + + + + + +Methods inherited from classjava.lang.http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Object +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#clone--; title="class or interface in java.lang">clone, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#equals-java.lang.Object-; title="class or interface in java.lang">equals, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#finalize--; title="class or interface in java.lang">finalize, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#getClass--; title="class or interface in java.lang">getClass, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#hashCode--; title="class or interface in java.lang">hashCode, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#notify--; title="class or interface in java.lang">notify, http://docs.oracle.com/javase/7/docs/api/java/lang /Object.html?is-external=true#notifyAll--" title="class or interface in java.lang">notifyAll, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#toString--; title="class or interface in java.lang">toString, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#wait--; title="class or interface in java.lang">wait, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#wait-long-; title="class or interface in java.lang">wait, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#wait-long-int-; title="class or interface in java.lang">wait + + + + + +Methods inherited from interfacejava.util.http://docs.oracle.com/javase/7/docs/api/java/util/Iterator.html?is-external=true; title="class or interface
[03/52] [partial] beam-site git commit: Publish Javadoc for version 0.4.0
http://git-wip-us.apache.org/repos/asf/beam-site/blob/de6a0417/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/dataflow/DataflowPipelineRegistrar.Runner.html -- diff --git a/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/dataflow/DataflowPipelineRegistrar.Runner.html b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/dataflow/DataflowPipelineRegistrar.Runner.html new file mode 100644 index 000..b7b11a7 --- /dev/null +++ b/content/documentation/sdks/javadoc/0.4.0/org/apache/beam/runners/dataflow/DataflowPipelineRegistrar.Runner.html @@ -0,0 +1,292 @@ +http://www.w3.org/TR/html4/loose.dtd;> + + + + + +DataflowPipelineRegistrar.Runner (Apache Beam SDK for Java, version 0.4.0) + + + + + + +var methods = {"i0":10}; +var tabs = {65535:["t0","All Methods"],2:["t2","Instance Methods"],8:["t4","Concrete Methods"]}; +var altColor = "altColor"; +var rowColor = "rowColor"; +var tableTab = "tableTab"; +var activeTableTab = "activeTableTab"; + + +JavaScript is disabled on your browser. + + + + + +Skip navigation links + + + + +Overview +Package +Class +Use +Tree +Deprecated +Index +Help + + + + +PrevClass +NextClass + + +Frames +NoFrames + + +AllClasses + + + + + + + +Summary: +Nested| +Field| +Constr| +Method + + +Detail: +Field| +Constr| +Method + + + + + + + + +org.apache.beam.runners.dataflow +Class DataflowPipelineRegistrar.Runner + + + +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">java.lang.Object + + +org.apache.beam.runners.dataflow.DataflowPipelineRegistrar.Runner + + + + + + + +All Implemented Interfaces: +PipelineRunnerRegistrar + + +Enclosing class: +DataflowPipelineRegistrar + + + +@AutoService(value=PipelineRunnerRegistrar.class) +public static class DataflowPipelineRegistrar.Runner +extends http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Object +implements PipelineRunnerRegistrar +Register the DataflowRunner. + + + + + + + + + + + +Constructor Summary + +Constructors + +Constructor and Description + + +Runner() + + + + + + + + + +Method Summary + +All MethodsInstance MethodsConcrete Methods + +Modifier and Type +Method and Description + + +http://docs.oracle.com/javase/7/docs/api/java/lang/Iterable.html?is-external=true; title="class or interface in java.lang">Iterablehttp://docs.oracle.com/javase/7/docs/api/java/lang/Class.html?is-external=true; title="class or interface in java.lang">Class? extends PipelineRunner? +getPipelineRunners() +Get the set of PipelineRunners to register. + + + + + + + +Methods inherited from classjava.lang.http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true; title="class or interface in java.lang">Object +http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#clone--; title="class or interface in java.lang">clone, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#equals-java.lang.Object-; title="class or interface in java.lang">equals, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#finalize--; title="class or interface in java.lang">finalize, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#getClass--; title="class or interface in java.lang">getClass, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#hashCode--; title="class or interface in java.lang">hashCode, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#notify--; title="class or interface in java.lang">notify, http://docs.oracle.com/javase/7/docs/api/java/lang /Object.html?is-external=true#notifyAll--" title="class or interface in java.lang">notifyAll, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#toString--; title="class or interface in java.lang">toString, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#wait--; title="class or interface in java.lang">wait, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#wait-long-; title="class or interface in java.lang">wait, http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html?is-external=true#wait-long-int-; title="class or interface in java.lang">wait + + + + + + + + + + + + + + +Constructor Detail + + + + + +Runner +publicRunner() +
[GitHub] beam pull request #1752: Make DataflowPipelineTranslator public
GitHub user kennknowles opened a pull request: https://github.com/apache/beam/pull/1752 Make DataflowPipelineTranslator public There are a couple places outside the Beam codebase that still refer to this, and no great reason to break them. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- R: @lukecwik You can merge this pull request into a Git repository by running: $ git pull https://github.com/kennknowles/beam DataflowPipelineTranslator-public Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/1752.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1752 commit 67dfe787c632450de688507da6efebfdf2bc9b44 Author: Kenneth KnowlesDate: 2017-01-09T21:37:55Z Make DataflowPipelineTranslator public There are a couple places outside the Beam codebase that still refer to this, and no great reason to break them. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Closed] (BEAM-1246) Update README.md to remove incubating notion
[ https://issues.apache.org/jira/browse/BEAM-1246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Younghee Kwon closed BEAM-1246. --- Resolution: Fixed Fix Version/s: Not applicable PR merged. > Update README.md to remove incubating notion > > > Key: BEAM-1246 > URL: https://issues.apache.org/jira/browse/BEAM-1246 > Project: Beam > Issue Type: Task > Components: sdk-py >Reporter: Younghee Kwon >Assignee: Ahmet Altay >Priority: Trivial > Labels: documentation > Fix For: Not applicable > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (BEAM-1233) Implement TFRecordIO (Reading/writing Tensorflow Standard format)
[ https://issues.apache.org/jira/browse/BEAM-1233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Younghee Kwon closed BEAM-1233. --- > Implement TFRecordIO (Reading/writing Tensorflow Standard format) > - > > Key: BEAM-1233 > URL: https://issues.apache.org/jira/browse/BEAM-1233 > Project: Beam > Issue Type: New Feature > Components: sdk-py >Reporter: Younghee Kwon >Assignee: Ahmet Altay > Fix For: Not applicable > > > Tensorflow is an open source Machine Learning project, which is getting lots > of attention these days. Apache Beam can be used as a good preprocessing tool > for this Machine Learning tool, however Tensorflow supports limited number of > input file formats -- only csv and its own record format (so called TFRecord). > On the other hand, Apache Beam doesn't support reading/writing in TFRecord > format. This would be useful once it supports TFRecordIO natively. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (BEAM-1245) Use @unittest.skip instead of try/except in avroio_test
[ https://issues.apache.org/jira/browse/BEAM-1245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Younghee Kwon closed BEAM-1245. --- Resolution: Fixed Fix Version/s: Not applicable PR 1736 merged. > Use @unittest.skip instead of try/except in avroio_test > --- > > Key: BEAM-1245 > URL: https://issues.apache.org/jira/browse/BEAM-1245 > Project: Beam > Issue Type: Improvement > Components: sdk-py >Reporter: Younghee Kwon >Assignee: Ahmet Altay >Priority: Minor > Fix For: Not applicable > > > As said in the summary. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (BEAM-1233) Implement TFRecordIO (Reading/writing Tensorflow Standard format)
[ https://issues.apache.org/jira/browse/BEAM-1233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Younghee Kwon resolved BEAM-1233. - Resolution: Fixed Fix Version/s: Not applicable The PR that adds TFRecordIO is pushed to python-sdk branch. > Implement TFRecordIO (Reading/writing Tensorflow Standard format) > - > > Key: BEAM-1233 > URL: https://issues.apache.org/jira/browse/BEAM-1233 > Project: Beam > Issue Type: New Feature > Components: sdk-py >Reporter: Younghee Kwon >Assignee: Ahmet Altay > Fix For: Not applicable > > > Tensorflow is an open source Machine Learning project, which is getting lots > of attention these days. Apache Beam can be used as a good preprocessing tool > for this Machine Learning tool, however Tensorflow supports limited number of > input file formats -- only csv and its own record format (so called TFRecord). > On the other hand, Apache Beam doesn't support reading/writing in TFRecord > format. This would be useful once it supports TFRecordIO natively. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BEAM-1233) Implement TFRecordIO (Reading/writing Tensorflow Standard format)
[ https://issues.apache.org/jira/browse/BEAM-1233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15812873#comment-15812873 ] ASF GitHub Bot commented on BEAM-1233: -- Github user yk5 closed the pull request at: https://github.com/apache/beam/pull/1749 > Implement TFRecordIO (Reading/writing Tensorflow Standard format) > - > > Key: BEAM-1233 > URL: https://issues.apache.org/jira/browse/BEAM-1233 > Project: Beam > Issue Type: New Feature > Components: sdk-py >Reporter: Younghee Kwon >Assignee: Ahmet Altay > > Tensorflow is an open source Machine Learning project, which is getting lots > of attention these days. Apache Beam can be used as a good preprocessing tool > for this Machine Learning tool, however Tensorflow supports limited number of > input file formats -- only csv and its own record format (so called TFRecord). > On the other hand, Apache Beam doesn't support reading/writing in TFRecord > format. This would be useful once it supports TFRecordIO natively. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] beam pull request #1749: [BEAM-1233] Create TFRecordIO, providing source/sin...
Github user yk5 closed the pull request at: https://github.com/apache/beam/pull/1749 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[2/3] beam git commit: Provided temporary directory management for test cases.
Provided temporary directory management for test cases. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/93e8d19e Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/93e8d19e Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/93e8d19e Branch: refs/heads/python-sdk Commit: 93e8d19e32807fb5279ed711f0f06c3123adfb2e Parents: 88833ba Author: Younghee KwonAuthored: Mon Jan 9 11:50:57 2017 -0800 Committer: Robert Bradshaw Committed: Mon Jan 9 13:13:46 2017 -0800 -- sdks/python/apache_beam/io/tfrecordio_test.py | 58 +++--- 1 file changed, 41 insertions(+), 17 deletions(-) -- http://git-wip-us.apache.org/repos/asf/beam/blob/93e8d19e/sdks/python/apache_beam/io/tfrecordio_test.py -- diff --git a/sdks/python/apache_beam/io/tfrecordio_test.py b/sdks/python/apache_beam/io/tfrecordio_test.py index ee287b3..ecd58f5 100644 --- a/sdks/python/apache_beam/io/tfrecordio_test.py +++ b/sdks/python/apache_beam/io/tfrecordio_test.py @@ -20,8 +20,10 @@ import cStringIO import glob import gzip import logging +import os import pickle import random +import shutil import tempfile import unittest @@ -134,7 +136,29 @@ class TestTFRecordUtil(unittest.TestCase): self.assertEqual(record, actual) -class TestTFRecordSink(unittest.TestCase): +class _TestCaseWithTempDirCleanUp(unittest.TestCase): + """Base class for TestCases that deals with TempDir clean-up. + + Inherited test cases will call self._new_tempdir() to start a temporary dir + which will be deleted at the end of the tests (when tearDown() is called). + """ + + def setUp(self): +self._tempdirs = [] + + def tearDown(self): +for path in self._tempdirs: + if os.path.exists(path): +shutil.rmtree(path) +self._tempdirs = [] + + def _new_tempdir(self): +result = tempfile.mkdtemp() +self._tempdirs.append(result) +return result + + +class TestTFRecordSink(_TestCaseWithTempDirCleanUp): def _write_lines(self, sink, path, lines): f = sink.open(path) @@ -143,7 +167,7 @@ class TestTFRecordSink(unittest.TestCase): sink.close(f) def test_write_record_single(self): -path = tempfile.NamedTemporaryFile().name +path = os.path.join(self._new_tempdir(), 'result') record = binascii.a2b_base64(FOO_RECORD_BASE64) sink = _TFRecordSink( path, @@ -158,7 +182,7 @@ class TestTFRecordSink(unittest.TestCase): self.assertEqual(f.read(), record) def test_write_record_multiple(self): -path = tempfile.NamedTemporaryFile().name +path = os.path.join(self._new_tempdir(), 'result') record = binascii.a2b_base64(FOO_BAR_RECORD_BASE64) sink = _TFRecordSink( path, @@ -177,8 +201,8 @@ class TestTFRecordSink(unittest.TestCase): class TestWriteToTFRecord(TestTFRecordSink): def test_write_record_gzip(self): +file_path_prefix = os.path.join(self._new_tempdir(), 'result') with beam.Pipeline(DirectRunner()) as p: - file_path_prefix = tempfile.NamedTemporaryFile().name input_data = ['foo', 'bar'] _ = p | beam.Create(input_data) | WriteToTFRecord( file_path_prefix, compression_type=fileio.CompressionTypes.GZIP) @@ -192,8 +216,8 @@ class TestWriteToTFRecord(TestTFRecordSink): self.assertEqual(actual, input_data) def test_write_record_auto(self): +file_path_prefix = os.path.join(self._new_tempdir(), 'result') with beam.Pipeline(DirectRunner()) as p: - file_path_prefix = tempfile.NamedTemporaryFile().name input_data = ['foo', 'bar'] _ = p | beam.Create(input_data) | WriteToTFRecord( file_path_prefix, file_name_suffix='.gz') @@ -207,7 +231,7 @@ class TestWriteToTFRecord(TestTFRecordSink): self.assertEqual(actual, input_data) -class TestTFRecordSource(unittest.TestCase): +class TestTFRecordSource(_TestCaseWithTempDirCleanUp): def _write_file(self, path, base64_records): record = binascii.a2b_base64(base64_records) @@ -220,7 +244,7 @@ class TestTFRecordSource(unittest.TestCase): f.write(record) def test_process_single(self): -path = tempfile.NamedTemporaryFile().name +path = os.path.join(self._new_tempdir(), 'result') self._write_file(path, FOO_RECORD_BASE64) with beam.Pipeline(DirectRunner()) as p: result = (p @@ -232,7 +256,7 @@ class TestTFRecordSource(unittest.TestCase): beam.assert_that(result, beam.equal_to(['foo'])) def test_process_multiple(self): -path = tempfile.NamedTemporaryFile().name +path = os.path.join(self._new_tempdir(), 'result') self._write_file(path, FOO_BAR_RECORD_BASE64) with beam.Pipeline(DirectRunner()) as p: result = (p @@ -244,7 +268,7
[3/3] beam git commit: Closes #1749
Closes #1749 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/a2551517 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/a2551517 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/a2551517 Branch: refs/heads/python-sdk Commit: a2551517184e55ecf01d07dffd921b105f7dc448 Parents: 69d8f2b 93e8d19 Author: Robert BradshawAuthored: Mon Jan 9 13:13:55 2017 -0800 Committer: Robert Bradshaw Committed: Mon Jan 9 13:13:55 2017 -0800 -- sdks/python/apache_beam/io/__init__.py| 1 + sdks/python/apache_beam/io/tfrecordio.py | 271 ++ sdks/python/apache_beam/io/tfrecordio_test.py | 389 + sdks/python/setup.py | 1 + 4 files changed, 662 insertions(+) --
[jira] [Commented] (BEAM-643) Allow users to specify a custom service account
[ https://issues.apache.org/jira/browse/BEAM-643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15812789#comment-15812789 ] Chamikara Jayalath commented on BEAM-643: - Java SDK was updated in https://github.com/apache/beam/pull/1163 > Allow users to specify a custom service account > --- > > Key: BEAM-643 > URL: https://issues.apache.org/jira/browse/BEAM-643 > Project: Beam > Issue Type: New Feature > Components: sdk-py >Reporter: Chamikara Jayalath >Assignee: Chamikara Jayalath > Fix For: Not applicable > > > Users should be able to specify a custom service account which can be used > when creating VMs. This feature is specify to DataflowRunner and > corresponding user option will be added to GoogleCloudOptions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (BEAM-578) Update FileBasedSource so that implementations can control whether the source gets split into ranges
[ https://issues.apache.org/jira/browse/BEAM-578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-578. - Resolution: Fixed Fix Version/s: Not applicable > Update FileBasedSource so that implementations can control whether the source > gets split into ranges > > > Key: BEAM-578 > URL: https://issues.apache.org/jira/browse/BEAM-578 > Project: Beam > Issue Type: Bug > Components: sdk-py >Reporter: Chamikara Jayalath >Assignee: Chamikara Jayalath > Fix For: Not applicable > > > Some FileBasedSource implementations need to prevent the source from being > split into data ranges through initial or dynamic splitting. We should > provide a way to control this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (BEAM-553) Add a custom text source
[ https://issues.apache.org/jira/browse/BEAM-553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath resolved BEAM-553. - Resolution: Fixed Fix Version/s: Not applicable > Add a custom text source > > > Key: BEAM-553 > URL: https://issues.apache.org/jira/browse/BEAM-553 > Project: Beam > Issue Type: New Feature > Components: sdk-py >Reporter: Chamikara Jayalath >Assignee: Chamikara Jayalath > Fix For: Not applicable > > > Currently, the text source implementation available for Python SDK [1] is a > Dataflow native source which only works efficiently for Dataflow runner. We > should add a custom text source on top of custom file-based source framework > [2] so that other runner implementations can potentially use the same text > source implementation. > Custom text source implementation for Java SDK is at [3]. > [1] > https://github.com/apache/incubator-beam/blob/python-sdk/sdks/python/apache_beam/io/fileio.py#L70 > [2] > https://github.com/apache/incubator-beam/blob/python-sdk/sdks/python/apache_beam/io/filebasedsource.py > [3] > https://github.com/apache/incubator-beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/io/TextIO.java#L745 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BEAM-106) Native support for conditional iteration
[ https://issues.apache.org/jira/browse/BEAM-106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Diomin updated BEAM-106: --- Description: Ported from: https://github.com/GoogleCloudPlatform/DataflowJavaSDK/issues/50 There are a variety of use cases which would benefit from native support for conditional iteration. For instance, http://stackoverflow.com/questions/31654421/conditional-iterations-in-google-cloud-dataflow/31659923?noredirect=1#comment51264604_31659923 asks about being able to write a loop like the following: {code} PCollection data = ... while(needsMoreWork(data)) { data = doAStep(data) } {code} If there are specific use cases please let us know the details. In the future we will use this issue to post progress updates. was: Ported from: https://github.com/GoogleCloudPlatform/DataflowJavaSDK/issues/50 There are a variety of use cases which would benefit from native support for conditional iteration. For instance, http://stackoverflow.com/questions/31654421/conditional-iterations-in-google-cloud-dataflow/31659923?noredirect=1#comment51264604_31659923 asks about being able to write a loop like the following: PCollection data = ... while(needsMoreWork(data)) { data = doAStep(data) } If there are specific use cases please let us know the details. In the future we will use this issue to post progress updates. > Native support for conditional iteration > > > Key: BEAM-106 > URL: https://issues.apache.org/jira/browse/BEAM-106 > Project: Beam > Issue Type: New Feature > Components: sdk-ideas >Reporter: Luke Cwik >Assignee: James Malone > > Ported from: https://github.com/GoogleCloudPlatform/DataflowJavaSDK/issues/50 > There are a variety of use cases which would benefit from native support for > conditional iteration. > For instance, > http://stackoverflow.com/questions/31654421/conditional-iterations-in-google-cloud-dataflow/31659923?noredirect=1#comment51264604_31659923 > asks about being able to write a loop like the following: > {code} > PCollection data = ... > while(needsMoreWork(data)) { > data = doAStep(data) > } > {code} > If there are specific use cases please let us know the details. In the future > we will use this issue to post progress updates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BEAM-849) Redesign PipelineResult API
[ https://issues.apache.org/jira/browse/BEAM-849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15812425#comment-15812425 ] Luke Cwik commented on BEAM-849: [~staslev], a user needs to understand that a Pipeline will be executing asynchronously which means the State from `getState()` can be stale or out of date as soon as it is returned. I would argue for returning state from `waitUntilFinish()` or `cancel()`. I would argue that in the case of timeout, `waitUntilFinish()` should throw an exception instead of returning `null` since the user requested for a terminal state yet we failed to do so. > Redesign PipelineResult API > --- > > Key: BEAM-849 > URL: https://issues.apache.org/jira/browse/BEAM-849 > Project: Beam > Issue Type: Improvement > Components: sdk-java-core >Reporter: Pei He > > Current state: > Jira https://issues.apache.org/jira/browse/BEAM-443 addresses > waitUntilFinish() and cancel(). > However, there are additional work around PipelineResult: > need clearly defined contract and verification across all runners > need to revisit how to handle metrics/aggregators > need to be able to get logs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BEAM-773) Implement Metrics support for Flink runner
[ https://issues.apache.org/jira/browse/BEAM-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15811610#comment-15811610 ] Aljoscha Krettek commented on BEAM-773: --- [~bchambers] is correct on both issues: AFAIK no one is working on this right now and wiring the Beam Metrics to the Flink Metrics will only provide "attempted metrics". [~celix] If your interested in working on this I'm happy to help. > Implement Metrics support for Flink runner > -- > > Key: BEAM-773 > URL: https://issues.apache.org/jira/browse/BEAM-773 > Project: Beam > Issue Type: Sub-task > Components: runner-flink >Reporter: Ben Chambers > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (BEAM-301) Add a Beam SQL DSL
[ https://issues.apache.org/jira/browse/BEAM-301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aravind S updated BEAM-301: --- Comment: was deleted (was: [~neelesh77] I'm looking for this feature. I could help in building this feature. It will be very powerful.) > Add a Beam SQL DSL > -- > > Key: BEAM-301 > URL: https://issues.apache.org/jira/browse/BEAM-301 > Project: Beam > Issue Type: New Feature > Components: sdk-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Neelesh Srinivas Salian > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BEAM-301) Add a Beam SQL DSL
[ https://issues.apache.org/jira/browse/BEAM-301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15811482#comment-15811482 ] Aravind S commented on BEAM-301: [~neelesh77] I'm looking for this feature. I could help in building this feature. It will be very powerful. > Add a Beam SQL DSL > -- > > Key: BEAM-301 > URL: https://issues.apache.org/jira/browse/BEAM-301 > Project: Beam > Issue Type: New Feature > Components: sdk-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Neelesh Srinivas Salian > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BEAM-301) Add a Beam SQL DSL
[ https://issues.apache.org/jira/browse/BEAM-301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15811481#comment-15811481 ] Aravind S commented on BEAM-301: [~neelesh77] I'm looking for this feature. I could help in building this feature. It will be very powerful. > Add a Beam SQL DSL > -- > > Key: BEAM-301 > URL: https://issues.apache.org/jira/browse/BEAM-301 > Project: Beam > Issue Type: New Feature > Components: sdk-ideas >Reporter: Jean-Baptiste Onofré >Assignee: Neelesh Srinivas Salian > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] beam pull request #1751: Add perf tests to Jenkins
GitHub user jasonkuster opened a pull request: https://github.com/apache/beam/pull/1751 Add perf tests to Jenkins Signed-off-by: Jason KusterBe sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/jasonkuster/incubator-beam perf Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/1751.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1751 commit 07d3198953baabd36ce29210cbb8a0d9150e1869 Author: Jason Kuster Date: 2017-01-09T10:26:22Z Add perf tests to Jenkins Signed-off-by: Jason Kuster --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---