[2/2] beam git commit: This closes #2420

This closes #2420 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/a0cfccda Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/a0cfccda Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/a0cfccda Branch:

[1/2] beam git commit: [BEAM-1737] Implement a Single-output ParDo as a Multi-output ParDo with a single output.

Repository: beam Updated Branches: refs/heads/master 810db7f99 -> a0cfccda4 [BEAM-1737] Implement a Single-output ParDo as a Multi-output ParDo with a single output. remove use of EvaluationContext in DStream lambda, it is not serializable and also redundant in this case. implement pardo

[GitHub] beam pull request #2420: [BEAM-1737] Interpreting a Single-output ParDo as a...

GitHub user amitsela opened a pull request: https://github.com/apache/beam/pull/2420 [BEAM-1737] Interpreting a Single-output ParDo as a Multi-output ParDo with a single output causes serialization failuresremove use of EvaluationContext in DStream lambda, it is not serializ… â

[GitHub] beam pull request #2419: [BEAM-1875] Remove Spark runner custom Hadoop and A...

GitHub user amitsela opened a pull request: https://github.com/apache/beam/pull/2419 [BEAM-1875] Remove Spark runner custom Hadoop and Avro IOs. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[1/2] beam git commit: [BEAM-1827] Fix use of deprecated Spark APIs in the runner.

Repository: beam Updated Branches: refs/heads/master 99056df36 -> 8a33591d9 [BEAM-1827] Fix use of deprecated Spark APIs in the runner. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/6671b5b6 Tree:

[2/2] beam git commit: This closes #2354

This closes #2354 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/8a33591d Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/8a33591d Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/8a33591d Branch:

[GitHub] beam pull request #2354: [BEAM-1827] Fix use of deprecated Spark APIs in the...

GitHub user amitsela opened a pull request: https://github.com/apache/beam/pull/2354 [BEAM-1827] Fix use of deprecated Spark APIs in the runner. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[2/2] beam git commit: This closes #2334

This closes #2334 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/434eadb5 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/434eadb5 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/434eadb5 Branch:

[GitHub] beam pull request #2334: [BEAM-1815] Avoid shuffling twice in GABW.

GitHub user amitsela opened a pull request: https://github.com/apache/beam/pull/2334 [BEAM-1815] Avoid shuffling twice in GABW. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted

[1/2] beam git commit: Fix caching in the Spark streaming, doing the cache update in the streaming context

Repository: beam Updated Branches: refs/heads/master fe2369933 -> 3473055f2 Fix caching in the Spark streaming, doing the cache update in the streaming context Fix order of call Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit:

[GitHub] beam pull request #2303: [BEAM-1717] maven-javadoc-plugin execution id shoul...

GitHub user amitsela opened a pull request: https://github.com/apache/beam/pull/2303 [BEAM-1717] maven-javadoc-plugin execution id should match the id in … …apache parent. Be sure to do all of the following to help us incorporate your contribution quickly and easily

[2/2] beam git commit: This closes #2272

This closes #2272 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/faa9645d Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/faa9645d Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/faa9645d Branch:

[1/2] beam git commit: [BEAM-1752, BEAM-1582] execute tests that recover from checkpoint in post-commit. Add UsesCheckpointRecovery tag to mark Spark runner tests that recover from checkpoint.

Repository: beam Updated Branches: refs/heads/master 3f0fe91aa -> faa9645d2 [BEAM-1752, BEAM-1582] execute tests that recover from checkpoint in post-commit. Add UsesCheckpointRecovery tag to mark Spark runner tests that recover from checkpoint. Mark ResumeFromCheckpointStreamingTest with

[2/2] beam git commit: This closes #2259

This closes #2259 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/b75888e3 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/b75888e3 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/b75888e3 Branch:

[1/2] beam git commit: Fix shading opt out in io/google-cloud-platform

Repository: beam Updated Branches: refs/heads/master db005fffb -> b75888e32 Fix shading opt out in io/google-cloud-platform Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/dd1b0017 Tree:

[2/2] beam git commit: This closes #2226

This closes #2226 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/781e4172 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/781e4172 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/781e4172 Branch:

[1/2] beam git commit: Revert BigQueryIO bit of 'Make all uses of CountingOutputStream close their resources'

Repository: beam Updated Branches: refs/heads/master b6ca062fc -> 781e4172c Revert BigQueryIO bit of 'Make all uses of CountingOutputStream close their resources' This reverts the portion of commit 3115dbdca1858511e98476b5c79e6cca98782b0b that touches BigQueryIO, which caused a double close

[1/2] beam git commit: [BEAM-797] A PipelineVisitor that creates a Spark-native pipeline.

Repository: beam Updated Branches: refs/heads/master 2c2424cb4 -> c12d432d8 [BEAM-797] A PipelineVisitor that creates a Spark-native pipeline. [BEAM-797] Remove unnecessary temp dir from test Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit:

[2/2] beam git commit: This closes #1868

This closes #1868 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/c12d432d Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/c12d432d Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/c12d432d Branch:

[2/2] beam git commit: This closes #2168

This closes #2168 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/efc701ed Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/efc701ed Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/efc701ed Branch:

[1/2] beam git commit: Test runner to stop on EOT watermark, or timeout.

Repository: beam Updated Branches: refs/heads/master 7954896a5 -> efc701ed6 Test runner to stop on EOT watermark, or timeout. Remove timeout since it is already a pipeline option. Advance to infinity at the end of pipelines. Add EOT watermark and expected assertions test options.

[2/2] beam git commit: This closes #2180

This closes #2180 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/c79bd95b Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/c79bd95b Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/c79bd95b Branch:

[1/2] beam git commit: [BEAM-1636] UnboundedDataset action() does not materialize RDD

Repository: beam Updated Branches: refs/heads/master 1fd52f53c -> c79bd95bd [BEAM-1636] UnboundedDataset action() does not materialize RDD Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/a889597e Tree:

[GitHub] beam pull request #2169: [BEAM-1556] Make PipelineOptions a lazy-singleton a...

GitHub user amitsela opened a pull request: https://github.com/apache/beam/pull/2169 [BEAM-1556] Make PipelineOptions a lazy-singleton and init IOs as par… …t of it. Be sure to do all of the following to help us incorporate your contribution quickly and easily

[GitHub] beam pull request #2168: [BEAM-1582, BEAM-1562] Stop streaming tests on EOT ...

GitHub user amitsela opened a pull request: https://github.com/apache/beam/pull/2168 [BEAM-1582, BEAM-1562] Stop streaming tests on EOT Watermark. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR

[2/2] beam git commit: This closes #2160

This closes #2160 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/34b38ef9 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/34b38ef9 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/34b38ef9 Branch:

[1/2] beam git commit: [BEAM-1623] Transform Reshuffle directly in Spark runner

Repository: beam Updated Branches: refs/heads/master 69d951225 -> 34b38ef95 [BEAM-1623] Transform Reshuffle directly in Spark runner Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/d8bc618e Tree:

[2/2] beam git commit: This closes #2163

This closes #2163 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/69d95122 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/69d95122 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/69d95122 Branch:

[GitHub] beam pull request #2163: [BEAM-1626] Remove cache of MapWithStateDStream on ...

GitHub user amitsela opened a pull request: https://github.com/apache/beam/pull/2163 [BEAM-1626] Remove cache of MapWithStateDStream on read. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[2/2] beam git commit: This closes #2141

This closes #2141 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/6a33dc66 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/6a33dc66 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/6a33dc66 Branch:

[1/2] beam git commit: [BEAM-1565] Update Spark runner PostCommit Jenkins job.

Repository: beam Updated Branches: refs/heads/master 1a770ef2f -> 6a33dc666 [BEAM-1565] Update Spark runner PostCommit Jenkins job. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/15e14951 Tree:

[1/2] beam git commit: [BEAM-111] Move WritableCoder to hadoop-common

Repository: beam Updated Branches: refs/heads/master b49ec3fa2 -> a81c45781 [BEAM-111] Move WritableCoder to hadoop-common Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/44624c38 Tree:

[1/2] beam git commit: [BEAM-351] Add DisplayData to KafkaIO

Repository: beam Updated Branches: refs/heads/master d84b06791 -> 3b3d6b81a [BEAM-351] Add DisplayData to KafkaIO Changes after review. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/52e2d3a7 Tree:

[2/2] beam git commit: This closes #2111

This closes #2111 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/3b3d6b81 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/3b3d6b81 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/3b3d6b81 Branch:

[11/23] beam git commit: Add multi stream and flattened stream tests.

Add multi stream and flattened stream tests. Misc. fixups. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/24ab6053 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/24ab6053 Diff:

[16/23] beam git commit: Batch doesn't use checkpoint dir so nothing to clean.

Batch doesn't use checkpoint dir so nothing to clean. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/82d754cf Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/82d754cf Diff:

[06/23] beam git commit: Utils for SparkGroupAlsoByWindowViaWindowSet.

Utils for SparkGroupAlsoByWindowViaWindowSet. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/8c379704 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/8c379704 Diff:

[09/23] beam git commit: Fix streaming translation of Flatten and Window, make CreateStream eager.

Fix streaming translation of Flatten and Window, make CreateStream eager. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/9f143504 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/9f143504 Diff:

[05/23] beam git commit: Refactor translators according to new GroupAlsoByWindow implemenation for the Spark runnner.

Refactor translators according to new GroupAlsoByWindow implemenation for the Spark runnner. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/96abe4f0 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/96abe4f0 Diff:

[20/23] beam git commit: Make CreateStream a TimestampedValue Source.

Make CreateStream a TimestampedValue Source. fixup! a nicer DSL to construct CreateStream. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/62ddca63 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/62ddca63 Diff:

[01/23] beam git commit: Implementation of GroupAlsoByWindowViaWindowSet for the Spark runner.

Repository: beam Updated Branches: refs/heads/master a41afdc68 -> 61e31e622 Implementation of GroupAlsoByWindowViaWindowSet for the Spark runner. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/32a9d61e Tree:

[03/23] beam git commit: Handle test failures in "graceful stop peroid".

Handle test failures in "graceful stop peroid". Further refactoring following changes. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/b21de69e Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/b21de69e Diff:

[21/23] beam git commit: Spark GABWVOB to use UnsupportedSideInputReader.

Spark GABWVOB to use UnsupportedSideInputReader. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/3c94ff20 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/3c94ff20 Diff:

[13/23] beam git commit: Fix runnable-on-service profile in the Spark runner.

Fix runnable-on-service profile in the Spark runner. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/c08e7d86 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/c08e7d86 Diff:

[14/23] beam git commit: Batch executions should block without timeout.

Batch executions should block without timeout. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/3867dcd7 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/3867dcd7 Diff:

[17/23] beam git commit: Build trigger state machine from Runner API Trigger proto directly

Build trigger state machine from Runner API Trigger proto directly Remove printouts of stage progress in tests. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/bd41d9a3 Tree:

[15/23] beam git commit: Tests that can should run with TestSparkRunner.

Tests that can should run with TestSparkRunner. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/b88e54a9 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/b88e54a9 Diff:

[23/23] beam git commit: This closes #2050

This closes #2050 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/61e31e62 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/61e31e62 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/61e31e62 Branch:

[04/23] beam git commit: Use TestSparkRunner in tests.

Use TestSparkRunner in tests. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/2bcd40c2 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/2bcd40c2 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/2bcd40c2 Branch:

[12/23] beam git commit: Make TimestampTransform Serializable.

Make TimestampTransform Serializable. Rebase leftover fixes. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/f47e0eba Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/f47e0eba Diff:

[18/23] beam git commit: Use a PipelineRule for test pipelines.

Use a PipelineRule for test pipelines. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/4ca56806 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/4ca56806 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/4ca56806

[19/23] beam git commit: Move LateDataUtils and UnsupportedSideInputReader to runners-core.

Move LateDataUtils and UnsupportedSideInputReader to runners-core. Add missing licenses. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/123f4820 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/123f4820 Diff:

[02/23] beam git commit: Better name for batch implementation of GroupAlsoByWindow.

Better name for batch implementation of GroupAlsoByWindow. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/bf0c119b Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/bf0c119b Diff:

[08/23] beam git commit: Test triggers, panes and watermarks via CreateStream.

Test triggers, panes and watermarks via CreateStream. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/96d373fe Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/96d373fe Diff:

[22/23] beam git commit: Streaming tests, especially the ones using checkpoints, need a time buffer to finish.

Streaming tests, especially the ones using checkpoints, need a time buffer to finish. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/2c707431 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/2c707431 Diff:

[10/23] beam git commit: Serialize state stream with coders for shuffle and checkpointing.

Serialize state stream with coders for shuffle and checkpointing. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/da5f8497 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/da5f8497 Diff:

[GitHub] beam pull request #495: Spark runner upgrade to Spark 2.0

Github user amitsela closed the pull request at: https://github.com/apache/beam/pull/495 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[1/2] beam git commit: [BEAM-1526] Suppress Spark WatermarkTest#testInDoFn due to flakes

Repository: beam Updated Branches: refs/heads/master ed0d4577d -> 5ca149de2 [BEAM-1526] Suppress Spark WatermarkTest#testInDoFn due to flakes Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/146ce988 Tree:

[GitHub] beam pull request #2050: [BEAM-920] Support triggers, panes and watermarks.

GitHub user amitsela opened a pull request: https://github.com/apache/beam/pull/2050 [BEAM-920] Support triggers, panes and watermarks. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[2/2] beam git commit: This closes #2046

This closes #2046 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/4e5a762e Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/4e5a762e Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/4e5a762e Branch:

[1/2] beam git commit: [BEAM-1512] Optimize leaf transforms materialization

Repository: beam Updated Branches: refs/heads/master aa45ccb08 -> 4e5a762ef [BEAM-1512] Optimize leaf transforms materialization Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/a2f0615f Tree:

[2/9] beam git commit: Advance watermarks onBatchCompleted hook.

Advance watermarks onBatchCompleted hook. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/fa31f18e Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/fa31f18e Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/fa31f18e

[7/9] beam git commit: Streaming sources tracking test.

Streaming sources tracking test. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/add87166 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/add87166 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/add87166 Branch:

[8/9] beam git commit: This relied on a wrong functionality as described in BEAM-1444 and should be revisited there.

This relied on a wrong functionality as described in BEAM-1444 and should be revisited there. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/3d25b9cc Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/3d25b9cc Diff:

[3/9] beam git commit: Handle QueuedStream (for testing) and track sources upstream.

Handle QueuedStream (for testing) and track sources upstream. Refactor according to changes. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/705695eb Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/705695eb Diff:

[9/9] beam git commit: This closes #1987

This closes #1987 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/aa45ccb0 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/aa45ccb0 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/aa45ccb0 Branch:

[4/9] beam git commit: Ingest the input watermarks into the GlobalWatermarkHolder.

Ingest the input watermarks into the GlobalWatermarkHolder. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/bbf3744d Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/bbf3744d Diff:

[5/9] beam git commit: Watermark tests.

Watermark tests. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/9784f204 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/9784f204 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/9784f204 Branch:

[1/9] beam git commit: A global Watermark holder to update and broadcast to workers.

Repository: beam Updated Branches: refs/heads/master 92190ba5d -> aa45ccb08 A global Watermark holder to update and broadcast to workers. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/a6206535 Tree:

[6/9] beam git commit: Exopse RegexMatcher as part of the SDK.

Exopse RegexMatcher as part of the SDK. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/c18f8a2c Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/c18f8a2c Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/c18f8a2c

[2/7] beam git commit: Throw UnsupportedOperationException for committed metrics results in spark runner

Throw UnsupportedOperationException for committed metrics results in spark runner Added metrics support for MultiDo Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/d7d49ce8 Tree:

[3/7] beam git commit: Remove duplicate classes from spark runner marking sdk classes Serializable instead.

Remove duplicate classes from spark runner marking sdk classes Serializable instead. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/22865780 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/22865780 Diff:

[4/7] beam git commit: Register beam metrics with a MetricSource in Spark

http://git-wip-us.apache.org/repos/asf/beam/blob/31624fed/runners/spark/src/test/resources/metrics.properties -- diff --git a/runners/spark/src/test/resources/metrics.properties

[1/7] beam git commit: [BEAM-774] Implement Metrics support for Spark runner

Repository: beam Updated Branches: refs/heads/master e720a7c43 -> 24ecf6bbf [BEAM-774] Implement Metrics support for Spark runner Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/8e203ea2 Tree:

[5/7] beam git commit: Register beam metrics with a MetricSource in Spark

Register beam metrics with a MetricSource in Spark Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/31624fed Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/31624fed Diff:

[6/7] beam git commit: Recover metrics values from checkpoint

Recover metrics values from checkpoint Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/3784b541 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/3784b541 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/3784b541

[7/7] beam git commit: This closes #1750

This closes #1750 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/24ecf6bb Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/24ecf6bb Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/24ecf6bb Branch:

[GitHub] beam pull request #1987: [BEAM-920] Add support for Watermarks in the Spark ...

GitHub user amitsela opened a pull request: https://github.com/apache/beam/pull/1987 [BEAM-920] Add support for Watermarks in the Spark runner. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[1/3] beam git commit: [BEAM-1405] Refactor to remove repeated code from test

Repository: beam Updated Branches: refs/heads/master 9dad73c29 -> ed7b82e7e [BEAM-1405] Refactor to remove repeated code from test Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/92707b9a Tree:

[2/3] beam git commit: [BEAM-1405] Skip stopping context when spark context is provided

[BEAM-1405] Skip stopping context when spark context is provided Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/5edcdff9 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/5edcdff9 Diff:

[2/2] beam git commit: This closes #1922

This closes #1922 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/e0189f35 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/e0189f35 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/e0189f35 Branch:

[1/2] beam git commit: [BEAM-1395] Remove chunking.

Repository: beam Updated Branches: refs/heads/master 1e49ee8f2 -> e0189f352 [BEAM-1395] Remove chunking. fixup! formatting. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/71197ae6 Tree:

[GitHub] beam pull request #1922: [BEAM-1395] SparkGroupAlsoByWindowFn not sorting gr...

GitHub user amitsela opened a pull request: https://github.com/apache/beam/pull/1922 [BEAM-1395] SparkGroupAlsoByWindowFn not sorting grouped elements by timestamp Be sure to do all of the following to help us incorporate your contribution quickly and easily

[2/2] beam git commit: This closes #1920

This closes #1920 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/e5afbb27 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/e5afbb27 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/e5afbb27 Branch:

[1/2] beam git commit: [BEAM-648] Persist and restore Aggergator values in case of recovery from failure

Repository: beam Updated Branches: refs/heads/master 343176c00 -> 847e4e9f0 [BEAM-648] Persist and restore Aggergator values in case of recovery from failure Added javadoc and minor refactor Moved creation of beam checkpoint dir Project: http://git-wip-us.apache.org/repos/asf/beam/repo

[2/2] beam git commit: This closes #1815

This closes #1815 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/847e4e9f Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/847e4e9f Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/847e4e9f Branch:

[1/3] beam git commit: [BEAM-1291] KafkaIO: don't log warnig in offset fetcher while closing.

Repository: beam Updated Branches: refs/heads/master f799a57af -> 09d131ced [BEAM-1291] KafkaIO: don't log warnig in offset fetcher while closing. add partition to the message. break when the reader is closed. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit:

[2/3] beam git commit: added comment on ignore exception.

added comment on ignore exception. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/2e9cde24 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/2e9cde24 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/2e9cde24

[1/2] beam git commit: Add jackson specific version + shade only relocate classes

Repository: beam Updated Branches: refs/heads/master b2d300858 -> 420367088 Add jackson specific version + shade only relocate classes Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/224f5208 Tree:

[GitHub] beam pull request #1758: [BEAM-1256] SparkPipelineResult.StreamingMode shoul...

Github user amitsela closed the pull request at: https://github.com/apache/beam/pull/1758 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] beam pull request #1758: [BEAM-1256] SparkPipelineResult.StreamingMode shoul...

GitHub user amitsela opened a pull request: https://github.com/apache/beam/pull/1758 [BEAM-1256] SparkPipelineResult.StreamingMode should not return null … …on awaitTermination() Be sure to do all of the following to help us incorporate your contribution quickly

[3/3] beam git commit: This closes #1624

This closes #1624 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/c1b7f869 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/c1b7f869 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/c1b7f869 Branch:

[2/3] beam git commit: Create broadcast lazily

Create broadcast lazily Fix Amit's comments + rename BroadcastHelper to SideInputBroadcast Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/662934b1 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/662934b1 Diff:

[1/2] beam git commit: [BEAM-1250] Remove leaf when materializing PCollection to avoid re-evaluation.

Repository: beam Updated Branches: refs/heads/master 418c597c4 -> 70dd54625 [BEAM-1250] Remove leaf when materializing PCollection to avoid re-evaluation. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/8d3d381c Tree:

[GitHub] beam pull request #1747: [BEAM-1250] Remove leaf when materializing PCollect...

GitHub user amitsela opened a pull request: https://github.com/apache/beam/pull/1747 [BEAM-1250] Remove leaf when materializing PCollection to avoid re-ev… …aluation. Be sure to do all of the following to help us incorporate your contribution quickly and easily

[2/2] beam git commit: This closes #1594

This closes #1594 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/70ff6bfd Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/70ff6bfd Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/70ff6bfd Branch:

[1/2] beam git commit: [BEAM-1145] Remove classifier from shaded spark runner artifact

Repository: beam Updated Branches: refs/heads/master ee69825ea -> 70ff6bfd9 [BEAM-1145] Remove classifier from shaded spark runner artifact Hanging integration test fix Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit:

[1/2] beam git commit: [BEAM-1146] Decrease spark runner startup overhead

Repository: beam Updated Branches: refs/heads/master e136f12c3 -> ee69825ea [BEAM-1146] Decrease spark runner startup overhead Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/d0fe004d Tree: