[jira] [Resolved] (BEAM-2175) Support state API in Spark batch mode.

2017-11-01 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur resolved BEAM-2175. - Resolution: Implemented > Support state API in Spark batch mode. > --

[jira] [Commented] (BEAM-2175) Support state API in Spark batch mode.

2017-11-01 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234069#comment-16234069 ] Aviem Zur commented on BEAM-2175: - Marked as Resolved > Support state API in Spark batch mode. >

[jira] [Updated] (BEAM-2175) Support state API in Spark batch mode.

2017-11-01 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2175: Fix Version/s: 2.1.0 > Support state API in Spark batch mode. > -- > >

[jira] [Created] (BEAM-2968) Master build broken due to error when running autocomplete_test.py

2017-09-18 Thread Aviem Zur (JIRA)
Aviem Zur created BEAM-2968: --- Summary: Master build broken due to error when running autocomplete_test.py Key: BEAM-2968 URL: https://issues.apache.org/jira/browse/BEAM-2968 Project: Beam Issue

[jira] [Comment Edited] (BEAM-2954) opt-out shade in extension/sql

2017-09-13 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165183#comment-16165183 ] Aviem Zur edited comment on BEAM-2954 at 9/13/17 8:25 PM: -- We cannot opt-out of

[jira] [Updated] (BEAM-2954) opt-out shade in extension/sql

2017-09-13 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2954: Description: {{guava}} is shaded by default in beam modules, while {{calcite}}(a dependency of SQL)

[jira] [Comment Edited] (BEAM-2954) opt-out shade in extension/sql

2017-09-13 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165183#comment-16165183 ] Aviem Zur edited comment on BEAM-2954 at 9/13/17 7:48 PM: -- We cannot opt-out of

[jira] [Commented] (BEAM-2954) opt-out shade in extension/sql

2017-09-13 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165183#comment-16165183 ] Aviem Zur commented on BEAM-2954: - We cannot opt-out of the shading rules in the SQL module as we cannot

[jira] [Created] (BEAM-2847) Stop streaming pipeline when global watermark reaches +INF

2017-09-06 Thread Aviem Zur (JIRA)
Aviem Zur created BEAM-2847: --- Summary: Stop streaming pipeline when global watermark reaches +INF Key: BEAM-2847 URL: https://issues.apache.org/jira/browse/BEAM-2847 Project: Beam Issue Type:

[jira] [Closed] (BEAM-2719) Beam job hangs at Evaluating ParMultiDo when submitted via spark-runner

2017-09-06 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur closed BEAM-2719. --- Resolution: Not A Problem Fix Version/s: Not applicable > Beam job hangs at Evaluating ParMultiDo when

[jira] [Assigned] (BEAM-2719) Beam job hangs at Evaluating ParMultiDo when submitted via spark-runner

2017-09-06 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-2719: --- Assignee: Aviem Zur (was: Jean-Baptiste Onofré) > Beam job hangs at Evaluating ParMultiDo when

[jira] [Commented] (BEAM-2719) Beam job hangs at Evaluating ParMultiDo when submitted via spark-runner

2017-09-06 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16155300#comment-16155300 ] Aviem Zur commented on BEAM-2719: - Problem was identified, the job here was being run on a Spark 2.1.1

[jira] [Assigned] (BEAM-2278) ServiceResourceTransformer missing from Spark runner documentation on shading

2017-09-06 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-2278: --- Assignee: (was: Aviem Zur) > ServiceResourceTransformer missing from Spark runner documentation

[jira] [Commented] (BEAM-1763) TestPipeline should ensure that all assertions succeeded

2017-08-29 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146568#comment-16146568 ] Aviem Zur commented on BEAM-1763: - Currently, {{TestPipeline}} does the validation using the success

[jira] [Updated] (BEAM-2669) Kryo serialization exception when DStreams containing non-Kryo-serializable data are cached

2017-08-29 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2669: Fix Version/s: 2.2.0 > Kryo serialization exception when DStreams containing non-Kryo-serializable > data

[jira] [Updated] (BEAM-2669) Kryo serialization exception when DStreams containing non-Kryo-serializable data are cached

2017-08-29 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2669: Description: Today, when we detect re-use of a dataset in a pipeline in Spark runner we eagerly cache it to

[jira] [Updated] (BEAM-2812) Dropped windows counters / log prints no longer working

2017-08-28 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2812: Description: In https://github.com/apache/beam/pull/2838 aggregators were removed from Spark runner, this

[jira] [Created] (BEAM-2812) Dropped windows counters / log prints no longer working

2017-08-28 Thread Aviem Zur (JIRA)
Aviem Zur created BEAM-2812: --- Summary: Dropped windows counters / log prints no longer working Key: BEAM-2812 URL: https://issues.apache.org/jira/browse/BEAM-2812 Project: Beam Issue Type: Bug

[jira] [Created] (BEAM-2811) Complete the removal of aggregators from Spark runner.

2017-08-28 Thread Aviem Zur (JIRA)
Aviem Zur created BEAM-2811: --- Summary: Complete the removal of aggregators from Spark runner. Key: BEAM-2811 URL: https://issues.apache.org/jira/browse/BEAM-2811 Project: Beam Issue Type: Task

[jira] [Updated] (BEAM-2789) Watermark can become unavailable for executors while it's updated with new values

2017-08-24 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2789: Affects Version/s: (was: 2.0.0) > Watermark can become unavailable for executors while it's updated with

[jira] [Commented] (BEAM-2789) Watermark can become unavailable for executors while it's updated with new values

2017-08-24 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139764#comment-16139764 ] Aviem Zur commented on BEAM-2789: - While this affects the tests greatly since these are in a single JVM, I

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

2017-08-08 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118451#comment-16118451 ] Aviem Zur commented on BEAM-2671: - [~jkff] we've been running Spark runner in streaming mode using 2.0.0

[jira] [Comment Edited] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

2017-08-05 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115336#comment-16115336 ] Aviem Zur edited comment on BEAM-2671 at 8/5/17 8:54 AM: - [~jkff] Please see the

[jira] [Comment Edited] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

2017-08-05 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115339#comment-16115339 ] Aviem Zur edited comment on BEAM-2671 at 8/5/17 8:54 AM: - If we want streaming to

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

2017-08-05 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115339#comment-16115339 ] Aviem Zur commented on BEAM-2671: - If we want streaming to be working in Spark runner in 2.1.0 we should

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

2017-08-05 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115336#comment-16115336 ] Aviem Zur commented on BEAM-2671: - [~jkff] Please see the comments made by JB and myself which help

[jira] [Updated] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

2017-08-05 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2671: Fix Version/s: (was: 2.2.0) 2.1.0 > CreateStreamTest.testFirstElementLate

[jira] [Comment Edited] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

2017-08-02 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106440#comment-16106440 ] Aviem Zur edited comment on BEAM-2671 at 8/2/17 6:44 AM: - CreateStreamTest has

[jira] [Comment Edited] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

2017-07-30 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106440#comment-16106440 ] Aviem Zur edited comment on BEAM-2671 at 7/30/17 11:41 AM: --- CreateStreamTest has

[jira] [Comment Edited] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

2017-07-30 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106440#comment-16106440 ] Aviem Zur edited comment on BEAM-2671 at 7/30/17 11:13 AM: --- CreateStreamTest has

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

2017-07-30 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106440#comment-16106440 ] Aviem Zur commented on BEAM-2671: - 20820fa5477ffcdd4a9ef2e9340353ed3c5691a9 added an element of flakiness

[jira] [Updated] (BEAM-2669) Kryo serialization exception when DStreams containing non-Kryo-serializable data are cached

2017-07-24 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2669: Description: Today, when we detect re-use of a dataset in a pipeline in Spark runner we eagerly cache it to

[jira] [Created] (BEAM-2669) Kryo serialization exception when DStreams containing non-Kryo-serializable data are cached

2017-07-24 Thread Aviem Zur (JIRA)
Aviem Zur created BEAM-2669: --- Summary: Kryo serialization exception when DStreams containing non-Kryo-serializable data are cached Key: BEAM-2669 URL: https://issues.apache.org/jira/browse/BEAM-2669

[jira] [Assigned] (BEAM-2176) Support state API in Spark streaming mode.

2017-07-05 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-2176: --- Assignee: (was: Aviem Zur) > Support state API in Spark streaming mode. >

[jira] [Commented] (BEAM-2359) SparkTimerInternals inputWatermarkTime does not get updated in cluster mode

2017-07-04 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16073258#comment-16073258 ] Aviem Zur commented on BEAM-2359: - Yeah, the PR was merged, marking as resolved. > SparkTimerInternals

[jira] [Resolved] (BEAM-2359) SparkTimerInternals inputWatermarkTime does not get updated in cluster mode

2017-07-04 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur resolved BEAM-2359. - Resolution: Fixed > SparkTimerInternals inputWatermarkTime does not get updated in cluster mode >

[jira] [Assigned] (BEAM-2359) SparkTimerInternals inputWatermarkTime does not get updated in cluster mode

2017-06-19 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-2359: --- Assignee: Aviem Zur (was: Amit Sela) > SparkTimerInternals inputWatermarkTime does not get updated

[jira] [Assigned] (BEAM-2409) Spark runner produces exactly twice the number of results in streaming mode when use triggers to re-window results on global window.

2017-06-12 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-2409: --- Assignee: Amit Sela (was: Aviem Zur) > Spark runner produces exactly twice the number of results in

[jira] [Updated] (BEAM-2359) SparkTimerInternals inputWatermarkTime does not get updated in cluster mode

2017-06-11 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2359: Description: {{SparkTimerInternals#inputWatermarkTime}} does not get updated in cluster mode. This causes

[jira] [Updated] (BEAM-2359) SparkTimerInternals inputWatermarkTime does not get updated in cluster mode

2017-06-11 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2359: Description: {{SparkTimerInternals#inputWatermarkTime}} does not get updated in cluster mode. This causes

[jira] [Created] (BEAM-2359) SparkTimerInternals inputWatermarkTime does not get updated in cluster mode

2017-05-24 Thread Aviem Zur (JIRA)
Aviem Zur created BEAM-2359: --- Summary: SparkTimerInternals inputWatermarkTime does not get updated in cluster mode Key: BEAM-2359 URL: https://issues.apache.org/jira/browse/BEAM-2359 Project: Beam

[jira] [Updated] (BEAM-2277) IllegalArgumentException when using Hadoop file system for WordCount example.

2017-05-12 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2277: Description: IllegalArgumentException when using Hadoop file system for WordCount example. Occurred when

[jira] [Assigned] (BEAM-2277) IllegalArgumentException when using Hadoop file system for WordCount example.

2017-05-12 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-2277: --- Assignee: Aviem Zur (was: Davor Bonaci) > IllegalArgumentException when using Hadoop file system for

[jira] [Updated] (BEAM-2279) Hadoop file system support should be included in examples/archetype profiles of Spark runner.

2017-05-12 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2279: Description: Hadoop file system support (dependency on {{beam-sdks-java-io-hadoop-file-system}}) should be

[jira] [Updated] (BEAM-2279) Hadoop file system support should be included in examples/archetype profiles of Spark runner.

2017-05-12 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2279: Summary: Hadoop file system support should be included in examples/archetype profiles of Spark runner.

[jira] [Updated] (BEAM-2277) IllegalArgumentException when using Hadoop file system for WordCount example.

2017-05-12 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2277: Description: IllegalArgumentException when using Hadoop file system for WordCount example. Occurred when

[jira] [Updated] (BEAM-2277) IllegalArgumentException when using Hadoop file system for WordCount example.

2017-05-12 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2277: Description: IllegalArgumentException when using Hadoop file system for WordCount example. Occurred when

[jira] [Created] (BEAM-2278) ServiceResourceTransformer missing from Spark runner documentation on shading

2017-05-12 Thread Aviem Zur (JIRA)
Aviem Zur created BEAM-2278: --- Summary: ServiceResourceTransformer missing from Spark runner documentation on shading Key: BEAM-2278 URL: https://issues.apache.org/jira/browse/BEAM-2278 Project: Beam

[jira] [Created] (BEAM-2277) IllegalArgumentException when using Hadoop file system for WordCount example.

2017-05-12 Thread Aviem Zur (JIRA)
Aviem Zur created BEAM-2277: --- Summary: IllegalArgumentException when using Hadoop file system for WordCount example. Key: BEAM-2277 URL: https://issues.apache.org/jira/browse/BEAM-2277 Project: Beam

[jira] [Assigned] (BEAM-1482) Support SetState in Spark runner

2017-05-10 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-1482: --- Assignee: (was: Aviem Zur) > Support SetState in Spark runner >

[jira] [Resolved] (BEAM-1482) Support SetState in Spark runner

2017-05-10 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur resolved BEAM-1482. - Resolution: Duplicate Fix Version/s: Not applicable > Support SetState in Spark runner >

[jira] [Assigned] (BEAM-1477) Support MapState in Spark runner

2017-05-10 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-1477: --- Assignee: (was: Aviem Zur) > Support MapState in Spark runner >

[jira] [Resolved] (BEAM-1477) Support MapState in Spark runner

2017-05-10 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur resolved BEAM-1477. - Resolution: Duplicate Fix Version/s: Not applicable > Support MapState in Spark runner >

[jira] [Assigned] (BEAM-2111) ClassCastException in streaming Spark pipeline.

2017-05-10 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-2111: --- Assignee: (was: Aviem Zur) > ClassCastException in streaming Spark pipeline. >

[jira] [Commented] (BEAM-981) Not possible to directly submit a pipeline on spark cluster

2017-05-10 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004990#comment-16004990 ] Aviem Zur commented on BEAM-981: Moving this to version 2.1.0 as it is not a blocker for 2.0.0 and can be

[jira] [Updated] (BEAM-981) Not possible to directly submit a pipeline on spark cluster

2017-05-10 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-981: --- Fix Version/s: (was: 2.0.0) 2.1.0 > Not possible to directly submit a pipeline on spark

[jira] [Comment Edited] (BEAM-1672) Accumulable MetricsContainers.

2017-05-08 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16001125#comment-16001125 ] Aviem Zur edited comment on BEAM-1672 at 5/8/17 5:16 PM: - Has not been

[jira] [Commented] (BEAM-1672) Accumulable MetricsContainers.

2017-05-08 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16001125#comment-16001125 ] Aviem Zur commented on BEAM-1672: - Has not been cherry-picked to the release branch yet, so not marked as

[jira] [Updated] (BEAM-1672) Accumulable MetricsContainers.

2017-05-06 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-1672: Fix Version/s: 2.0.0 > Accumulable MetricsContainers. > -- > >

[jira] [Assigned] (BEAM-1035) Support for new State API in SparkRunner

2017-05-05 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-1035: --- Assignee: (was: Aviem Zur) > Support for new State API in SparkRunner >

[jira] [Created] (BEAM-2175) Support state API in Spark batch mode.

2017-05-05 Thread Aviem Zur (JIRA)
Aviem Zur created BEAM-2175: --- Summary: Support state API in Spark batch mode. Key: BEAM-2175 URL: https://issues.apache.org/jira/browse/BEAM-2175 Project: Beam Issue Type: Sub-task

[jira] [Assigned] (BEAM-2176) Support state API in Spark streaming mode.

2017-05-05 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-2176: --- Assignee: Aviem Zur (was: Amit Sela) > Support state API in Spark streaming mode. >

[jira] [Assigned] (BEAM-2175) Support state API in Spark batch mode.

2017-05-05 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-2175: --- Assignee: Jingsong Lee (was: Amit Sela) > Support state API in Spark batch mode. >

[jira] [Created] (BEAM-2176) Support state API in Spark streaming mode.

2017-05-05 Thread Aviem Zur (JIRA)
Aviem Zur created BEAM-2176: --- Summary: Support state API in Spark streaming mode. Key: BEAM-2176 URL: https://issues.apache.org/jira/browse/BEAM-2176 Project: Beam Issue Type: Sub-task

[jira] [Comment Edited] (BEAM-1582) ResumeFromCheckpointStreamingTest flakes with what appears as a second firing.

2017-05-04 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15997742#comment-15997742 ] Aviem Zur edited comment on BEAM-1582 at 5/5/17 3:03 AM: - I don't think we should

[jira] [Assigned] (BEAM-1582) ResumeFromCheckpointStreamingTest flakes with what appears as a second firing.

2017-05-04 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-1582: --- Assignee: (was: Amit Sela) > ResumeFromCheckpointStreamingTest flakes with what appears as a

[jira] [Updated] (BEAM-1582) ResumeFromCheckpointStreamingTest flakes with what appears as a second firing.

2017-05-04 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-1582: Fix Version/s: (was: First stable release) > ResumeFromCheckpointStreamingTest flakes with what appears

[jira] [Comment Edited] (BEAM-1582) ResumeFromCheckpointStreamingTest flakes with what appears as a second firing.

2017-05-04 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15997742#comment-15997742 ] Aviem Zur edited comment on BEAM-1582 at 5/5/17 3:02 AM: - I don't think we should

[jira] [Updated] (BEAM-1582) ResumeFromCheckpointStreamingTest flakes with what appears as a second firing.

2017-05-04 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-1582: Priority: Minor (was: Major) > ResumeFromCheckpointStreamingTest flakes with what appears as a second

[jira] [Comment Edited] (BEAM-1582) ResumeFromCheckpointStreamingTest flakes with what appears as a second firing.

2017-05-04 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15997742#comment-15997742 ] Aviem Zur edited comment on BEAM-1582 at 5/5/17 3:01 AM: - I don't think we should

[jira] [Comment Edited] (BEAM-1582) ResumeFromCheckpointStreamingTest flakes with what appears as a second firing.

2017-05-04 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15997742#comment-15997742 ] Aviem Zur edited comment on BEAM-1582 at 5/5/17 3:02 AM: - I don't think we should

[jira] [Commented] (BEAM-1582) ResumeFromCheckpointStreamingTest flakes with what appears as a second firing.

2017-05-04 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15997742#comment-15997742 ] Aviem Zur commented on BEAM-1582: - I don't think we should disable this test as it is a very important one.

[jira] [Resolved] (BEAM-2002) Verify PAssert execution in TestSparkRunner

2017-05-04 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur resolved BEAM-2002. - Resolution: Done > Verify PAssert execution in TestSparkRunner >

[jira] [Resolved] (BEAM-1726) Verify PAssert execution in TestFlinkRunner

2017-05-04 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur resolved BEAM-1726. - Resolution: Done > Verify PAssert execution in TestFlinkRunner >

[jira] [Resolved] (BEAM-1763) TestPipeline should ensure that all assertions succeeded

2017-05-04 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur resolved BEAM-1763. - Resolution: Implemented > TestPipeline should ensure that all assertions succeeded >

[jira] [Updated] (BEAM-1763) TestPipeline should ensure that all assertions succeeded

2017-05-04 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-1763: Fix Version/s: First stable release > TestPipeline should ensure that all assertions succeeded >

[jira] [Updated] (BEAM-2111) ClassCastException in streaming Spark pipeline.

2017-05-02 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2111: Summary: ClassCastException in streaming Spark pipeline. (was: java.lang.ClassCastException in spark in

[jira] [Updated] (BEAM-2139) Disable SplittableDoFn ValidatesRunner tests for Streaming Flink Runner

2017-05-02 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2139: Issue Type: Task (was: Bug) > Disable SplittableDoFn ValidatesRunner tests for Streaming Flink Runner >

[jira] [Updated] (BEAM-2139) Disable SplittableDoFn ValidatesRunner tests for Streaming Flink Runner

2017-05-02 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2139: Summary: Disable SplittableDoFn ValidatesRunner tests for Streaming Flink Runner (was: Disable

[jira] [Assigned] (BEAM-2139) Disable SplittableDoFn ValidatesRunner tests for Flink Runner

2017-05-02 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-2139: --- Assignee: Aviem Zur (was: Aljoscha Krettek) > Disable SplittableDoFn ValidatesRunner tests for Flink

[jira] [Resolved] (BEAM-2129) KafkaIOTest.testUnboundedSourceMetrics is flaky

2017-05-01 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur resolved BEAM-2129. - Resolution: Fixed Fix Version/s: Not applicable > KafkaIOTest.testUnboundedSourceMetrics is flaky >

[jira] [Assigned] (BEAM-2127) Test Spark streaming pipelines with maxRecordsPerBatch

2017-04-30 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-2127: --- Assignee: (was: Amit Sela) > Test Spark streaming pipelines with maxRecordsPerBatch >

[jira] [Updated] (BEAM-2127) Test Spark streaming pipelines with maxRecordsPerBatch

2017-04-30 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2127: Priority: Minor (was: Major) > Test Spark streaming pipelines with maxRecordsPerBatch >

[jira] [Created] (BEAM-2127) Test Spark streaming pipelines with maxRecordsPerBatch

2017-04-30 Thread Aviem Zur (JIRA)
Aviem Zur created BEAM-2127: --- Summary: Test Spark streaming pipelines with maxRecordsPerBatch Key: BEAM-2127 URL: https://issues.apache.org/jira/browse/BEAM-2127 Project: Beam Issue Type: Test

[jira] [Resolved] (BEAM-2057) Test metrics are reported to Spark Metrics sink.

2017-04-29 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur resolved BEAM-2057. - Resolution: Done > Test metrics are reported to Spark Metrics sink. >

[jira] [Resolved] (BEAM-1398) KafkaIO metrics

2017-04-29 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur resolved BEAM-1398. - Resolution: Implemented > KafkaIO metrics > --- > > Key: BEAM-1398 >

[jira] [Updated] (BEAM-1398) KafkaIO metrics

2017-04-29 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-1398: Fix Version/s: First stable release > KafkaIO metrics > --- > > Key: BEAM-1398 >

[jira] [Updated] (BEAM-2057) Test metrics are reported to Spark Metrics sink.

2017-04-29 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2057: Fix Version/s: First stable release > Test metrics are reported to Spark Metrics sink. >

[jira] [Assigned] (BEAM-2111) java.lang.ClassCastException in spark in streaming mode

2017-04-29 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-2111: --- Assignee: Aviem Zur (was: Amit Sela) > java.lang.ClassCastException in spark in streaming mode >

[jira] [Resolved] (BEAM-1814) Support for new State API in Spark runner

2017-04-29 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur resolved BEAM-1814. - Resolution: Duplicate Fix Version/s: Not applicable > Support for new State API in Spark runner >

[jira] [Updated] (BEAM-1148) Port PAssert away from Aggregators

2017-04-29 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-1148: Issue Type: Improvement (was: New Feature) > Port PAssert away from Aggregators >

[jira] [Resolved] (BEAM-1148) Port PAssert away from Aggregators

2017-04-29 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur resolved BEAM-1148. - Resolution: Done > Port PAssert away from Aggregators > -- > >

[jira] [Assigned] (BEAM-1148) Port PAssert away from Aggregators

2017-04-29 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-1148: --- Assignee: Pablo Estrada > Port PAssert away from Aggregators > -- > >

[jira] [Updated] (BEAM-1148) Port PAssert away from Aggregators

2017-04-29 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-1148: Fix Version/s: First stable release > Port PAssert away from Aggregators >

[jira] [Resolved] (BEAM-2106) NotSerializableException thrown when serializing EvaluationContext

2017-04-27 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur resolved BEAM-2106. - Resolution: Fixed > NotSerializableException thrown when serializing EvaluationContext >

[jira] [Updated] (BEAM-2029) NullPointerException when using multi output ParDo in Spark runner in streaming mode.

2017-04-27 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2029: Fix Version/s: First stable release > NullPointerException when using multi output ParDo in Spark runner in

[jira] [Updated] (BEAM-2029) NullPointerException when using multi output ParDo in Spark runner in streaming mode.

2017-04-27 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2029: Summary: NullPointerException when using multi output ParDo in Spark runner in streaming mode. (was:

[jira] [Updated] (BEAM-2001) Verify PAssert execution in all runners

2017-04-27 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2001: Fix Version/s: (was: First stable release) > Verify PAssert execution in all runners >

[jira] [Commented] (BEAM-2004) Verify PAssert execution in TestApexRunner

2017-04-26 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985987#comment-15985987 ] Aviem Zur commented on BEAM-2004: - Adding this back to First stable release. [~thw] is there anything we

[jira] [Updated] (BEAM-2004) Verify PAssert execution in TestApexRunner

2017-04-26 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2004: Fix Version/s: First stable release > Verify PAssert execution in TestApexRunner >

  1   2   3   4   >