[jira] [Commented] (BEAM-2789) Watermark can become unavailable for executors while it's updated with new values

[ https://issues.apache.org/jira/browse/BEAM-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16139764#comment-16139764 ] Aviem Zur commented on BEAM-2789: - While this affects the tests greatly since these are in

[jira] [Updated] (BEAM-2789) Watermark can become unavailable for executors while it's updated with new values

[ https://issues.apache.org/jira/browse/BEAM-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2789: Affects Version/s: (was: 2.0.0) > Watermark can become unavailable for executors while it's updated with

[jira] [Created] (BEAM-2811) Complete the removal of aggregators from Spark runner.

Aviem Zur created BEAM-2811: --- Summary: Complete the removal of aggregators from Spark runner. Key: BEAM-2811 URL: https://issues.apache.org/jira/browse/BEAM-2811 Project: Beam Issue Type: Task

[jira] [Created] (BEAM-2812) Dropped windows counters / log prints no longer working

Aviem Zur created BEAM-2812: --- Summary: Dropped windows counters / log prints no longer working Key: BEAM-2812 URL: https://issues.apache.org/jira/browse/BEAM-2812 Project: Beam Issue Type: Bug

[jira] [Updated] (BEAM-2812) Dropped windows counters / log prints no longer working

[ https://issues.apache.org/jira/browse/BEAM-2812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2812: Description: In https://github.com/apache/beam/pull/2838 aggregators were removed from Spark runner, this ca

[jira] [Updated] (BEAM-2669) Kryo serialization exception when DStreams containing non-Kryo-serializable data are cached

[ https://issues.apache.org/jira/browse/BEAM-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2669: Description: Today, when we detect re-use of a dataset in a pipeline in Spark runner we eagerly cache it to

[jira] [Updated] (BEAM-2669) Kryo serialization exception when DStreams containing non-Kryo-serializable data are cached

[ https://issues.apache.org/jira/browse/BEAM-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2669: Fix Version/s: 2.2.0 > Kryo serialization exception when DStreams containing non-Kryo-serializable > data ar

[jira] [Commented] (BEAM-1763) TestPipeline should ensure that all assertions succeeded

[ https://issues.apache.org/jira/browse/BEAM-1763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16146568#comment-16146568 ] Aviem Zur commented on BEAM-1763: - Currently, {{TestPipeline}} does the validation using th

[jira] [Assigned] (BEAM-2278) ServiceResourceTransformer missing from Spark runner documentation on shading

[ https://issues.apache.org/jira/browse/BEAM-2278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-2278: --- Assignee: (was: Aviem Zur) > ServiceResourceTransformer missing from Spark runner documentation on

[jira] [Commented] (BEAM-2719) Beam job hangs at Evaluating ParMultiDo when submitted via spark-runner

[ https://issues.apache.org/jira/browse/BEAM-2719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155300#comment-16155300 ] Aviem Zur commented on BEAM-2719: - Problem was identified, the job here was being run on a

[jira] [Assigned] (BEAM-2719) Beam job hangs at Evaluating ParMultiDo when submitted via spark-runner

[ https://issues.apache.org/jira/browse/BEAM-2719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-2719: --- Assignee: Aviem Zur (was: Jean-Baptiste Onofré) > Beam job hangs at Evaluating ParMultiDo when submit

[jira] [Closed] (BEAM-2719) Beam job hangs at Evaluating ParMultiDo when submitted via spark-runner

[ https://issues.apache.org/jira/browse/BEAM-2719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur closed BEAM-2719. --- Resolution: Not A Problem Fix Version/s: Not applicable > Beam job hangs at Evaluating ParMultiDo when s

[jira] [Created] (BEAM-2847) Stop streaming pipeline when global watermark reaches +INF

Aviem Zur created BEAM-2847: --- Summary: Stop streaming pipeline when global watermark reaches +INF Key: BEAM-2847 URL: https://issues.apache.org/jira/browse/BEAM-2847 Project: Beam Issue Type: Imp

[jira] [Updated] (BEAM-2029) NullPointerException when using multi output ParDo in Spark runner in streaming mode.

[ https://issues.apache.org/jira/browse/BEAM-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2029: Summary: NullPointerException when using multi output ParDo in Spark runner in streaming mode. (was: NullPoi

[jira] [Updated] (BEAM-2029) NullPointerException when using multi output ParDo in Spark runner in streaming mode.

[ https://issues.apache.org/jira/browse/BEAM-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2029: Fix Version/s: First stable release > NullPointerException when using multi output ParDo in Spark runner in

[jira] [Created] (BEAM-2106) NotSerializableException thrown when serializing EvaluationContext

Aviem Zur created BEAM-2106: --- Summary: NotSerializableException thrown when serializing EvaluationContext Key: BEAM-2106 URL: https://issues.apache.org/jira/browse/BEAM-2106 Project: Beam Issue Ty

[jira] [Resolved] (BEAM-2106) NotSerializableException thrown when serializing EvaluationContext

[ https://issues.apache.org/jira/browse/BEAM-2106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur resolved BEAM-2106. - Resolution: Fixed > NotSerializableException thrown when serializing EvaluationContext > --

[jira] [Updated] (BEAM-1148) Port PAssert away from Aggregators

[ https://issues.apache.org/jira/browse/BEAM-1148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-1148: Fix Version/s: First stable release > Port PAssert away from Aggregators > --

[jira] [Assigned] (BEAM-1148) Port PAssert away from Aggregators

[ https://issues.apache.org/jira/browse/BEAM-1148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-1148: --- Assignee: Pablo Estrada > Port PAssert away from Aggregators > -- > >

[jira] [Resolved] (BEAM-1148) Port PAssert away from Aggregators

[ https://issues.apache.org/jira/browse/BEAM-1148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur resolved BEAM-1148. - Resolution: Done > Port PAssert away from Aggregators > -- > >

[jira] [Updated] (BEAM-1148) Port PAssert away from Aggregators

[ https://issues.apache.org/jira/browse/BEAM-1148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-1148: Issue Type: Improvement (was: New Feature) > Port PAssert away from Aggregators > --

[jira] [Resolved] (BEAM-1814) Support for new State API in Spark runner

[ https://issues.apache.org/jira/browse/BEAM-1814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur resolved BEAM-1814. - Resolution: Duplicate Fix Version/s: Not applicable > Support for new State API in Spark runner > --

[jira] [Assigned] (BEAM-2111) java.lang.ClassCastException in spark in streaming mode

[ https://issues.apache.org/jira/browse/BEAM-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-2111: --- Assignee: Aviem Zur (was: Amit Sela) > java.lang.ClassCastException in spark in streaming mode >

[jira] [Updated] (BEAM-1398) KafkaIO metrics

[ https://issues.apache.org/jira/browse/BEAM-1398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-1398: Fix Version/s: First stable release > KafkaIO metrics > --- > > Key: BEAM-1398 >

[jira] [Resolved] (BEAM-2057) Test metrics are reported to Spark Metrics sink.

[ https://issues.apache.org/jira/browse/BEAM-2057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur resolved BEAM-2057. - Resolution: Done > Test metrics are reported to Spark Metrics sink. > -

[jira] [Updated] (BEAM-2057) Test metrics are reported to Spark Metrics sink.

[ https://issues.apache.org/jira/browse/BEAM-2057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2057: Fix Version/s: First stable release > Test metrics are reported to Spark Metrics sink. >

[jira] [Resolved] (BEAM-1398) KafkaIO metrics

[ https://issues.apache.org/jira/browse/BEAM-1398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur resolved BEAM-1398. - Resolution: Implemented > KafkaIO metrics > --- > > Key: BEAM-1398 >

[jira] [Created] (BEAM-2127) Test Spark streaming pipelines with maxRecordsPerBatch

Aviem Zur created BEAM-2127: --- Summary: Test Spark streaming pipelines with maxRecordsPerBatch Key: BEAM-2127 URL: https://issues.apache.org/jira/browse/BEAM-2127 Project: Beam Issue Type: Test

[jira] [Updated] (BEAM-2127) Test Spark streaming pipelines with maxRecordsPerBatch

[ https://issues.apache.org/jira/browse/BEAM-2127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2127: Priority: Minor (was: Major) > Test Spark streaming pipelines with maxRecordsPerBatch >

[jira] [Assigned] (BEAM-2127) Test Spark streaming pipelines with maxRecordsPerBatch

[ https://issues.apache.org/jira/browse/BEAM-2127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-2127: --- Assignee: (was: Amit Sela) > Test Spark streaming pipelines with maxRecordsPerBatch >

[jira] [Resolved] (BEAM-2129) KafkaIOTest.testUnboundedSourceMetrics is flaky

[ https://issues.apache.org/jira/browse/BEAM-2129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur resolved BEAM-2129. - Resolution: Fixed Fix Version/s: Not applicable > KafkaIOTest.testUnboundedSourceMetrics is flaky >

[jira] [Commented] (BEAM-2111) java.lang.ClassCastException in spark in streaming mode

[ https://issues.apache.org/jira/browse/BEAM-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15991017#comment-15991017 ] Aviem Zur commented on BEAM-2111: - I was able to create a test to reproduce it, however it

[jira] [Assigned] (BEAM-2139) Disable SplittableDoFn ValidatesRunner tests for Flink Runner

[ https://issues.apache.org/jira/browse/BEAM-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-2139: --- Assignee: Aviem Zur (was: Aljoscha Krettek) > Disable SplittableDoFn ValidatesRunner tests for Flink

[jira] [Updated] (BEAM-2139) Disable SplittableDoFn ValidatesRunner tests for Streaming Flink Runner

[ https://issues.apache.org/jira/browse/BEAM-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2139: Summary: Disable SplittableDoFn ValidatesRunner tests for Streaming Flink Runner (was: Disable SplittableDoF

[jira] [Updated] (BEAM-2139) Disable SplittableDoFn ValidatesRunner tests for Streaming Flink Runner

[ https://issues.apache.org/jira/browse/BEAM-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2139: Issue Type: Task (was: Bug) > Disable SplittableDoFn ValidatesRunner tests for Streaming Flink Runner >

[jira] [Updated] (BEAM-2111) ClassCastException in streaming Spark pipeline.

[ https://issues.apache.org/jira/browse/BEAM-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2111: Summary: ClassCastException in streaming Spark pipeline. (was: java.lang.ClassCastException in spark in stre

[jira] [Updated] (BEAM-1763) TestPipeline should ensure that all assertions succeeded

[ https://issues.apache.org/jira/browse/BEAM-1763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-1763: Fix Version/s: First stable release > TestPipeline should ensure that all assertions succeeded >

[jira] [Resolved] (BEAM-1763) TestPipeline should ensure that all assertions succeeded

[ https://issues.apache.org/jira/browse/BEAM-1763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur resolved BEAM-1763. - Resolution: Implemented > TestPipeline should ensure that all assertions succeeded > --

[jira] [Resolved] (BEAM-1726) Verify PAssert execution in TestFlinkRunner

[ https://issues.apache.org/jira/browse/BEAM-1726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur resolved BEAM-1726. - Resolution: Done > Verify PAssert execution in TestFlinkRunner > --

[jira] [Resolved] (BEAM-2002) Verify PAssert execution in TestSparkRunner

[ https://issues.apache.org/jira/browse/BEAM-2002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur resolved BEAM-2002. - Resolution: Done > Verify PAssert execution in TestSparkRunner > --

[jira] [Commented] (BEAM-1582) ResumeFromCheckpointStreamingTest flakes with what appears as a second firing.

[ https://issues.apache.org/jira/browse/BEAM-1582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15997742#comment-15997742 ] Aviem Zur commented on BEAM-1582: - I don't think we should disable this test as it is a ver

[jira] [Comment Edited] (BEAM-1582) ResumeFromCheckpointStreamingTest flakes with what appears as a second firing.

[ https://issues.apache.org/jira/browse/BEAM-1582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15997742#comment-15997742 ] Aviem Zur edited comment on BEAM-1582 at 5/5/17 3:02 AM: - I don't t

[jira] [Comment Edited] (BEAM-1582) ResumeFromCheckpointStreamingTest flakes with what appears as a second firing.

[ https://issues.apache.org/jira/browse/BEAM-1582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15997742#comment-15997742 ] Aviem Zur edited comment on BEAM-1582 at 5/5/17 3:01 AM: - I don't t

[jira] [Updated] (BEAM-1582) ResumeFromCheckpointStreamingTest flakes with what appears as a second firing.

[ https://issues.apache.org/jira/browse/BEAM-1582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-1582: Priority: Minor (was: Major) > ResumeFromCheckpointStreamingTest flakes with what appears as a second firing

[jira] [Comment Edited] (BEAM-1582) ResumeFromCheckpointStreamingTest flakes with what appears as a second firing.

[ https://issues.apache.org/jira/browse/BEAM-1582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15997742#comment-15997742 ] Aviem Zur edited comment on BEAM-1582 at 5/5/17 3:02 AM: - I don't t

[jira] [Updated] (BEAM-1582) ResumeFromCheckpointStreamingTest flakes with what appears as a second firing.

[ https://issues.apache.org/jira/browse/BEAM-1582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-1582: Fix Version/s: (was: First stable release) > ResumeFromCheckpointStreamingTest flakes with what appears a

[jira] [Assigned] (BEAM-1582) ResumeFromCheckpointStreamingTest flakes with what appears as a second firing.

[ https://issues.apache.org/jira/browse/BEAM-1582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-1582: --- Assignee: (was: Amit Sela) > ResumeFromCheckpointStreamingTest flakes with what appears as a secon

[jira] [Comment Edited] (BEAM-1582) ResumeFromCheckpointStreamingTest flakes with what appears as a second firing.

[ https://issues.apache.org/jira/browse/BEAM-1582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15997742#comment-15997742 ] Aviem Zur edited comment on BEAM-1582 at 5/5/17 3:03 AM: - I don't t

[jira] [Assigned] (BEAM-2175) Support state API in Spark batch mode.

[ https://issues.apache.org/jira/browse/BEAM-2175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-2175: --- Assignee: Jingsong Lee (was: Amit Sela) > Support state API in Spark batch mode. > --

[jira] [Created] (BEAM-2176) Support state API in Spark streaming mode.

Aviem Zur created BEAM-2176: --- Summary: Support state API in Spark streaming mode. Key: BEAM-2176 URL: https://issues.apache.org/jira/browse/BEAM-2176 Project: Beam Issue Type: Sub-task Co

[jira] [Created] (BEAM-2175) Support state API in Spark batch mode.

Aviem Zur created BEAM-2175: --- Summary: Support state API in Spark batch mode. Key: BEAM-2175 URL: https://issues.apache.org/jira/browse/BEAM-2175 Project: Beam Issue Type: Sub-task Compon

[jira] [Assigned] (BEAM-2176) Support state API in Spark streaming mode.

[ https://issues.apache.org/jira/browse/BEAM-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-2176: --- Assignee: Aviem Zur (was: Amit Sela) > Support state API in Spark streaming mode. > -

[jira] [Assigned] (BEAM-1035) Support for new State API in SparkRunner

[ https://issues.apache.org/jira/browse/BEAM-1035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-1035: --- Assignee: (was: Aviem Zur) > Support for new State API in SparkRunner > --

[jira] [Updated] (BEAM-1672) Accumulable MetricsContainers.

[ https://issues.apache.org/jira/browse/BEAM-1672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-1672: Fix Version/s: 2.0.0 > Accumulable MetricsContainers. > -- > > Ke

[jira] [Commented] (BEAM-1672) Accumulable MetricsContainers.

[ https://issues.apache.org/jira/browse/BEAM-1672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001125#comment-16001125 ] Aviem Zur commented on BEAM-1672: - Has not been cherry-picked to the release branch yet, so

[jira] [Comment Edited] (BEAM-1672) Accumulable MetricsContainers.

[ https://issues.apache.org/jira/browse/BEAM-1672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001125#comment-16001125 ] Aviem Zur edited comment on BEAM-1672 at 5/8/17 5:16 PM: - Has not b

[jira] [Created] (BEAM-2669) Kryo serialization exception when DStreams containing non-Kryo-serializable data are cached

Aviem Zur created BEAM-2669: --- Summary: Kryo serialization exception when DStreams containing non-Kryo-serializable data are cached Key: BEAM-2669 URL: https://issues.apache.org/jira/browse/BEAM-2669 Project

[jira] [Updated] (BEAM-2669) Kryo serialization exception when DStreams containing non-Kryo-serializable data are cached

[ https://issues.apache.org/jira/browse/BEAM-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2669: Description: Today, when we detect re-use of a dataset in a pipeline in Spark runner we eagerly cache it to

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[ https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16106440#comment-16106440 ] Aviem Zur commented on BEAM-2671: - 20820fa5477ffcdd4a9ef2e9340353ed3c5691a9 added an elemen

[jira] [Comment Edited] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[ https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16106440#comment-16106440 ] Aviem Zur edited comment on BEAM-2671 at 7/30/17 11:13 AM: --- Creat

[jira] [Comment Edited] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[ https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16106440#comment-16106440 ] Aviem Zur edited comment on BEAM-2671 at 7/30/17 11:41 AM: --- Creat

[jira] [Comment Edited] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[ https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16106440#comment-16106440 ] Aviem Zur edited comment on BEAM-2671 at 8/2/17 6:44 AM: - CreateStr

[jira] [Updated] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[ https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2671: Fix Version/s: (was: 2.2.0) 2.1.0 > CreateStreamTest.testFirstElementLate validatesRun

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[ https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16115336#comment-16115336 ] Aviem Zur commented on BEAM-2671: - [~jkff] Please see the comments made by JB and myself wh

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[ https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16115339#comment-16115339 ] Aviem Zur commented on BEAM-2671: - If we want streaming to be working in Spark runner in 2.

[jira] [Comment Edited] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[ https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16115339#comment-16115339 ] Aviem Zur edited comment on BEAM-2671 at 8/5/17 8:54 AM: - If we wan

[jira] [Comment Edited] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

[ https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16115336#comment-16115336 ] Aviem Zur edited comment on BEAM-2671 at 8/5/17 8:54 AM: - [~jkff] P

[jira] [Commented] (BEAM-1146) Decrease spark runner startup overhead

[ https://issues.apache.org/jira/browse/BEAM-1146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15769796#comment-15769796 ] Aviem Zur commented on BEAM-1146: - Using the {{@Bind}} annotation would be more elegant tha

[jira] [Issue Comment Deleted] (BEAM-1146) Decrease spark runner startup overhead

[ https://issues.apache.org/jira/browse/BEAM-1146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-1146: Comment: was deleted (was: Using the {{@Bind}} annotation would be more elegant than the wrapper classes, th

[jira] [Updated] (BEAM-147) Introduce an easy API for pipeline metrics

[ https://issues.apache.org/jira/browse/BEAM-147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-147: --- Description: The existing Aggregators are confusing both because of their name and because they serve multiple

[jira] [Assigned] (BEAM-774) Implement Metrics support for Spark runner

[ https://issues.apache.org/jira/browse/BEAM-774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-774: -- Assignee: Aviem Zur (was: Amit Sela) > Implement Metrics support for Spark runner >

[jira] [Resolved] (BEAM-1155) Spark runner aggregators only support a handfuls of combiners

[ https://issues.apache.org/jira/browse/BEAM-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur resolved BEAM-1155. - Resolution: Won't Fix Fix Version/s: Not applicable > Spark runner aggregators only support a handfu

[jira] [Created] (BEAM-1334) Split UsesMetrics category and tests into UsesCommittedMetrics and UsesAttemptedMetrics

Aviem Zur created BEAM-1334: --- Summary: Split UsesMetrics category and tests into UsesCommittedMetrics and UsesAttemptedMetrics Key: BEAM-1334 URL: https://issues.apache.org/jira/browse/BEAM-1334 Project: Be

[jira] [Updated] (BEAM-1334) Split UsesMetrics category and tests into UsesCommittedMetrics and UsesAttemptedMetrics

[ https://issues.apache.org/jira/browse/BEAM-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-1334: Assignee: Ben Chambers (was: Davor Bonaci) > Split UsesMetrics category and tests into UsesCommittedMetrics

[jira] [Assigned] (BEAM-1334) Split UsesMetrics category and tests into UsesCommittedMetrics and UsesAttemptedMetrics

[ https://issues.apache.org/jira/browse/BEAM-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-1334: --- Assignee: Aviem Zur (was: Ben Chambers) > Split UsesMetrics category and tests into UsesCommittedMetr

[jira] (BEAM-1344) Uniform metrics step name semantics

Title: Message Title Aviem Zur assigned an issue to Ben Chambers

[jira] (BEAM-1344) Uniform metrics step name semantics

Title: Message Title Aviem Zur created an issue

[jira] (BEAM-1354) TextIO should comply with PTransform style guide

Title: Message Title Aviem Zur updated an issue

[jira] [Commented] (BEAM-1092) Shade commonly used libraries (e.g. Guava) to avoid class conflicts

[ https://issues.apache.org/jira/browse/BEAM-1092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15848318#comment-15848318 ] Aviem Zur commented on BEAM-1092: - [~davor] I agree that a general solution requires some t

[jira] [Created] (BEAM-1379) Shade Guava in beam-sdks-java-io-kafka module

Aviem Zur created BEAM-1379: --- Summary: Shade Guava in beam-sdks-java-io-kafka module Key: BEAM-1379 URL: https://issues.apache.org/jira/browse/BEAM-1379 Project: Beam Issue Type: Task Com

[jira] [Commented] (BEAM-1379) Shade Guava in beam-sdks-java-io-kafka module

[ https://issues.apache.org/jira/browse/BEAM-1379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15850030#comment-15850030 ] Aviem Zur commented on BEAM-1379: - Kafka IO is commonly used in clusters which may have a d

[jira] [Commented] (BEAM-1092) Shade commonly used libraries (e.g. Guava) to avoid class conflicts

[ https://issues.apache.org/jira/browse/BEAM-1092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15850033#comment-15850033 ] Aviem Zur commented on BEAM-1092: - [~dhalp...@google.com] Sure. See: https://issues.apache.

[jira] [Created] (BEAM-1392) DoFn teardown not called on empty partitions

Aviem Zur created BEAM-1392: --- Summary: DoFn teardown not called on empty partitions Key: BEAM-1392 URL: https://issues.apache.org/jira/browse/BEAM-1392 Project: Beam Issue Type: Bug Compo

[jira] [Assigned] (BEAM-1354) TextIO should comply with PTransform style guide

[ https://issues.apache.org/jira/browse/BEAM-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-1354: --- Assignee: Aviem Zur > TextIO should comply with PTransform style guide > -

[jira] [Resolved] (BEAM-1334) Split UsesMetrics category and tests into UsesCommittedMetrics and UsesAttemptedMetrics

[ https://issues.apache.org/jira/browse/BEAM-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur resolved BEAM-1334. - Resolution: Done Fix Version/s: 0.6.0 > Split UsesMetrics category and tests into UsesCommittedMetri

[jira] [Resolved] (BEAM-1379) Shade Guava in beam-sdks-java-io-kafka module

[ https://issues.apache.org/jira/browse/BEAM-1379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur resolved BEAM-1379. - Resolution: Fixed Fix Version/s: 0.6.0 > Shade Guava in beam-sdks-java-io-kafka module > ---

[jira] [Resolved] (BEAM-1126) Expose UnboundedSource split backlog in number of events

[ https://issues.apache.org/jira/browse/BEAM-1126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur resolved BEAM-1126. - Resolution: Invalid Fix Version/s: Not applicable > Expose UnboundedSource split backlog in number o

[jira] [Assigned] (BEAM-1397) Introduce IO metrics

[ https://issues.apache.org/jira/browse/BEAM-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-1397: --- Assignee: Aviem Zur (was: Daniel Halperin) > Introduce IO metrics > > >

[jira] [Created] (BEAM-1397) Introduce IO metrics

Aviem Zur created BEAM-1397: --- Summary: Introduce IO metrics Key: BEAM-1397 URL: https://issues.apache.org/jira/browse/BEAM-1397 Project: Beam Issue Type: New Feature Components: sdk-java-

[jira] [Assigned] (BEAM-1397) Introduce IO metrics

[ https://issues.apache.org/jira/browse/BEAM-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-1397: --- Assignee: Daniel Halperin (was: Davor Bonaci) > Introduce IO metrics > > >

[jira] [Assigned] (BEAM-1398) KafkaIO metrics

[ https://issues.apache.org/jira/browse/BEAM-1398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-1398: --- Assignee: Aviem Zur (was: Daniel Halperin) > KafkaIO metrics > --- > > Ke

[jira] [Created] (BEAM-1398) KafkaIO metrics

Aviem Zur created BEAM-1398: --- Summary: KafkaIO metrics Key: BEAM-1398 URL: https://issues.apache.org/jira/browse/BEAM-1398 Project: Beam Issue Type: New Feature Components: sdk-java-exten

[jira] [Assigned] (BEAM-1398) KafkaIO metrics

[ https://issues.apache.org/jira/browse/BEAM-1398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-1398: --- Assignee: Daniel Halperin (was: Davor Bonaci) > KafkaIO metrics > --- > >

[jira] [Updated] (BEAM-1398) KafkaIO metrics

[ https://issues.apache.org/jira/browse/BEAM-1398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-1398: Description: Add metrics to {{KafkaIO}} using the metrics API. Metrics (Feel free to add more ideas here) pe

[jira] [Commented] (BEAM-1354) TextIO should comply with PTransform style guide

[ https://issues.apache.org/jira/browse/BEAM-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15857335#comment-15857335 ] Aviem Zur commented on BEAM-1354: - I've started working on this (locally). I've gone ahead

[jira] [Commented] (BEAM-1402) Make TextIO and AvroIO use best-practice types.

[ https://issues.apache.org/jira/browse/BEAM-1402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15857339#comment-15857339 ] Aviem Zur commented on BEAM-1402: - I've started working on some of what [~jkff] mentioned i

[jira] [Created] (BEAM-1433) Remove coder from TextIO

Aviem Zur created BEAM-1433: --- Summary: Remove coder from TextIO Key: BEAM-1433 URL: https://issues.apache.org/jira/browse/BEAM-1433 Project: Beam Issue Type: Sub-task Components: sdk-java

[jira] [Commented] (BEAM-1402) Make TextIO and AvroIO use best-practice types.

[ https://issues.apache.org/jira/browse/BEAM-1402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15857408#comment-15857408 ] Aviem Zur commented on BEAM-1402: - OK. Created sub-task BEAM-1433 and PR. > Make TextIO an

[jira] [Assigned] (BEAM-1437) Spark runner StreamingListeners are not recoverable.

[ https://issues.apache.org/jira/browse/BEAM-1437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur reassigned BEAM-1437: --- Assignee: Aviem Zur > Spark runner StreamingListeners are not recoverable. > -

[jira] [Updated] (BEAM-2175) Support state API in Spark batch mode.

[ https://issues.apache.org/jira/browse/BEAM-2175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2175: Fix Version/s: 2.1.0 > Support state API in Spark batch mode. > -- > >

<    1   2   3   4   5   >