[jira] [Commented] (SPARK-16545) Structured Streaming : foreachSink creates the Physical Plan multiple times per TriggerInterval

[ https://issues.apache.org/jira/browse/SPARK-16545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15646665#comment-15646665 ] Mario Briggs commented on SPARK-16545: -- [~lwlin] I agree with the PR discussion. I am not terribly

[jira] [Commented] (SPARK-17630) jvm-exit-on-fatal-error handler for spark.rpc.netty like there is available for akka

[ https://issues.apache.org/jira/browse/SPARK-17630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587664#comment-15587664 ] Mario Briggs commented on SPARK-17630: -- [~zsxwing] thanks much. any pointers on how/where to add

[jira] [Comment Edited] (SPARK-17917) Convert 'Initial job has not accepted any resources..' logWarning to a SparkListener event

[ https://issues.apache.org/jira/browse/SPARK-17917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574639#comment-15574639 ] Mario Briggs edited comment on SPARK-17917 at 10/18/16 6:49 PM: >> I

[jira] [Commented] (SPARK-17917) Convert 'Initial job has not accepted any resources..' logWarning to a SparkListener event

[ https://issues.apache.org/jira/browse/SPARK-17917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574639#comment-15574639 ] Mario Briggs commented on SPARK-17917: -- >> I don't have a strong feeling on this partly because I'm

[jira] [Commented] (SPARK-17917) Convert 'Initial job has not accepted any resources..' logWarning to a SparkListener event

[ https://issues.apache.org/jira/browse/SPARK-17917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15572795#comment-15572795 ] Mario Briggs commented on SPARK-17917: -- would appreciate if the spark devs comment in whether they

[jira] [Created] (SPARK-17917) Convert 'Initial job has not accepted any resources..' logWarning to a SparkListener event

Mario Briggs created SPARK-17917: Summary: Convert 'Initial job has not accepted any resources..' logWarning to a SparkListener event Key: SPARK-17917 URL: https://issues.apache.org/jira/browse/SPARK-17917

[jira] [Updated] (SPARK-17630) jvm-exit-on-fatal-error handler for spark.rpc.netty like there is available for akka

[ https://issues.apache.org/jira/browse/SPARK-17630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mario Briggs updated SPARK-17630: - Summary: jvm-exit-on-fatal-error handler for spark.rpc.netty like there is available for akka

[jira] [Updated] (SPARK-17630) jvm-exit-on-fatal-error for spark.rpc.netty like there is available for akka

[ https://issues.apache.org/jira/browse/SPARK-17630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mario Briggs updated SPARK-17630: - Description: Hi, I have 2 code-paths from my app that result in a jvm OOM. In the first code

[jira] [Updated] (SPARK-17630) jvm-exit-on-fatal-error for spark.rpc.netty like there is available for akka

[ https://issues.apache.org/jira/browse/SPARK-17630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mario Briggs updated SPARK-17630: - Attachment: SecondCodePath.txt > jvm-exit-on-fatal-error for spark.rpc.netty like there is

[jira] [Updated] (SPARK-17630) jvm-exit-on-fatal-error for spark.rpc.netty like there is available for akka

[ https://issues.apache.org/jira/browse/SPARK-17630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mario Briggs updated SPARK-17630: - Attachment: firstCodepath.txt > jvm-exit-on-fatal-error for spark.rpc.netty like there is

[jira] [Updated] (SPARK-17630) jvm-exit-on-fatal-error for spark.rpc.netty like there is available for akka

[ https://issues.apache.org/jira/browse/SPARK-17630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mario Briggs updated SPARK-17630: - Description: Hi, I have 2 code-paths from my app that result in a jvm OOM. In the first code

[jira] [Created] (SPARK-17630) jvm-exit-on-fatal-error for spark.rpc.netty like there is available for akka

Mario Briggs created SPARK-17630: Summary: jvm-exit-on-fatal-error for spark.rpc.netty like there is available for akka Key: SPARK-17630 URL: https://issues.apache.org/jira/browse/SPARK-17630

[jira] [Comment Edited] (SPARK-16545) Structured Streaming : foreachSink creates the Physical Plan multiple times per TriggerInterval

[ https://issues.apache.org/jira/browse/SPARK-16545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15378811#comment-15378811 ] Mario Briggs edited comment on SPARK-16545 at 7/15/16 3:57 AM: --- thanks. I

[jira] [Commented] (SPARK-16545) Structured Streaming : foreachSink creates the Physical Plan multiple times per TriggerInterval

[ https://issues.apache.org/jira/browse/SPARK-16545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15378811#comment-15378811 ] Mario Briggs commented on SPARK-16545: -- thanks. I looked into it too and i was not getting a fix

[jira] [Comment Edited] (SPARK-16545) Structured Streaming : foreachSink creates the Physical Plan multiple times per TriggerInterval

[ https://issues.apache.org/jira/browse/SPARK-16545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15376758#comment-15376758 ] Mario Briggs edited comment on SPARK-16545 at 7/14/16 11:01 AM: While

[jira] [Comment Edited] (SPARK-16545) Structured Streaming : foreachSink creates the Physical Plan multiple times per TriggerInterval

[ https://issues.apache.org/jira/browse/SPARK-16545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15376758#comment-15376758 ] Mario Briggs edited comment on SPARK-16545 at 7/14/16 10:53 AM: While

[jira] [Commented] (SPARK-16545) Structured Streaming : foreachSink creates the Physical Plan multiple times per TriggerInterval

[ https://issues.apache.org/jira/browse/SPARK-16545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15376758#comment-15376758 ] Mario Briggs commented on SPARK-16545: -- While looking at the performance of Structured streaming,

[jira] [Created] (SPARK-16545) Structured Streaming : foreachSink creates the Physical Plan multiple times per TriggerInterval

Mario Briggs created SPARK-16545: Summary: Structured Streaming : foreachSink creates the Physical Plan multiple times per TriggerInterval Key: SPARK-16545 URL: https://issues.apache.org/jira/browse/SPARK-16545

[jira] [Commented] (SPARK-15089) kafka-spark consumer with SSL problem

[ https://issues.apache.org/jira/browse/SPARK-15089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15270203#comment-15270203 ] Mario Briggs commented on SPARK-15089: -- Kafka supports SSL only with the 0.9x Kakfa client API's.

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259530#comment-15259530 ] Mario Briggs commented on SPARK-12177: -- what's the thinking on this one with respect to Structured

[jira] [Commented] (SPARK-14745) CEP support in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-14745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251353#comment-15251353 ] Mario Briggs commented on SPARK-14745: -- Hi Sean, thanks for 'fixing' the target. To answer you

[jira] [Commented] (SPARK-14597) Streaming Listener timing metrics should include time spent in JobGenerator's graph.generateJobs

[ https://issues.apache.org/jira/browse/SPARK-14597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251285#comment-15251285 ] Mario Briggs commented on SPARK-14597: -- Yeah i think it is neat. Good to me. Thanks > Streaming

[jira] [Comment Edited] (SPARK-14745) CEP support in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-14745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15249261#comment-15249261 ] Mario Briggs edited comment on SPARK-14745 at 4/20/16 5:18 AM: --- Document

[jira] [Updated] (SPARK-14745) CEP support in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-14745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mario Briggs updated SPARK-14745: - Attachment: SparkStreamingCEP.pdf Examples, Features and possible API > CEP support in Spark

[jira] [Created] (SPARK-14745) CEP support in Spark Streaming

Mario Briggs created SPARK-14745: Summary: CEP support in Spark Streaming Key: SPARK-14745 URL: https://issues.apache.org/jira/browse/SPARK-14745 Project: Spark Issue Type: New Feature

[jira] [Comment Edited] (SPARK-14597) Streaming Listener timing metrics should include time spent in JobGenerator's graph.generateJobs

[ https://issues.apache.org/jira/browse/SPARK-14597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241358#comment-15241358 ] Mario Briggs edited comment on SPARK-14597 at 4/14/16 3:32 PM: --- I think

[jira] [Commented] (SPARK-14597) Streaming Listener timing metrics should include time spent in JobGenerator's graph.generateJobs

[ https://issues.apache.org/jira/browse/SPARK-14597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241358#comment-15241358 ] Mario Briggs commented on SPARK-14597: -- I think there is an opportunity to merge both your

[jira] [Commented] (SPARK-13650) Usage of the window() function on DStream

[ https://issues.apache.org/jira/browse/SPARK-13650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15179347#comment-15179347 ] Mario Briggs commented on SPARK-13650: -- Running locally on my MAC, here is the output {quote}

[jira] [Created] (SPARK-13650) Usage of the window() function on DStream

Mario Briggs created SPARK-13650: Summary: Usage of the window() function on DStream Key: SPARK-13650 URL: https://issues.apache.org/jira/browse/SPARK-13650 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-13027) Add API for updateStateByKey to provide batch time as input

[ https://issues.apache.org/jira/browse/SPARK-13027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133643#comment-15133643 ] Mario Briggs commented on SPARK-13027: -- [~rameshaaditya117] sure, more than happy to review > Add

[jira] [Commented] (SPARK-13027) Add API for updateStateByKey to provide batch time as input

[ https://issues.apache.org/jira/browse/SPARK-13027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15132641#comment-15132641 ] Mario Briggs commented on SPARK-13027: -- [~rameshaaditya117] if you are tied up on something else, i

[jira] [Commented] (SPARK-12739) Details of batch in Streaming tab uses two Duration columns

[ https://issues.apache.org/jira/browse/SPARK-12739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15125974#comment-15125974 ] Mario Briggs commented on SPARK-12739: -- Maybe the 1st duration can be named 'Output Op Duration' and

[jira] [Commented] (SPARK-13009) spark-streaming-twitter_2.10 does not make it possible to access the raw twitter json

[ https://issues.apache.org/jira/browse/SPARK-13009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15125940#comment-15125940 ] Mario Briggs commented on SPARK-13009: -- Andrew, as you yourself noted, i think it is more

[jira] [Commented] (SPARK-13009) spark-streaming-twitter_2.10 does not make it possible to access the raw twitter json

[ https://issues.apache.org/jira/browse/SPARK-13009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127718#comment-15127718 ] Mario Briggs commented on SPARK-13009: -- IMHO even if the StatusJSONImpl was not final, still should

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110994#comment-15110994 ] Mario Briggs commented on SPARK-12177: -- bq. If one uses the kafka v9 jar even when using the old

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15109056#comment-15109056 ] Mario Briggs commented on SPARK-12177: -- bq. I totally understand what you mean. However, kafka has

[jira] [Comment Edited] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15109056#comment-15109056 ] Mario Briggs edited comment on SPARK-12177 at 1/20/16 6:16 PM: --- bq. I

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15105560#comment-15105560 ] Mario Briggs commented on SPARK-12177: -- Hi Nikita, great. 1 - We should also have a

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15082674#comment-15082674 ] Mario Briggs commented on SPARK-12177: -- implemented here -

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15081565#comment-15081565 ] Mario Briggs commented on SPARK-12177: -- Confirmed that the frequent KafkaConsumer object creation

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15075230#comment-15075230 ] Mario Briggs commented on SPARK-12177: -- you could also get just a few of the records you want i.e.

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15075172#comment-15075172 ] Mario Briggs commented on SPARK-12177: -- Nikita, thank you. A-C : Looks good to me. (BTW i didn't

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15075205#comment-15075205 ] Mario Briggs commented on SPARK-12177: -- Very good point about creation of KafkaConsumer frequently.

[jira] [Comment Edited] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15075230#comment-15075230 ] Mario Briggs edited comment on SPARK-12177 at 12/30/15 5:24 PM: you could

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074131#comment-15074131 ] Mario Briggs commented on SPARK-12177: -- with regards to review comment 'C' (2 comments above), the

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15073035#comment-15073035 ] Mario Briggs commented on SPARK-12177: -- On the issue of lots of duplicate code from the original, 1

[jira] [Comment Edited] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15073020#comment-15073020 ] Mario Briggs edited comment on SPARK-12177 at 12/28/15 7:02 PM: Hi

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15073020#comment-15073020 ] Mario Briggs commented on SPARK-12177: -- Hi Nikita, thanks. Here are my review comments. I couldnt