[jira] [Commented] (SPARK-25762) Upgrade guava version in spark dependency lists due to CVE issue

[ https://issues.apache.org/jira/browse/SPARK-25762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844246#comment-16844246 ] Arun Mahadevan commented on SPARK-25762: Though its shaded, the guava jar is explicitly packaged

[jira] [Created] (SPARK-27754) Introduce spark on k8s config for driver request cores

Arun Mahadevan created SPARK-27754: -- Summary: Introduce spark on k8s config for driver request cores Key: SPARK-27754 URL: https://issues.apache.org/jira/browse/SPARK-27754 Project: Spark

[jira] [Commented] (SPARK-21029) All StreamingQuery should be stopped when the SparkSession is stopped

[ https://issues.apache.org/jira/browse/SPARK-21029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16779765#comment-16779765 ] Arun Mahadevan commented on SPARK-21029: I encountered this issue and propose an alternative

[jira] [Updated] (SPARK-26655) Support multiple aggregates in Structured Streaming append mode

[ https://issues.apache.org/jira/browse/SPARK-26655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Mahadevan updated SPARK-26655: --- Attachment: Watermarks and multiple aggregates in Spark strucutred streaming_v1.pdf >

[jira] [Commented] (SPARK-26655) Support multiple aggregates in Structured Streaming append mode

[ https://issues.apache.org/jira/browse/SPARK-26655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16772415#comment-16772415 ] Arun Mahadevan commented on SPARK-26655: Design doc link:

[jira] [Created] (SPARK-26655) Support multiple aggregates in Structured Streaming append mode

Arun Mahadevan created SPARK-26655: -- Summary: Support multiple aggregates in Structured Streaming append mode Key: SPARK-26655 URL: https://issues.apache.org/jira/browse/SPARK-26655 Project: Spark

[jira] [Commented] (SPARK-10816) EventTime based sessionization

[ https://issues.apache.org/jira/browse/SPARK-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16630697#comment-16630697 ] Arun Mahadevan commented on SPARK-10816: Reiterating earlier point. "map/flapMapGroupsWithState"

[jira] [Created] (SPARK-25555) Generic constructs for windowing and support for custom windows

Arun Mahadevan created SPARK-2: -- Summary: Generic constructs for windowing and support for custom windows Key: SPARK-2 URL: https://issues.apache.org/jira/browse/SPARK-2 Project: Spark

[jira] [Commented] (SPARK-10816) EventTime based sessionization

[ https://issues.apache.org/jira/browse/SPARK-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622729#comment-16622729 ] Arun Mahadevan commented on SPARK-10816: [~rxin], "map/flapMapGroupsWithState" looks to suited

[jira] [Created] (SPARK-25260) Fix namespace handling in SchemaConverters.toAvroType

Arun Mahadevan created SPARK-25260: -- Summary: Fix namespace handling in SchemaConverters.toAvroType Key: SPARK-25260 URL: https://issues.apache.org/jira/browse/SPARK-25260 Project: Spark

[jira] [Created] (SPARK-25000) Identify common interfaces to reduce the code duplication between micro-batch and continuous sources

Arun Mahadevan created SPARK-25000: -- Summary: Identify common interfaces to reduce the code duplication between micro-batch and continuous sources Key: SPARK-25000 URL:

[jira] [Created] (SPARK-24863) Report offset lag as a custom metrics for Kafka structured streaming source

Arun Mahadevan created SPARK-24863: -- Summary: Report offset lag as a custom metrics for Kafka structured streaming source Key: SPARK-24863 URL: https://issues.apache.org/jira/browse/SPARK-24863

[jira] [Created] (SPARK-24748) Support for reporting custom metrics via Streaming Query Progress

Arun Mahadevan created SPARK-24748: -- Summary: Support for reporting custom metrics via Streaming Query Progress Key: SPARK-24748 URL: https://issues.apache.org/jira/browse/SPARK-24748 Project: Spark

[jira] [Updated] (SPARK-24697) Fix the reported start offsets in streaming query progress

[ https://issues.apache.org/jira/browse/SPARK-24697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Mahadevan updated SPARK-24697: --- Summary: Fix the reported start offsets in streaming query progress (was: Fix the reported

[jira] [Updated] (SPARK-24697) Fix the reported start offsets in streaming query

[ https://issues.apache.org/jira/browse/SPARK-24697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Mahadevan updated SPARK-24697: --- Summary: Fix the reported start offsets in streaming query (was: Fix the reported start

[jira] [Created] (SPARK-24697) Fix the reported start and end offsets in streaming query

Arun Mahadevan created SPARK-24697: -- Summary: Fix the reported start and end offsets in streaming query Key: SPARK-24697 URL: https://issues.apache.org/jira/browse/SPARK-24697 Project: Spark

[jira] [Commented] (SPARK-24479) Register StreamingQueryListener in Spark Conf

[ https://issues.apache.org/jira/browse/SPARK-24479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504120#comment-16504120 ] Arun Mahadevan commented on SPARK-24479: PR raised - https://github.com/apache/spark/pull/21504

[jira] [Resolved] (SPARK-24480) Add a config to register custom StreamingQueryListeners

[ https://issues.apache.org/jira/browse/SPARK-24480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Mahadevan resolved SPARK-24480. Resolution: Duplicate Issue is already raised. Duplicate of SPARK-24479 > Add a config

[jira] [Created] (SPARK-24480) Add a config to register custom StreamingQueryListeners

Arun Mahadevan created SPARK-24480: -- Summary: Add a config to register custom StreamingQueryListeners Key: SPARK-24480 URL: https://issues.apache.org/jira/browse/SPARK-24480 Project: Spark

[jira] [Created] (SPARK-24462) Text socket micro-batch reader throws error when a query is restarted with saved state

Arun Mahadevan created SPARK-24462: -- Summary: Text socket micro-batch reader throws error when a query is restarted with saved state Key: SPARK-24462 URL: https://issues.apache.org/jira/browse/SPARK-24462

[jira] [Commented] (SPARK-23887) update query progress

[ https://issues.apache.org/jira/browse/SPARK-23887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16491281#comment-16491281 ] Arun Mahadevan commented on SPARK-23887: We could probably invoke

[jira] [Commented] (SPARK-24036) Stateful operators in continuous processing

[ https://issues.apache.org/jira/browse/SPARK-24036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489700#comment-16489700 ] Arun Mahadevan commented on SPARK-24036: If I understand correctly, continuous job would have a

[jira] [Created] (SPARK-24308) Handle DataReaderFactory to InputPartition renames in left over classes

Arun Mahadevan created SPARK-24308: -- Summary: Handle DataReaderFactory to InputPartition renames in left over classes Key: SPARK-24308 URL: https://issues.apache.org/jira/browse/SPARK-24308 Project:

[jira] [Created] (SPARK-24136) MemoryStreamDataReader.next should skip sleeping if record is available

Arun Mahadevan created SPARK-24136: -- Summary: MemoryStreamDataReader.next should skip sleeping if record is available Key: SPARK-24136 URL: https://issues.apache.org/jira/browse/SPARK-24136 Project:

[jira] [Created] (SPARK-24127) Support text socket source in continuous mode

Arun Mahadevan created SPARK-24127: -- Summary: Support text socket source in continuous mode Key: SPARK-24127 URL: https://issues.apache.org/jira/browse/SPARK-24127 Project: Spark Issue

[jira] [Commented] (SPARK-24036) Stateful operators in continuous processing

[ https://issues.apache.org/jira/browse/SPARK-24036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453058#comment-16453058 ] Arun Mahadevan commented on SPARK-24036: Hi [~joseph.torres], I am also interested to contribute