[jira] [Updated] (SPARK-32376) Make unionByName null-filling behavior work with struct columns

2020-08-18 Thread Mukul Murthy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Murthy updated SPARK-32376: - Attachment: tests.scala > Make unionByName null-filling behavior work with struct columns >

[jira] [Commented] (SPARK-32376) Make unionByName null-filling behavior work with struct columns

2020-08-18 Thread Mukul Murthy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17179450#comment-17179450 ] Mukul Murthy commented on SPARK-32376: -- I'm sorry, I only saw the mail for this now. Attaching the

[jira] [Created] (SPARK-32376) Make unionByName null-filling behavior work with struct columns

2020-07-20 Thread Mukul Murthy (Jira)
Mukul Murthy created SPARK-32376: Summary: Make unionByName null-filling behavior work with struct columns Key: SPARK-32376 URL: https://issues.apache.org/jira/browse/SPARK-32376 Project: Spark

[jira] [Commented] (SPARK-31324) StreamingQuery stop() timeout exception should include the stream ID

2020-04-01 Thread Mukul Murthy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17073289#comment-17073289 ] Mukul Murthy commented on SPARK-31324: -- Gotcha, thanks. Was not planning on backporting this, but I

[jira] [Created] (SPARK-31324) StreamingQuery stop() timeout exception should include the stream ID

2020-04-01 Thread Mukul Murthy (Jira)
Mukul Murthy created SPARK-31324: Summary: StreamingQuery stop() timeout exception should include the stream ID Key: SPARK-31324 URL: https://issues.apache.org/jira/browse/SPARK-31324 Project: Spark

[jira] [Commented] (SPARK-29358) Make unionByName optionally fill missing columns with nulls

2019-10-10 Thread Mukul Murthy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16948800#comment-16948800 ] Mukul Murthy commented on SPARK-29358: -- That would be a start to make us not have to do #1, but #2

[jira] [Commented] (SPARK-29358) Make unionByName optionally fill missing columns with nulls

2019-10-09 Thread Mukul Murthy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16947982#comment-16947982 ] Mukul Murthy commented on SPARK-29358: -- [~hyukjin.kwon], I disagree that the workaround is pretty

[jira] [Commented] (SPARK-29358) Make unionByName optionally fill missing columns with nulls

2019-10-07 Thread Mukul Murthy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16946069#comment-16946069 ] Mukul Murthy commented on SPARK-29358: -- I agree that it should not change the current behavior of

[jira] [Created] (SPARK-29358) Make unionByName optionally fill missing columns with nulls

2019-10-04 Thread Mukul Murthy (Jira)
Mukul Murthy created SPARK-29358: Summary: Make unionByName optionally fill missing columns with nulls Key: SPARK-29358 URL: https://issues.apache.org/jira/browse/SPARK-29358 Project: Spark

[jira] [Updated] (SPARK-26046) Add a way for StreamingQueryManager to remove all listeners

2019-08-30 Thread Mukul Murthy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Murthy updated SPARK-26046: - Description: StreamingQueryManager should have a way to clear out all listeners. There's

[jira] [Reopened] (SPARK-26046) Add a way for StreamingQueryManager to remove all listeners

2019-08-30 Thread Mukul Murthy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Murthy reopened SPARK-26046: -- >From some other discussions I've had, I actually think it's a reasonable to >have a way to

[jira] [Updated] (SPARK-28043) Reading json with duplicate columns drops the first column value

2019-06-13 Thread Mukul Murthy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Murthy updated SPARK-28043: - Description: When reading a JSON blob with duplicate fields, Spark appears to ignore the value

[jira] [Created] (SPARK-28043) Reading json with duplicate columns drops the first column value

2019-06-13 Thread Mukul Murthy (JIRA)
Mukul Murthy created SPARK-28043: Summary: Reading json with duplicate columns drops the first column value Key: SPARK-28043 URL: https://issues.apache.org/jira/browse/SPARK-28043 Project: Spark

[jira] [Created] (SPARK-26586) Streaming queries should have isolated SparkSessions and confs

2019-01-09 Thread Mukul Murthy (JIRA)
Mukul Murthy created SPARK-26586: Summary: Streaming queries should have isolated SparkSessions and confs Key: SPARK-26586 URL: https://issues.apache.org/jira/browse/SPARK-26586 Project: Spark

[jira] [Created] (SPARK-26046) Add a way for StreamingQueryManager to remove all listeners

2018-11-13 Thread Mukul Murthy (JIRA)
Mukul Murthy created SPARK-26046: Summary: Add a way for StreamingQueryManager to remove all listeners Key: SPARK-26046 URL: https://issues.apache.org/jira/browse/SPARK-26046 Project: Spark

[jira] [Created] (SPARK-25449) Don't send zero accumulators in heartbeats

2018-09-17 Thread Mukul Murthy (JIRA)
Mukul Murthy created SPARK-25449: Summary: Don't send zero accumulators in heartbeats Key: SPARK-25449 URL: https://issues.apache.org/jira/browse/SPARK-25449 Project: Spark Issue Type: Task

[jira] [Updated] (SPARK-25399) Reusing execution threads from continuous processing for microbatch streaming can result in correctness issues

2018-09-10 Thread Mukul Murthy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Murthy updated SPARK-25399: - Priority: Major (was: Blocker) > Reusing execution threads from continuous processing for

[jira] [Commented] (SPARK-25399) Reusing execution threads from continuous processing for microbatch streaming can result in correctness issues

2018-09-10 Thread Mukul Murthy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16609819#comment-16609819 ] Mukul Murthy commented on SPARK-25399: -- cc [~joseph.torres] and [~tdas] > Reusing execution

[jira] [Created] (SPARK-25399) Reusing execution threads from continuous processing for microbatch streaming can result in correctness issues

2018-09-10 Thread Mukul Murthy (JIRA)
Mukul Murthy created SPARK-25399: Summary: Reusing execution threads from continuous processing for microbatch streaming can result in correctness issues Key: SPARK-25399 URL:

[jira] [Resolved] (SPARK-25182) Block Manager master and slave thread pools are unbounded

2018-08-21 Thread Mukul Murthy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Murthy resolved SPARK-25182. -- Resolution: Duplicate Target Version/s: (was: 2.4.0) > Block Manager master and

[jira] [Created] (SPARK-25182) Block Manager master and slave thread pools are unbounded

2018-08-21 Thread Mukul Murthy (JIRA)
Mukul Murthy created SPARK-25182: Summary: Block Manager master and slave thread pools are unbounded Key: SPARK-25182 URL: https://issues.apache.org/jira/browse/SPARK-25182 Project: Spark

[jira] [Created] (SPARK-25181) Block Manager master and slave thread pools are unbounded

2018-08-21 Thread Mukul Murthy (JIRA)
Mukul Murthy created SPARK-25181: Summary: Block Manager master and slave thread pools are unbounded Key: SPARK-25181 URL: https://issues.apache.org/jira/browse/SPARK-25181 Project: Spark

[jira] [Commented] (SPARK-24438) Empty strings and null strings are written to the same partition

2018-07-05 Thread Mukul Murthy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16533977#comment-16533977 ] Mukul Murthy commented on SPARK-24438: -- Are null and empty string both invalid partition values? I

[jira] [Commented] (SPARK-24662) Structured Streaming should support LIMIT

2018-06-28 Thread Mukul Murthy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16526848#comment-16526848 ] Mukul Murthy commented on SPARK-24662: -- Calling .limit(n) on a DataFrame (or in SQL, SELECT ...

[jira] [Created] (SPARK-24662) Structured Streaming should support LIMIT

2018-06-26 Thread Mukul Murthy (JIRA)
Mukul Murthy created SPARK-24662: Summary: Structured Streaming should support LIMIT Key: SPARK-24662 URL: https://issues.apache.org/jira/browse/SPARK-24662 Project: Spark Issue Type: New

[jira] [Created] (SPARK-24525) Provide an option to limit MemorySink memory usage

2018-06-11 Thread Mukul Murthy (JIRA)
Mukul Murthy created SPARK-24525: Summary: Provide an option to limit MemorySink memory usage Key: SPARK-24525 URL: https://issues.apache.org/jira/browse/SPARK-24525 Project: Spark Issue

[jira] [Created] (SPARK-24438) Empty strings and null strings are written to the same partition

2018-05-30 Thread Mukul Murthy (JIRA)
Mukul Murthy created SPARK-24438: Summary: Empty strings and null strings are written to the same partition Key: SPARK-24438 URL: https://issues.apache.org/jira/browse/SPARK-24438 Project: Spark