[jira] [Commented] (SPARK-32285) Add PySpark support for nested timestamps with arrow

[ https://issues.apache.org/jira/browse/SPARK-32285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413482#comment-17413482 ] pralabhkumar commented on SPARK-32285: -- [~hyukjin.kwon] [~emkornfi...@gmail.com] I

[jira] [Commented] (SPARK-36721) Simplify boolean equalities if one side is literal

[ https://issues.apache.org/jira/browse/SPARK-36721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413404#comment-17413404 ] Kazuyuki Tanimura commented on SPARK-36721: --- I am working on this > Simplify

[jira] [Created] (SPARK-36721) Simplify boolean equalities if one side is literal

Kazuyuki Tanimura created SPARK-36721: - Summary: Simplify boolean equalities if one side is literal Key: SPARK-36721 URL: https://issues.apache.org/jira/browse/SPARK-36721 Project: Spark

[jira] [Comment Edited] (SPARK-24943) Convert a SQL Struct to StructType

[ https://issues.apache.org/jira/browse/SPARK-24943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413358#comment-17413358 ] Varun Bharill edited comment on SPARK-24943 at 9/10/21, 7:39 PM: -

[jira] [Commented] (SPARK-24943) Convert a SQL Struct to StructType

[ https://issues.apache.org/jira/browse/SPARK-24943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413358#comment-17413358 ] Varun Bharill commented on SPARK-24943: --- Hi [~hyukjin.kwon],  I am on pyspark-3.1.

[jira] [Assigned] (SPARK-36636) SparkContextSuite random failure in Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-36636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36636: Assignee: (was: Apache Spark) > SparkContextSuite random failure in Scala 2.13 >

[jira] [Assigned] (SPARK-36636) SparkContextSuite random failure in Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-36636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36636: Assignee: Apache Spark > SparkContextSuite random failure in Scala 2.13 > ---

[jira] [Commented] (SPARK-36636) SparkContextSuite random failure in Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-36636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413330#comment-17413330 ] Apache Spark commented on SPARK-36636: -- User 'LuciferYang' has created a pull reque

[jira] [Commented] (SPARK-36636) SparkContextSuite random failure in Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-36636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413295#comment-17413295 ] Yang Jie commented on SPARK-36636: -- [~srowen]  I also re-produce this problem with Scal

[jira] [Commented] (SPARK-36636) SparkContextSuite random failure in Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-36636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413287#comment-17413287 ] Yang Jie commented on SPARK-36636: -- Using tmp dir can effectively avoid this problem. 

[jira] [Comment Edited] (SPARK-36636) SparkContextSuite random failure in Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-36636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413282#comment-17413282 ] Yang Jie edited comment on SPARK-36636 at 9/10/21, 4:40 PM:

[jira] [Commented] (SPARK-36636) SparkContextSuite random failure in Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-36636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413282#comment-17413282 ] Yang Jie commented on SPARK-36636: -- !image-2021-09-11-00-30-20-237.png|width=751,height

[jira] [Assigned] (SPARK-35011) Avoid Block Manager registerations when StopExecutor msg is in-flight.

[ https://issues.apache.org/jira/browse/SPARK-35011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35011: Assignee: Apache Spark > Avoid Block Manager registerations when StopExecutor msg is in-f

[jira] [Commented] (SPARK-35011) Avoid Block Manager registerations when StopExecutor msg is in-flight.

[ https://issues.apache.org/jira/browse/SPARK-35011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413281#comment-17413281 ] Dongjoon Hyun commented on SPARK-35011: --- This is reverted from all branches. > Av

[jira] [Assigned] (SPARK-35011) Avoid Block Manager registerations when StopExecutor msg is in-flight.

[ https://issues.apache.org/jira/browse/SPARK-35011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35011: Assignee: (was: Apache Spark) > Avoid Block Manager registerations when StopExecutor

[jira] [Reopened] (SPARK-35011) Avoid Block Manager registerations when StopExecutor msg is in-flight.

[ https://issues.apache.org/jira/browse/SPARK-35011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reopened SPARK-35011: --- Assignee: (was: Sumeet) > Avoid Block Manager registerations when StopExecutor msg is

[jira] [Updated] (SPARK-35011) Avoid Block Manager registerations when StopExecutor msg is in-flight.

[ https://issues.apache.org/jira/browse/SPARK-35011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35011: -- Fix Version/s: (was: 3.0.4) (was: 3.1.3) (was

[jira] [Updated] (SPARK-36636) SparkContextSuite random failure in Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-36636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-36636: - Attachment: image-2021-09-11-00-30-20-237.png > SparkContextSuite random failure in Scala 2.13 > ---

[jira] [Updated] (SPARK-36636) SparkContextSuite random failure in Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-36636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-36636: - Attachment: image-2021-09-11-00-29-21-168.png > SparkContextSuite random failure in Scala 2.13 > ---

[jira] [Commented] (SPARK-36636) SparkContextSuite random failure in Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-36636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413262#comment-17413262 ] Sean R. Owen commented on SPARK-36636: -- Right, I doubt it's specific to 2.13. It is

[jira] [Comment Edited] (SPARK-36636) SparkContextSuite random failure in Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-36636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413252#comment-17413252 ] Yang Jie edited comment on SPARK-36636 at 9/10/21, 3:54 PM:

[jira] [Commented] (SPARK-36636) SparkContextSuite random failure in Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-36636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413252#comment-17413252 ] Yang Jie commented on SPARK-36636: -- cc [~srowen] [~hyukjin.kwon] > SparkContextSuite r

[jira] [Commented] (SPARK-36636) SparkContextSuite random failure in Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-36636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413249#comment-17413249 ] Yang Jie commented on SPARK-36636: -- After some investigation, I found the reasons for t

[jira] [Created] (SPARK-36720) On overwrite mode, setting option truncate as true doesn't truncate the table

Balaji Balasubramaniam created SPARK-36720: -- Summary: On overwrite mode, setting option truncate as true doesn't truncate the table Key: SPARK-36720 URL: https://issues.apache.org/jira/browse/SPARK-36720

[jira] [Commented] (SPARK-36719) Supporting Netty Logging at the network layer

[ https://issues.apache.org/jira/browse/SPARK-36719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413186#comment-17413186 ] Apache Spark commented on SPARK-36719: -- User 'attilapiros' has created a pull reque

[jira] [Assigned] (SPARK-36719) Supporting Netty Logging at the network layer

[ https://issues.apache.org/jira/browse/SPARK-36719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36719: Assignee: Apache Spark > Supporting Netty Logging at the network layer >

[jira] [Assigned] (SPARK-36719) Supporting Netty Logging at the network layer

[ https://issues.apache.org/jira/browse/SPARK-36719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36719: Assignee: (was: Apache Spark) > Supporting Netty Logging at the network layer > -

[jira] [Updated] (SPARK-36717) Wrong order of variable initialization may lead to incorrect behavior

[ https://issues.apache.org/jira/browse/SPARK-36717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Jianmeng updated SPARK-36717: Description: Incorrect order of variable initialization may lead to incorrect behavior,  Related c

[jira] [Updated] (SPARK-36717) Wrong order of variable initialization may lead to incorrect behavior

[ https://issues.apache.org/jira/browse/SPARK-36717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Jianmeng updated SPARK-36717: Summary: Wrong order of variable initialization may lead to incorrect behavior (was: Incorrect or

[jira] [Commented] (SPARK-35011) Avoid Block Manager registerations when StopExecutor msg is in-flight.

[ https://issues.apache.org/jira/browse/SPARK-35011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413168#comment-17413168 ] Apache Spark commented on SPARK-35011: -- User 'Ngone51' has created a pull request f

[jira] [Commented] (SPARK-36719) Supporting Netty Logging at the network layer

[ https://issues.apache.org/jira/browse/SPARK-36719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413167#comment-17413167 ] Attila Zsolt Piros commented on SPARK-36719: I am working on this and soon I

[jira] [Created] (SPARK-36719) Supporting Netty Logging at the network layer

Attila Zsolt Piros created SPARK-36719: -- Summary: Supporting Netty Logging at the network layer Key: SPARK-36719 URL: https://issues.apache.org/jira/browse/SPARK-36719 Project: Spark Iss

[jira] [Commented] (SPARK-35011) Avoid Block Manager registerations when StopExecutor msg is in-flight.

[ https://issues.apache.org/jira/browse/SPARK-35011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413165#comment-17413165 ] Apache Spark commented on SPARK-35011: -- User 'Ngone51' has created a pull request f

[jira] [Commented] (SPARK-35011) Avoid Block Manager registerations when StopExecutor msg is in-flight.

[ https://issues.apache.org/jira/browse/SPARK-35011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413164#comment-17413164 ] Apache Spark commented on SPARK-35011: -- User 'Ngone51' has created a pull request f

[jira] [Commented] (SPARK-35011) Avoid Block Manager registerations when StopExecutor msg is in-flight.

[ https://issues.apache.org/jira/browse/SPARK-35011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413162#comment-17413162 ] Apache Spark commented on SPARK-35011: -- User 'Ngone51' has created a pull request f

[jira] [Assigned] (SPARK-36718) only collapse projects if we don't duplicate expensive expressions

[ https://issues.apache.org/jira/browse/SPARK-36718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36718: Assignee: (was: Apache Spark) > only collapse projects if we don't duplicate expensiv

[jira] [Assigned] (SPARK-36718) only collapse projects if we don't duplicate expensive expressions

[ https://issues.apache.org/jira/browse/SPARK-36718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36718: Assignee: Apache Spark > only collapse projects if we don't duplicate expensive expressio

[jira] [Commented] (SPARK-36718) only collapse projects if we don't duplicate expensive expressions

[ https://issues.apache.org/jira/browse/SPARK-36718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413161#comment-17413161 ] Apache Spark commented on SPARK-36718: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-36717) Incorrect order of variable initialization may lead to incorrect behavior

[ https://issues.apache.org/jira/browse/SPARK-36717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36717: Assignee: (was: Apache Spark) > Incorrect order of variable initialization may lead t

[jira] [Commented] (SPARK-36717) Incorrect order of variable initialization may lead to incorrect behavior

[ https://issues.apache.org/jira/browse/SPARK-36717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413150#comment-17413150 ] Apache Spark commented on SPARK-36717: -- User 'daugraph' has created a pull request

[jira] [Assigned] (SPARK-36717) Incorrect order of variable initialization may lead to incorrect behavior

[ https://issues.apache.org/jira/browse/SPARK-36717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36717: Assignee: Apache Spark > Incorrect order of variable initialization may lead to incorrect

[jira] [Comment Edited] (SPARK-36717) Incorrect order of variable initialization may lead to incorrect behavior

[ https://issues.apache.org/jira/browse/SPARK-36717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413149#comment-17413149 ] Li Jianmeng edited comment on SPARK-36717 at 9/10/21, 12:25 PM: --

[jira] [Updated] (SPARK-36717) Incorrect order of variable initialization may lead to incorrect behavior

[ https://issues.apache.org/jira/browse/SPARK-36717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Jianmeng updated SPARK-36717: Description: Incorrect order of variable initialization may lead to incorrect behavior,  Related c

[jira] [Commented] (SPARK-36717) Incorrect order of variable initialization may lead to incorrect behavior

[ https://issues.apache.org/jira/browse/SPARK-36717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413149#comment-17413149 ] Li Jianmeng commented on SPARK-36717: - I will work on it, https://github.com/apache/

[jira] [Created] (SPARK-36718) only collapse projects if we don't duplicate expensive expressions

Wenchen Fan created SPARK-36718: --- Summary: only collapse projects if we don't duplicate expensive expressions Key: SPARK-36718 URL: https://issues.apache.org/jira/browse/SPARK-36718 Project: Spark

[jira] [Updated] (SPARK-36717) Incorrect order of variable initialization may lead to incorrect behavior

[ https://issues.apache.org/jira/browse/SPARK-36717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Jianmeng updated SPARK-36717: Description: Incorrect order of variable initialization may lead to incorrect behavior,  Related c

[jira] [Created] (SPARK-36717) Incorrect order of variable initialization may lead to incorrect behavior

Li Jianmeng created SPARK-36717: --- Summary: Incorrect order of variable initialization may lead to incorrect behavior Key: SPARK-36717 URL: https://issues.apache.org/jira/browse/SPARK-36717 Project: Spar

[jira] [Assigned] (SPARK-36715) explode(UDF) throw an exception

[ https://issues.apache.org/jira/browse/SPARK-36715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36715: Assignee: (was: Apache Spark) > explode(UDF) throw an exception > ---

[jira] [Commented] (SPARK-36715) explode(UDF) throw an exception

[ https://issues.apache.org/jira/browse/SPARK-36715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413101#comment-17413101 ] Apache Spark commented on SPARK-36715: -- User 'cfmcgrady' has created a pull request

[jira] [Assigned] (SPARK-36715) explode(UDF) throw an exception

[ https://issues.apache.org/jira/browse/SPARK-36715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36715: Assignee: Apache Spark > explode(UDF) throw an exception > --

[jira] [Assigned] (SPARK-36715) explode(UDF) throw an exception

[ https://issues.apache.org/jira/browse/SPARK-36715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36715: Assignee: Apache Spark > explode(UDF) throw an exception > --

[jira] [Commented] (SPARK-36715) explode(UDF) throw an exception

[ https://issues.apache.org/jira/browse/SPARK-36715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413100#comment-17413100 ] Apache Spark commented on SPARK-36715: -- User 'cfmcgrady' has created a pull request

[jira] [Commented] (SPARK-36702) array_union should not produce duplicated result

[ https://issues.apache.org/jira/browse/SPARK-36702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413072#comment-17413072 ] Apache Spark commented on SPARK-36702: -- User 'AngersZh' has created a pull requ

[jira] [Commented] (SPARK-36716) Join estimation support LeftExistence join type

[ https://issues.apache.org/jira/browse/SPARK-36716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413067#comment-17413067 ] 007akan commented on SPARK-36716: - I'm working on. > Join estimation support LeftExiste

[jira] [Updated] (SPARK-36716) Join estimation support LeftExistence join type

[ https://issues.apache.org/jira/browse/SPARK-36716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 007akan updated SPARK-36716: Attachment: image-2021-09-10-17-17-33-082.png > Join estimation support LeftExistence join type >

[jira] [Created] (SPARK-36716) Join estimation support LeftExistence join type

Yuming Wang created SPARK-36716: --- Summary: Join estimation support LeftExistence join type Key: SPARK-36716 URL: https://issues.apache.org/jira/browse/SPARK-36716 Project: Spark Issue Type: Imp

[jira] [Created] (SPARK-36715) explode(UDF) throw an exception

Fu Chen created SPARK-36715: --- Summary: explode(UDF) throw an exception Key: SPARK-36715 URL: https://issues.apache.org/jira/browse/SPARK-36715 Project: Spark Issue Type: Bug Components: S

[jira] [Commented] (SPARK-36715) explode(UDF) throw an exception

[ https://issues.apache.org/jira/browse/SPARK-36715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413054#comment-17413054 ] Fu Chen commented on SPARK-36715: - working on this > explode(UDF) throw an exception >

[jira] [Created] (SPARK-36714) bugs in MIniLSH

shengzhang created SPARK-36714: -- Summary: bugs in MIniLSH Key: SPARK-36714 URL: https://issues.apache.org/jira/browse/SPARK-36714 Project: Spark Issue Type: Improvement Components: ML

[jira] [Assigned] (SPARK-36709) Support new syntax for specifying index type and name

[ https://issues.apache.org/jira/browse/SPARK-36709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36709: Assignee: Apache Spark > Support new syntax for specifying index type and name >

[jira] [Assigned] (SPARK-36709) Support new syntax for specifying index type and name

[ https://issues.apache.org/jira/browse/SPARK-36709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36709: Assignee: (was: Apache Spark) > Support new syntax for specifying index type and name

[jira] [Commented] (SPARK-36709) Support new syntax for specifying index type and name

[ https://issues.apache.org/jira/browse/SPARK-36709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413028#comment-17413028 ] Apache Spark commented on SPARK-36709: -- User 'HyukjinKwon' has created a pull reque

[jira] [Created] (SPARK-36713) Document new syntax for specifying index type

Hyukjin Kwon created SPARK-36713: Summary: Document new syntax for specifying index type Key: SPARK-36713 URL: https://issues.apache.org/jira/browse/SPARK-36713 Project: Spark Issue Type: Sub

[jira] [Assigned] (SPARK-36712) Published 2.13 POM lists `scala-parallel-collections` only in `scala-2.13` profile

[ https://issues.apache.org/jira/browse/SPARK-36712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36712: Assignee: Apache Spark > Published 2.13 POM lists `scala-parallel-collections` only in `s

[jira] [Assigned] (SPARK-36712) Published 2.13 POM lists `scala-parallel-collections` only in `scala-2.13` profile

[ https://issues.apache.org/jira/browse/SPARK-36712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36712: Assignee: (was: Apache Spark) > Published 2.13 POM lists `scala-parallel-collections`

[jira] [Commented] (SPARK-36712) Published 2.13 POM lists `scala-parallel-collections` only in `scala-2.13` profile

[ https://issues.apache.org/jira/browse/SPARK-36712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413020#comment-17413020 ] Apache Spark commented on SPARK-36712: -- User 'lrytz' has created a pull request for

[jira] [Created] (SPARK-36712) Published 2.13 POM lists `scala-parallel-collections` only in `scala-2.13` profile

Lukas Rytz created SPARK-36712: -- Summary: Published 2.13 POM lists `scala-parallel-collections` only in `scala-2.13` profile Key: SPARK-36712 URL: https://issues.apache.org/jira/browse/SPARK-36712 Projec