[jira] [Commented] (SPARK-41232) High-order function: array_append

[ https://issues.apache.org/jira/browse/SPARK-41232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17641272#comment-17641272 ] Senthil Kumar commented on SPARK-41232: --- [~podongfeng] Shall I work on this? > High-order

[jira] [Commented] (SPARK-40367) Total size of serialized results of 3730 tasks (64.0 GB) is bigger than spark.driver.maxResultSize (64.0 GB)

[ https://issues.apache.org/jira/browse/SPARK-40367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606414#comment-17606414 ] Senthil Kumar commented on SPARK-40367: --- Hi [~jackyjfhu]    Check if you are sending bytes/rows

[jira] [Commented] (SPARK-38213) support Metrics information report to kafkaSink.

[ https://issues.apache.org/jira/browse/SPARK-38213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492391#comment-17492391 ] Senthil Kumar commented on SPARK-38213: --- Working on this > support Metrics information report to

[jira] [Commented] (SPARK-37936) Use error classes in the parsing errors of intervals

[ https://issues.apache.org/jira/browse/SPARK-37936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17480099#comment-17480099 ] Senthil Kumar commented on SPARK-37936: --- [~maxgekk] ,  I have queries which matches    *

[jira] [Commented] (SPARK-37944) Use error classes in the execution errors of casting

[ https://issues.apache.org/jira/browse/SPARK-37944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477551#comment-17477551 ] Senthil Kumar commented on SPARK-37944: --- I will work on this > Use error classes in the execution

[jira] [Commented] (SPARK-37945) Use error classes in the execution errors of arithmetic ops

[ https://issues.apache.org/jira/browse/SPARK-37945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477550#comment-17477550 ] Senthil Kumar commented on SPARK-37945: --- I will work on this > Use error classes in the execution

[jira] [Commented] (SPARK-37940) Use error classes in the compilation errors of partitions

[ https://issues.apache.org/jira/browse/SPARK-37940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477549#comment-17477549 ] Senthil Kumar commented on SPARK-37940: --- I will work on this > Use error classes in the

[jira] [Commented] (SPARK-37939) Use error classes in the parsing errors of properties

[ https://issues.apache.org/jira/browse/SPARK-37939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477548#comment-17477548 ] Senthil Kumar commented on SPARK-37939: --- I will work on this > Use error classes in the parsing

[jira] [Commented] (SPARK-37936) Use error classes in the parsing errors of intervals

[ https://issues.apache.org/jira/browse/SPARK-37936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477544#comment-17477544 ] Senthil Kumar commented on SPARK-37936: --- Working on this > Use error classes in the parsing

[jira] [Commented] (SPARK-36996) fixing "SQL column nullable setting not retained as part of spark read" issue

[ https://issues.apache.org/jira/browse/SPARK-36996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17428141#comment-17428141 ] Senthil Kumar commented on SPARK-36996: --- Sample Output after this changes: SQL : mysql> CREATE

[jira] [Commented] (SPARK-36996) fixing "SQL column nullable setting not retained as part of spark read" issue

[ https://issues.apache.org/jira/browse/SPARK-36996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17428140#comment-17428140 ] Senthil Kumar commented on SPARK-36996: --- We need to consider 2 scenarios   # maintain NULLABLE

[jira] [Commented] (SPARK-36996) fixing "SQL column nullable setting not retained as part of spark read" issue

[ https://issues.apache.org/jira/browse/SPARK-36996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17428104#comment-17428104 ] Senthil Kumar commented on SPARK-36996: --- Based on further analysis, Spark is hard-coding

[jira] [Commented] (SPARK-36996) fixing "SQL column nullable setting not retained as part of spark read" issue

[ https://issues.apache.org/jira/browse/SPARK-36996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17428105#comment-17428105 ] Senthil Kumar commented on SPARK-36996: --- I m working on this > fixing "SQL column nullable

[jira] [Created] (SPARK-36996) fixing "SQL column nullable setting not retained as part of spark read" issue

Senthil Kumar created SPARK-36996: - Summary: fixing "SQL column nullable setting not retained as part of spark read" issue Key: SPARK-36996 URL: https://issues.apache.org/jira/browse/SPARK-36996

[jira] [Commented] (SPARK-36238) Spark UI load event timeline too slow for huge stage

[ https://issues.apache.org/jira/browse/SPARK-36238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17423419#comment-17423419 ] Senthil Kumar commented on SPARK-36238: --- [~angerszhuuu] Did you try increasing heap memory for

[jira] [Commented] (SPARK-36901) ERROR exchange.BroadcastExchangeExec: Could not execute broadcast in 300 secs

[ https://issues.apache.org/jira/browse/SPARK-36901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17423409#comment-17423409 ] Senthil Kumar commented on SPARK-36901: --- [~rangareddy.av...@gmail.com] It looks like normal

[jira] [Comment Edited] (SPARK-36861) Partition columns are overly eagerly parsed as dates

[ https://issues.apache.org/jira/browse/SPARK-36861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17423402#comment-17423402 ] Senthil Kumar edited comment on SPARK-36861 at 10/1/21, 7:24 PM: - Yes in

[jira] [Commented] (SPARK-36861) Partition columns are overly eagerly parsed as dates

[ https://issues.apache.org/jira/browse/SPARK-36861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17423402#comment-17423402 ] Senthil Kumar commented on SPARK-36861: --- Yes in Spark 3.3 hour column is created as "DateType" but

[jira] [Commented] (SPARK-36861) Partition columns are overly eagerly parsed as dates

[ https://issues.apache.org/jira/browse/SPARK-36861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421328#comment-17421328 ] Senthil Kumar commented on SPARK-36861: --- [~tanelk] This is issue not reproducable even in 3.1.2  

[jira] [Commented] (SPARK-36781) The log could not get the correct line number

[ https://issues.apache.org/jira/browse/SPARK-36781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421281#comment-17421281 ] Senthil Kumar commented on SPARK-36781: --- [~chenxusheng] Could you please share the sample code to

[jira] [Updated] (SPARK-36801) Document change for Spark sql jdbc

[ https://issues.apache.org/jira/browse/SPARK-36801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Senthil Kumar updated SPARK-36801: -- Description: Reading using Spark SQL jdbc DataSource  does not maintain nullable type and

[jira] [Updated] (SPARK-36801) Document change for Spark sql jdbc

[ https://issues.apache.org/jira/browse/SPARK-36801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Senthil Kumar updated SPARK-36801: -- Description: Reading using Spark SQL jdbc DataSource  does not maintain nullable type and

[jira] [Created] (SPARK-36801) Document change for Spark sql jdbc

Senthil Kumar created SPARK-36801: - Summary: Document change for Spark sql jdbc Key: SPARK-36801 URL: https://issues.apache.org/jira/browse/SPARK-36801 Project: Spark Issue Type:

[jira] [Commented] (SPARK-36743) Backporting SPARK-36327 changes into Spark 2.4 version

[ https://issues.apache.org/jira/browse/SPARK-36743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17414746#comment-17414746 ] Senthil Kumar commented on SPARK-36743: --- [~hyukjin.kwon], [~dongjoon]. Thanks for the kind and

[jira] [Commented] (SPARK-36743) Backporting SPARK-36327 changes into Spark 2.4 version

[ https://issues.apache.org/jira/browse/SPARK-36743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17414242#comment-17414242 ] Senthil Kumar commented on SPARK-36743: --- [~hyukjin.kwon], [~dongjoon] > Backporting SPARK-36327

[jira] [Updated] (SPARK-36743) Backporting SPARK-36327 changes into Spark 2.4 version

[ https://issues.apache.org/jira/browse/SPARK-36743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Senthil Kumar updated SPARK-36743: -- Summary: Backporting SPARK-36327 changes into Spark 2.4 version (was: Backporting changes

[jira] [Created] (SPARK-36743) Backporting changes into Spark 2.4 version

Senthil Kumar created SPARK-36743: - Summary: Backporting changes into Spark 2.4 version Key: SPARK-36743 URL: https://issues.apache.org/jira/browse/SPARK-36743 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-35623) Volcano resource manager for Spark on Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-35623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17408750#comment-17408750 ] Senthil Kumar edited comment on SPARK-35623 at 9/2/21, 11:56 AM: -

[jira] [Commented] (SPARK-35623) Volcano resource manager for Spark on Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-35623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17408750#comment-17408750 ] Senthil Kumar commented on SPARK-35623: --- [~dipanjanK] Include me too pls > Volcano resource

[jira] [Updated] (SPARK-36643) Add more information in ERROR log while SparkConf is modified when spark.sql.legacy.setCommandRejectsSparkCoreConfs is set

[ https://issues.apache.org/jira/browse/SPARK-36643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Senthil Kumar updated SPARK-36643: -- Component/s: SQL > Add more information in ERROR log while SparkConf is modified when >

[jira] [Comment Edited] (SPARK-36643) Add more information in ERROR log while SparkConf is modified when spark.sql.legacy.setCommandRejectsSparkCoreConfs is set

[ https://issues.apache.org/jira/browse/SPARK-36643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17408282#comment-17408282 ] Senthil Kumar edited comment on SPARK-36643 at 9/1/21, 5:20 PM: New

[jira] [Commented] (SPARK-36643) Add more information in ERROR log while SparkConf is modified when spark.sql.legacy.setCommandRejectsSparkCoreConfs is set

[ https://issues.apache.org/jira/browse/SPARK-36643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17408282#comment-17408282 ] Senthil Kumar commented on SPARK-36643: --- New ERROR message will be as below,   {code:java} scala>

[jira] [Commented] (SPARK-36643) Add more information in ERROR log while SparkConf is modified when spark.sql.legacy.setCommandRejectsSparkCoreConfs is set

[ https://issues.apache.org/jira/browse/SPARK-36643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17408274#comment-17408274 ] Senthil Kumar commented on SPARK-36643: --- Creating PR for this > Add more information in ERROR log

[jira] [Created] (SPARK-36643) Add more information in ERROR log while SparkConf is modified when spark.sql.legacy.setCommandRejectsSparkCoreConfs is set

Senthil Kumar created SPARK-36643: - Summary: Add more information in ERROR log while SparkConf is modified when spark.sql.legacy.setCommandRejectsSparkCoreConfs is set Key: SPARK-36643 URL:

[jira] [Commented] (SPARK-36604) timestamp type column analyze result is wrong

[ https://issues.apache.org/jira/browse/SPARK-36604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17407068#comment-17407068 ] Senthil Kumar commented on SPARK-36604: --- [~yghu] I tested this scenario in Spark2.4, but I don't

[jira] [Updated] (SPARK-36412) Add Test Coverage to meet viewFs(Hadoop federation) scenario

[ https://issues.apache.org/jira/browse/SPARK-36412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Senthil Kumar updated SPARK-36412: -- Summary: Add Test Coverage to meet viewFs(Hadoop federation) scenario (was: Create coverage

[jira] [Commented] (SPARK-36412) Create coverage Test to meet viewFs(Hadoop federation) scenario

[ https://issues.apache.org/jira/browse/SPARK-36412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17392976#comment-17392976 ] Senthil Kumar commented on SPARK-36412: --- I am working on this > Create coverage Test to meet

[jira] [Created] (SPARK-36412) Create coverage Test to meet viewFs(Hadoop federation) scenario

Senthil Kumar created SPARK-36412: - Summary: Create coverage Test to meet viewFs(Hadoop federation) scenario Key: SPARK-36412 URL: https://issues.apache.org/jira/browse/SPARK-36412 Project: Spark

[jira] [Commented] (SPARK-36327) Spark sql creates staging dir inside database directory rather than creating inside table directory

[ https://issues.apache.org/jira/browse/SPARK-36327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17390851#comment-17390851 ] Senthil Kumar commented on SPARK-36327: --- Hi [~sunchao] Hive is creating .staging directories

[jira] [Commented] (SPARK-36327) Spark sql creates staging dir inside database directory rather than creating inside table directory

[ https://issues.apache.org/jira/browse/SPARK-36327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17389827#comment-17389827 ] Senthil Kumar commented on SPARK-36327: --- Hi [~dongjoon],  [~hyukjin.kwon]   Could you please

[jira] [Updated] (SPARK-36327) Spark sql creates staging dir inside database directory rather than creating inside table directory

[ https://issues.apache.org/jira/browse/SPARK-36327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Senthil Kumar updated SPARK-36327: -- Component/s: SQL > Spark sql creates staging dir inside database directory rather than

[jira] [Commented] (SPARK-36327) Spark sql creates staging dir inside database directory rather than creating inside table directory

[ https://issues.apache.org/jira/browse/SPARK-36327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17389814#comment-17389814 ] Senthil Kumar commented on SPARK-36327: --- Created PR https://github.com/apache/spark/pull/33577 >

[jira] [Commented] (SPARK-36327) Spark sql creates staging dir inside database directory rather than creating inside table directory

[ https://issues.apache.org/jira/browse/SPARK-36327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17388597#comment-17388597 ] Senthil Kumar commented on SPARK-36327: --- Shall I work on this Jira to fix this issue? > Spark sql

[jira] [Created] (SPARK-36327) Spark sql creates staging dir inside database directory rather than creating inside table directory

Senthil Kumar created SPARK-36327: - Summary: Spark sql creates staging dir inside database directory rather than creating inside table directory Key: SPARK-36327 URL: