[jira] [Resolved] (SPARK-41355) Workaround hive table name validation issue

2022-12-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-41355. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38765 [https://gith

[jira] [Created] (SPARK-41389) Reuse `WRONG_NUM_ARGS` instead of `_LEGACY_ERROR_TEMP_1043` and `_LEGACY_ERROR_TEMP_1044`

2022-12-05 Thread Yang Jie (Jira)
Yang Jie created SPARK-41389: Summary: Reuse `WRONG_NUM_ARGS` instead of `_LEGACY_ERROR_TEMP_1043` and `_LEGACY_ERROR_TEMP_1044` Key: SPARK-41389 URL: https://issues.apache.org/jira/browse/SPARK-41389 Pro

[jira] [Assigned] (SPARK-41355) Workaround hive table name validation issue

2022-12-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-41355: --- Assignee: Wan Kun > Workaround hive table name validation issue > -

[jira] [Assigned] (SPARK-41388) getReusablePVCs should ignore recently created PVCs in the previous batch

2022-12-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41388: - Assignee: Dongjoon Hyun > getReusablePVCs should ignore recently created PVCs in the pr

[jira] [Commented] (SPARK-41389) Reuse `WRONG_NUM_ARGS` instead of `_LEGACY_ERROR_TEMP_1043` and `_LEGACY_ERROR_TEMP_1044`

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643159#comment-17643159 ] Apache Spark commented on SPARK-41389: -- User 'LuciferYang' has created a pull reque

[jira] [Assigned] (SPARK-41389) Reuse `WRONG_NUM_ARGS` instead of `_LEGACY_ERROR_TEMP_1043` and `_LEGACY_ERROR_TEMP_1044`

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41389: Assignee: Apache Spark > Reuse `WRONG_NUM_ARGS` instead of `_LEGACY_ERROR_TEMP_1043` and

[jira] [Assigned] (SPARK-41389) Reuse `WRONG_NUM_ARGS` instead of `_LEGACY_ERROR_TEMP_1043` and `_LEGACY_ERROR_TEMP_1044`

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41389: Assignee: (was: Apache Spark) > Reuse `WRONG_NUM_ARGS` instead of `_LEGACY_ERROR_TEMP

[jira] [Updated] (SPARK-41389) Reuse `WRONG_NUM_ARGS` instead of `_LEGACY_ERROR_TEMP_1044`

2022-12-05 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-41389: - Summary: Reuse `WRONG_NUM_ARGS` instead of `_LEGACY_ERROR_TEMP_1044` (was: Reuse `WRONG_NUM_ARGS` inste

[jira] [Assigned] (SPARK-41381) Implement count_distinct and sum_distinct functions

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41381: Assignee: Apache Spark > Implement count_distinct and sum_distinct functions > --

[jira] [Assigned] (SPARK-41381) Implement count_distinct and sum_distinct functions

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41381: Assignee: (was: Apache Spark) > Implement count_distinct and sum_distinct functions >

[jira] [Commented] (SPARK-41381) Implement count_distinct and sum_distinct functions

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643178#comment-17643178 ] Apache Spark commented on SPARK-41381: -- User 'zhengruifeng' has created a pull requ

[jira] [Resolved] (SPARK-41388) getReusablePVCs should ignore recently created PVCs in the previous batch

2022-12-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41388. --- Fix Version/s: 3.3.2 3.4.0 Resolution: Fixed Issue resolved by pul

[jira] [Updated] (SPARK-41388) getReusablePVCs should ignore recently created PVCs in the previous batch

2022-12-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-41388: -- Fix Version/s: 3.2.3 > getReusablePVCs should ignore recently created PVCs in the previous bat

[jira] [Updated] (SPARK-41388) getReusablePVCs should ignore recently created PVCs in the previous batch

2022-12-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-41388: -- Fix Version/s: 3.2.4 (was: 3.2.3) > getReusablePVCs should ignore recen

[jira] [Commented] (SPARK-41266) Spark does not parse timestamp strings when using the IN operator

2022-12-05 Thread huldar chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643192#comment-17643192 ] huldar chen commented on SPARK-41266: - I tried to reproduce the above problem, the r

[jira] (SPARK-41266) Spark does not parse timestamp strings when using the IN operator

2022-12-05 Thread huldar chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41266 ] huldar chen deleted comment on SPARK-41266: - was (Author: huldar): I tried to reproduce the above problem, the result is {code:java} sql("CREATE VIEW v1(c1) AS SELECT CAST('2019-08-11 19:33:0

[jira] [Commented] (SPARK-41266) Spark does not parse timestamp strings when using the IN operator

2022-12-05 Thread huldar chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643197#comment-17643197 ] huldar chen commented on SPARK-41266: - I tried to reproduce the above problem, the r

[jira] [Commented] (SPARK-41266) Spark does not parse timestamp strings when using the IN operator

2022-12-05 Thread Laurens Versluis (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643212#comment-17643212 ] Laurens Versluis commented on SPARK-41266: -- Hi [~huldar] I get a different outp

[jira] [Comment Edited] (SPARK-41266) Spark does not parse timestamp strings when using the IN operator

2022-12-05 Thread Laurens Versluis (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643212#comment-17643212 ] Laurens Versluis edited comment on SPARK-41266 at 12/5/22 10:11 AM: --

[jira] [Commented] (SPARK-41382) implement `product` function

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643214#comment-17643214 ] Apache Spark commented on SPARK-41382: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-41382) implement `product` function

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41382: Assignee: Apache Spark > implement `product` function > > >

[jira] [Assigned] (SPARK-41382) implement `product` function

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41382: Assignee: (was: Apache Spark) > implement `product` function > --

[jira] [Commented] (SPARK-41008) Isotonic regression result differs from sklearn implementation

2022-12-05 Thread Ahmed Mahran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643255#comment-17643255 ] Ahmed Mahran commented on SPARK-41008: -- [~srowen] I think you are right. Repeated f

[jira] [Updated] (SPARK-41386) There are some small files when using rebalance(column)

2022-12-05 Thread Zhe Dong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Dong updated SPARK-41386: - Summary: There are some small files when using rebalance(column) (was: Resolve small files with REBALAN

[jira] [Updated] (SPARK-41386) There are some small files when using rebalance(column)

2022-12-05 Thread Zhe Dong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Dong updated SPARK-41386: - Component/s: SQL (was: Spark Core) > There are some small files when using rebalanc

[jira] [Created] (SPARK-41390) Update the script used to generate register function in UDFRegistration

2022-12-05 Thread Yang Jie (Jira)
Yang Jie created SPARK-41390: Summary: Update the script used to generate register function in UDFRegistration Key: SPARK-41390 URL: https://issues.apache.org/jira/browse/SPARK-41390 Project: Spark

[jira] [Commented] (SPARK-41390) Update the script used to generate register function in UDFRegistration

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643279#comment-17643279 ] Apache Spark commented on SPARK-41390: -- User 'LuciferYang' has created a pull reque

[jira] [Assigned] (SPARK-41390) Update the script used to generate register function in UDFRegistration

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41390: Assignee: (was: Apache Spark) > Update the script used to generate register function

[jira] [Assigned] (SPARK-41390) Update the script used to generate register function in UDFRegistration

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41390: Assignee: Apache Spark > Update the script used to generate register function in UDFRegis

[jira] [Created] (SPARK-41391) The output column name of `groupby.agg(count_distinct)` is incorrect

2022-12-05 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-41391: - Summary: The output column name of `groupby.agg(count_distinct)` is incorrect Key: SPARK-41391 URL: https://issues.apache.org/jira/browse/SPARK-41391 Project: Spark

[jira] [Updated] (SPARK-41391) The output column name of `groupBy.agg(count_distinct)` is incorrect

2022-12-05 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-41391: -- Summary: The output column name of `groupBy.agg(count_distinct)` is incorrect (was: The outpu

[jira] [Assigned] (SPARK-41391) The output column name of `groupBy.agg(count_distinct)` is incorrect

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41391: Assignee: (was: Apache Spark) > The output column name of `groupBy.agg(count_distinct

[jira] [Assigned] (SPARK-41391) The output column name of `groupBy.agg(count_distinct)` is incorrect

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41391: Assignee: Apache Spark > The output column name of `groupBy.agg(count_distinct)` is incor

[jira] [Commented] (SPARK-41391) The output column name of `groupBy.agg(count_distinct)` is incorrect

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643296#comment-17643296 ] Apache Spark commented on SPARK-41391: -- User 'zhengruifeng' has created a pull requ

[jira] [Commented] (SPARK-41391) The output column name of `groupBy.agg(count_distinct)` is incorrect

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643297#comment-17643297 ] Apache Spark commented on SPARK-41391: -- User 'zhengruifeng' has created a pull requ

[jira] [Created] (SPARK-41392) spark builds against hadoop trunk/3.4.0-SNAPSHOT fail in scala-maven plugin

2022-12-05 Thread Steve Loughran (Jira)
Steve Loughran created SPARK-41392: -- Summary: spark builds against hadoop trunk/3.4.0-SNAPSHOT fail in scala-maven plugin Key: SPARK-41392 URL: https://issues.apache.org/jira/browse/SPARK-41392 Proje

[jira] [Commented] (SPARK-32530) SPIP: Kotlin support for Apache Spark

2022-12-05 Thread Maziyar PANAHI (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643344#comment-17643344 ] Maziyar PANAHI commented on SPARK-32530: Not sure if this matters, but as a Scal

[jira] [Assigned] (SPARK-41167) Optimize LikeSimplification rule to improve multi like performance

2022-12-05 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-41167: --- Assignee: Wan Kun > Optimize LikeSimplification rule to improve multi like performance > --

[jira] [Resolved] (SPARK-41167) Optimize LikeSimplification rule to improve multi like performance

2022-12-05 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-41167. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38682 [https://gith

[jira] [Commented] (SPARK-41008) Isotonic regression result differs from sklearn implementation

2022-12-05 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643373#comment-17643373 ] Sean R. Owen commented on SPARK-41008: -- No need for an option, this seems like a bu

[jira] [Commented] (SPARK-41008) Isotonic regression result differs from sklearn implementation

2022-12-05 Thread Ahmed Mahran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643374#comment-17643374 ] Ahmed Mahran commented on SPARK-41008: -- Thanks, I'll manage to have a PR in a coupl

[jira] [Resolved] (SPARK-36853) Code failing on checkstyle

2022-12-05 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-36853. -- Resolution: Won't Fix > Code failing on checkstyle > -- > >

[jira] [Resolved] (SPARK-40286) Load Data from S3 deletes data source file

2022-12-05 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-40286. -- Resolution: Not A Problem > Load Data from S3 deletes data source file > -

[jira] [Resolved] (SPARK-40253) Data read exception in orc format

2022-12-05 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-40253. -- Resolution: Won't Fix > Data read exception in orc format > -

[jira] [Resolved] (SPARK-40284) spark concurrent overwrite mode writes data to files in HDFS format, all request data write success

2022-12-05 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-40284. -- Resolution: Not A Problem > spark concurrent overwrite mode writes data to files in HDFS form

[jira] [Resolved] (SPARK-40141) Task listener overloads no longer needed with JDK 8+

2022-12-05 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-40141. -- Resolution: Won't Fix > Task listener overloads no longer needed with JDK 8+ > ---

[jira] [Resolved] (SPARK-39948) exclude velocity 1.5 jar

2022-12-05 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-39948. -- Resolution: Not A Problem > exclude velocity 1.5 jar > > >

[jira] [Updated] (SPARK-40141) Task listener overloads no longer needed with JDK 8+

2022-12-05 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-40141: - Priority: Minor (was: Major) > Task listener overloads no longer needed with JDK 8+ > -

[jira] [Resolved] (SPARK-40642) wrong doc on memory tuning regarding String object memory size, changed since version>=9

2022-12-05 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-40642. -- Resolution: Won't Fix > wrong doc on memory tuning regarding String object memory size, change

[jira] [Resolved] (SPARK-41389) Reuse `WRONG_NUM_ARGS` instead of `_LEGACY_ERROR_TEMP_1044`

2022-12-05 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-41389. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38913 [https://github.com

[jira] [Assigned] (SPARK-41389) Reuse `WRONG_NUM_ARGS` instead of `_LEGACY_ERROR_TEMP_1044`

2022-12-05 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-41389: Assignee: Yang Jie > Reuse `WRONG_NUM_ARGS` instead of `_LEGACY_ERROR_TEMP_1044` > --

[jira] [Created] (SPARK-41393) Upgrade slf4j to 2.0.5

2022-12-05 Thread Yang Jie (Jira)
Yang Jie created SPARK-41393: Summary: Upgrade slf4j to 2.0.5 Key: SPARK-41393 URL: https://issues.apache.org/jira/browse/SPARK-41393 Project: Spark Issue Type: Improvement Components:

[jira] [Commented] (SPARK-41393) Upgrade slf4j to 2.0.5

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643411#comment-17643411 ] Apache Spark commented on SPARK-41393: -- User 'LuciferYang' has created a pull reque

[jira] [Assigned] (SPARK-41393) Upgrade slf4j to 2.0.5

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41393: Assignee: Apache Spark > Upgrade slf4j to 2.0.5 > -- > >

[jira] [Assigned] (SPARK-41393) Upgrade slf4j to 2.0.5

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41393: Assignee: (was: Apache Spark) > Upgrade slf4j to 2.0.5 > -- > >

[jira] [Commented] (SPARK-41392) spark builds against hadoop trunk/3.4.0-SNAPSHOT fail in scala-maven plugin

2022-12-05 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643414#comment-17643414 ] Yang Jie commented on SPARK-41392: -- Can you give the complete compilation command and t

[jira] [Commented] (SPARK-40419) Integrate Grouped Aggregate Pandas UDFs into *.sql test cases

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643461#comment-17643461 ] Apache Spark commented on SPARK-40419: -- User 'MaxGekk' has created a pull request f

[jira] [Resolved] (SPARK-41372) Support DataFrame TempView

2022-12-05 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-41372. -- Resolution: Resolved > Support DataFrame TempView > -- > >

[jira] [Commented] (SPARK-41372) Support DataFrame TempView

2022-12-05 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643473#comment-17643473 ] Xinrong Meng commented on SPARK-41372: -- Resolved by https://github.com/apache/spark

[jira] [Commented] (SPARK-41392) spark builds against hadoop trunk/3.4.0-SNAPSHOT fail in scala-maven plugin

2022-12-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643492#comment-17643492 ] Steve Loughran commented on SPARK-41392: MBP m1 with {code} uname -a Darwin st

[jira] [Commented] (SPARK-39257) use spark.read.jdbc() to read data from SQL databse into dataframe, it fails silently, when the session is killed from SQL server side

2022-12-05 Thread Sandeep Katta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643501#comment-17643501 ] Sandeep Katta commented on SPARK-39257: --- Closing this jira as this is fixed by *ms

[jira] [Resolved] (SPARK-39257) use spark.read.jdbc() to read data from SQL databse into dataframe, it fails silently, when the session is killed from SQL server side

2022-12-05 Thread Sandeep Katta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Katta resolved SPARK-39257. --- Resolution: Not A Problem Issue is caused by mssql-jdbc driver and is fixed in 12.1.0 versio

[jira] [Created] (SPARK-41394) Skip MemoryProfilerTests when pandas is not installed

2022-12-05 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-41394: - Summary: Skip MemoryProfilerTests when pandas is not installed Key: SPARK-41394 URL: https://issues.apache.org/jira/browse/SPARK-41394 Project: Spark Issue

[jira] [Commented] (SPARK-41394) Skip MemoryProfilerTests when pandas is not installed

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643509#comment-17643509 ] Apache Spark commented on SPARK-41394: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-41394) Skip MemoryProfilerTests when pandas is not installed

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41394: Assignee: Apache Spark > Skip MemoryProfilerTests when pandas is not installed >

[jira] [Assigned] (SPARK-41394) Skip MemoryProfilerTests when pandas is not installed

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41394: Assignee: (was: Apache Spark) > Skip MemoryProfilerTests when pandas is not installed

[jira] [Commented] (SPARK-41394) Skip MemoryProfilerTests when pandas is not installed

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643510#comment-17643510 ] Apache Spark commented on SPARK-41394: -- User 'dongjoon-hyun' has created a pull req

[jira] [Commented] (SPARK-18502) Spark does not handle columns that contain backquote (`)

2022-12-05 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-18502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643524#comment-17643524 ] Bjørn Jørgensen commented on SPARK-18502: - I just answered this problem in u...@

[jira] [Comment Edited] (SPARK-18502) Spark does not handle columns that contain backquote (`)

2022-12-05 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-18502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643524#comment-17643524 ] Bjørn Jørgensen edited comment on SPARK-18502 at 12/5/22 7:36 PM:

[jira] [Created] (SPARK-41395) InterpretedMutableProjection can corrupt unsafe buffer when used with decimal data

2022-12-05 Thread Bruce Robbins (Jira)
Bruce Robbins created SPARK-41395: - Summary: InterpretedMutableProjection can corrupt unsafe buffer when used with decimal data Key: SPARK-41395 URL: https://issues.apache.org/jira/browse/SPARK-41395

[jira] [Updated] (SPARK-41395) InterpretedMutableProjection can corrupt unsafe buffer when used with decimal data

2022-12-05 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-41395: -- Description: The following returns the wrong answer: {noformat} set spark.sql.codegen.wholeSt

[jira] [Assigned] (SPARK-41390) Update the script used to generate register function in UDFRegistration

2022-12-05 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-41390: Assignee: Yang Jie > Update the script used to generate register function in UDFRegistration > -

[jira] [Resolved] (SPARK-41390) Update the script used to generate register function in UDFRegistration

2022-12-05 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-41390. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38916 [https://github.com

[jira] [Updated] (SPARK-41395) InterpretedMutableProjection can corrupt unsafe buffer when used with decimal data

2022-12-05 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-41395: -- Affects Version/s: 3.2.3 > InterpretedMutableProjection can corrupt unsafe buffer when used wi

[jira] [Updated] (SPARK-41395) InterpretedMutableProjection can corrupt unsafe buffer when used with decimal data

2022-12-05 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-41395: -- Affects Version/s: 3.3.1 > InterpretedMutableProjection can corrupt unsafe buffer when used wi

[jira] [Created] (SPARK-41396) Oneof field support and recursive fields

2022-12-05 Thread Sandish Kumar HN (Jira)
Sandish Kumar HN created SPARK-41396: Summary: Oneof field support and recursive fields Key: SPARK-41396 URL: https://issues.apache.org/jira/browse/SPARK-41396 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-41396) Oneof field support and recursive fields

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41396: Assignee: (was: Apache Spark) > Oneof field support and recursive fields > --

[jira] [Commented] (SPARK-41396) Oneof field support and recursive fields

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643567#comment-17643567 ] Apache Spark commented on SPARK-41396: -- User 'SandishKumarHN' has created a pull re

[jira] [Assigned] (SPARK-41396) Oneof field support and recursive fields

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41396: Assignee: Apache Spark > Oneof field support and recursive fields > -

[jira] [Commented] (SPARK-41396) Oneof field support and recursive fields

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643568#comment-17643568 ] Apache Spark commented on SPARK-41396: -- User 'SandishKumarHN' has created a pull re

[jira] [Updated] (SPARK-41395) InterpretedMutableProjection can corrupt unsafe buffer when used with decimal data

2022-12-05 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-41395: -- Description: The following returns the wrong answer: {noformat} set spark.sql.codegen.wholeSt

[jira] [Assigned] (SPARK-41394) Skip MemoryProfilerTests when pandas is not installed

2022-12-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41394: - Assignee: Dongjoon Hyun > Skip MemoryProfilerTests when pandas is not installed > -

[jira] [Resolved] (SPARK-41394) Skip MemoryProfilerTests when pandas is not installed

2022-12-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41394. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38920 [https://

[jira] [Assigned] (SPARK-41395) InterpretedMutableProjection can corrupt unsafe buffer when used with decimal data

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41395: Assignee: Apache Spark > InterpretedMutableProjection can corrupt unsafe buffer when used

[jira] [Commented] (SPARK-41395) InterpretedMutableProjection can corrupt unsafe buffer when used with decimal data

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643587#comment-17643587 ] Apache Spark commented on SPARK-41395: -- User 'bersprockets' has created a pull requ

[jira] [Assigned] (SPARK-41395) InterpretedMutableProjection can corrupt unsafe buffer when used with decimal data

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41395: Assignee: (was: Apache Spark) > InterpretedMutableProjection can corrupt unsafe buffe

[jira] [Commented] (SPARK-41395) InterpretedMutableProjection can corrupt unsafe buffer when used with decimal data

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643588#comment-17643588 ] Apache Spark commented on SPARK-41395: -- User 'bersprockets' has created a pull requ

[jira] [Updated] (SPARK-41397) Implement part of string/binary functions

2022-12-05 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-41397: - Summary: Implement part of string/binary functions (was: Implement part of String/Binary functi

[jira] [Created] (SPARK-41397) Implement part of String/Binary functions

2022-12-05 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-41397: Summary: Implement part of String/Binary functions Key: SPARK-41397 URL: https://issues.apache.org/jira/browse/SPARK-41397 Project: Spark Issue Type: Sub-tas

[jira] [Commented] (SPARK-41397) Implement part of string/binary functions

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643594#comment-17643594 ] Apache Spark commented on SPARK-41397: -- User 'xinrong-meng' has created a pull requ

[jira] [Assigned] (SPARK-41397) Implement part of string/binary functions

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41397: Assignee: (was: Apache Spark) > Implement part of string/binary functions > -

[jira] [Assigned] (SPARK-41397) Implement part of string/binary functions

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41397: Assignee: Apache Spark > Implement part of string/binary functions >

[jira] [Created] (SPARK-41398) Relax constraints on Storage-Partitioned Join when partition keys after runtime filtering do not match

2022-12-05 Thread Chao Sun (Jira)
Chao Sun created SPARK-41398: Summary: Relax constraints on Storage-Partitioned Join when partition keys after runtime filtering do not match Key: SPARK-41398 URL: https://issues.apache.org/jira/browse/SPARK-41398

[jira] [Commented] (SPARK-41398) Relax constraints on Storage-Partitioned Join when partition keys after runtime filtering do not match

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643610#comment-17643610 ] Apache Spark commented on SPARK-41398: -- User 'sunchao' has created a pull request f

[jira] [Assigned] (SPARK-41398) Relax constraints on Storage-Partitioned Join when partition keys after runtime filtering do not match

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41398: Assignee: (was: Apache Spark) > Relax constraints on Storage-Partitioned Join when pa

[jira] [Assigned] (SPARK-41398) Relax constraints on Storage-Partitioned Join when partition keys after runtime filtering do not match

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41398: Assignee: Apache Spark > Relax constraints on Storage-Partitioned Join when partition key

[jira] [Commented] (SPARK-41398) Relax constraints on Storage-Partitioned Join when partition keys after runtime filtering do not match

2022-12-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643611#comment-17643611 ] Apache Spark commented on SPARK-41398: -- User 'sunchao' has created a pull request f

[jira] [Resolved] (SPARK-41369) Refactor connect directory structure

2022-12-05 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-41369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-41369. --- Fix Version/s: 3.4.0 Resolution: Resolved > Refactor connect directory struct

[jira] [Assigned] (SPARK-41369) Refactor connect directory structure

2022-12-05 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-41369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell reassigned SPARK-41369: - Assignee: Venkata Sai Akhil Gudesa > Refactor connect directory structure > ---

[jira] [Created] (SPARK-41399) Refactor column related tests to test_connect_column

2022-12-05 Thread Rui Wang (Jira)
Rui Wang created SPARK-41399: Summary: Refactor column related tests to test_connect_column Key: SPARK-41399 URL: https://issues.apache.org/jira/browse/SPARK-41399 Project: Spark Issue Type: Sub-

  1   2   >