[jira] [Resolved] (SPARK-39060) Typo in error messages of decimal overflow

[ https://issues.apache.org/jira/browse/SPARK-39060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-39060. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36397 [https://github.com

[jira] [Assigned] (SPARK-39060) Typo in error messages of decimal overflow

[ https://issues.apache.org/jira/browse/SPARK-39060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-39060: Assignee: Vitalii Li > Typo in error messages of decimal overflow > -

[jira] [Updated] (SPARK-39087) Improve messages of error classes

[ https://issues.apache.org/jira/browse/SPARK-39087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-39087: - Fix Version/s: 3.3.0 > Improve messages of error classes > - > >

[jira] [Comment Edited] (SPARK-25355) Support --proxy-user for Spark on K8s

[ https://issues.apache.org/jira/browse/SPARK-25355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531480#comment-17531480 ] Shrikant edited comment on SPARK-25355 at 5/4/22 5:17 AM: -- [~ga

[jira] [Updated] (SPARK-25355) Support --proxy-user for Spark on K8s

[ https://issues.apache.org/jira/browse/SPARK-25355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shrikant updated SPARK-25355: - Attachment: client.log > Support --proxy-user for Spark on K8s > - >

[jira] [Comment Edited] (SPARK-25355) Support --proxy-user for Spark on K8s

[ https://issues.apache.org/jira/browse/SPARK-25355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531480#comment-17531480 ] Shrikant edited comment on SPARK-25355 at 5/4/22 4:52 AM: -- [~ga

[jira] [Comment Edited] (SPARK-25355) Support --proxy-user for Spark on K8s

[ https://issues.apache.org/jira/browse/SPARK-25355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531480#comment-17531480 ] Shrikant edited comment on SPARK-25355 at 5/4/22 4:45 AM: -- [~ga

[jira] [Comment Edited] (SPARK-25355) Support --proxy-user for Spark on K8s

[ https://issues.apache.org/jira/browse/SPARK-25355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531480#comment-17531480 ] Shrikant edited comment on SPARK-25355 at 5/4/22 4:32 AM: -- [~ga

[jira] [Created] (SPARK-39097) Improve the test coverage for pyspark/taskcontext.py

pralabhkumar created SPARK-39097: Summary: Improve the test coverage for pyspark/taskcontext.py Key: SPARK-39097 URL: https://issues.apache.org/jira/browse/SPARK-39097 Project: Spark Issue T

[jira] [Commented] (SPARK-25355) Support --proxy-user for Spark on K8s

[ https://issues.apache.org/jira/browse/SPARK-25355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531480#comment-17531480 ] Shrikant commented on SPARK-25355: -- [~gaborgsomogyi] I have attached the full driver lo

[jira] [Commented] (SPARK-39097) Improve the test coverage for pyspark/taskcontext.py

[ https://issues.apache.org/jira/browse/SPARK-39097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531481#comment-17531481 ] pralabhkumar commented on SPARK-39097: -- I am working on this.  > Improve the test

[jira] [Commented] (SPARK-38807) Error when starting spark shell on Windows system

[ https://issues.apache.org/jira/browse/SPARK-38807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531478#comment-17531478 ] Apache Spark commented on SPARK-38807: -- User '1104056452' has created a pull reques

[jira] [Commented] (SPARK-38807) Error when starting spark shell on Windows system

[ https://issues.apache.org/jira/browse/SPARK-38807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531477#comment-17531477 ] Apache Spark commented on SPARK-38807: -- User '1104056452' has created a pull reques

[jira] [Updated] (SPARK-25355) Support --proxy-user for Spark on K8s

[ https://issues.apache.org/jira/browse/SPARK-25355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shrikant updated SPARK-25355: - Attachment: driver.log > Support --proxy-user for Spark on K8s > - >

[jira] [Assigned] (SPARK-39029) Improve the test coverage for pyspark/broadcast.py

[ https://issues.apache.org/jira/browse/SPARK-39029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-39029: Assignee: pralabhkumar > Improve the test coverage for pyspark/broadcast.py > ---

[jira] [Resolved] (SPARK-39029) Improve the test coverage for pyspark/broadcast.py

[ https://issues.apache.org/jira/browse/SPARK-39029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-39029. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36432 [https://gi

[jira] [Commented] (SPARK-39075) IncompatibleSchemaException when selecting data from table stored from a DataFrame in Avro format with BYTE/SHORT

[ https://issues.apache.org/jira/browse/SPARK-39075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531457#comment-17531457 ] Erik Krogen commented on SPARK-39075: - I looked more closely at the code snippet you

[jira] [Commented] (SPARK-38869) Respect Table capability `ACCEPT_ANY_SCHEMA` in default column resolution

[ https://issues.apache.org/jira/browse/SPARK-38869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531444#comment-17531444 ] Daniel commented on SPARK-38869: Thanks for pointing this out. We can skip adding new co

[jira] [Assigned] (SPARK-39096) Support MERGE commands with DEFAULT values

[ https://issues.apache.org/jira/browse/SPARK-39096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39096: Assignee: Apache Spark > Support MERGE commands with DEFAULT values > ---

[jira] [Assigned] (SPARK-39096) Support MERGE commands with DEFAULT values

[ https://issues.apache.org/jira/browse/SPARK-39096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39096: Assignee: (was: Apache Spark) > Support MERGE commands with DEFAULT values >

[jira] [Commented] (SPARK-39096) Support MERGE commands with DEFAULT values

[ https://issues.apache.org/jira/browse/SPARK-39096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531441#comment-17531441 ] Apache Spark commented on SPARK-39096: -- User 'dtenedor' has created a pull request

[jira] [Created] (SPARK-39096) Support MERGE commands with DEFAULT values

Daniel created SPARK-39096: -- Summary: Support MERGE commands with DEFAULT values Key: SPARK-39096 URL: https://issues.apache.org/jira/browse/SPARK-39096 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-39095) Adjust `GroupBy.std` to match pandas 1.4

[ https://issues.apache.org/jira/browse/SPARK-39095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531433#comment-17531433 ] Apache Spark commented on SPARK-39095: -- User 'xinrong-databricks' has created a pul

[jira] [Commented] (SPARK-39095) Adjust `GroupBy.std` to match pandas 1.4

[ https://issues.apache.org/jira/browse/SPARK-39095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531431#comment-17531431 ] Apache Spark commented on SPARK-39095: -- User 'xinrong-databricks' has created a pul

[jira] [Commented] (SPARK-39033) Support --proxy-user for Spark on K8s not working

[ https://issues.apache.org/jira/browse/SPARK-39033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531430#comment-17531430 ] Gabor Somogyi commented on SPARK-39033: --- Simply the logs are trash just like I've

[jira] [Assigned] (SPARK-39095) Adjust `GroupBy.std` to match pandas 1.4

[ https://issues.apache.org/jira/browse/SPARK-39095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39095: Assignee: Apache Spark > Adjust `GroupBy.std` to match pandas 1.4 > -

[jira] [Assigned] (SPARK-39095) Adjust `GroupBy.std` to match pandas 1.4

[ https://issues.apache.org/jira/browse/SPARK-39095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39095: Assignee: (was: Apache Spark) > Adjust `GroupBy.std` to match pandas 1.4 > --

[jira] [Created] (SPARK-39095) Adjust `GroupBy.std` to match pandas 1.4

Xinrong Meng created SPARK-39095: Summary: Adjust `GroupBy.std` to match pandas 1.4 Key: SPARK-39095 URL: https://issues.apache.org/jira/browse/SPARK-39095 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-39088) Add support for forwarding Spark History requests to live running driver when present

[ https://issues.apache.org/jira/browse/SPARK-39088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531429#comment-17531429 ] Apache Spark commented on SPARK-39088: -- User 'holdenk' has created a pull request f

[jira] [Assigned] (SPARK-39088) Add support for forwarding Spark History requests to live running driver when present

[ https://issues.apache.org/jira/browse/SPARK-39088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39088: Assignee: Apache Spark (was: Holden Karau) > Add support for forwarding Spark History re

[jira] [Commented] (SPARK-39088) Add support for forwarding Spark History requests to live running driver when present

[ https://issues.apache.org/jira/browse/SPARK-39088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531428#comment-17531428 ] Apache Spark commented on SPARK-39088: -- User 'holdenk' has created a pull request f

[jira] [Assigned] (SPARK-39088) Add support for forwarding Spark History requests to live running driver when present

[ https://issues.apache.org/jira/browse/SPARK-39088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39088: Assignee: Holden Karau (was: Apache Spark) > Add support for forwarding Spark History re

[jira] [Assigned] (SPARK-39093) Dividing interval by integral can result in codegen compilation error

[ https://issues.apache.org/jira/browse/SPARK-39093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39093: Assignee: (was: Apache Spark) > Dividing interval by integral can result in codegen c

[jira] [Commented] (SPARK-39093) Dividing interval by integral can result in codegen compilation error

[ https://issues.apache.org/jira/browse/SPARK-39093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531408#comment-17531408 ] Apache Spark commented on SPARK-39093: -- User 'bersprockets' has created a pull requ

[jira] [Commented] (SPARK-39093) Dividing interval by integral can result in codegen compilation error

[ https://issues.apache.org/jira/browse/SPARK-39093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531407#comment-17531407 ] Apache Spark commented on SPARK-39093: -- User 'bersprockets' has created a pull requ

[jira] [Assigned] (SPARK-39093) Dividing interval by integral can result in codegen compilation error

[ https://issues.apache.org/jira/browse/SPARK-39093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39093: Assignee: Apache Spark > Dividing interval by integral can result in codegen compilation

[jira] [Comment Edited] (SPARK-39075) IncompatibleSchemaException when selecting data from table stored from a DataFrame in Avro format with BYTE/SHORT

[ https://issues.apache.org/jira/browse/SPARK-39075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531398#comment-17531398 ] xsys edited comment on SPARK-39075 at 5/3/22 8:34 PM: -- Thanks for t

[jira] [Commented] (SPARK-39075) IncompatibleSchemaException when selecting data from table stored from a DataFrame in Avro format with BYTE/SHORT

[ https://issues.apache.org/jira/browse/SPARK-39075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531398#comment-17531398 ] xsys commented on SPARK-39075: -- Thanks for the response, Erik. I understand the concern. O

[jira] [Updated] (SPARK-39093) Dividing interval by integral can result in codegen compilation error

[ https://issues.apache.org/jira/browse/SPARK-39093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-39093: -- Affects Version/s: 3.2.1 > Dividing interval by integral can result in codegen compilation err

[jira] [Updated] (SPARK-39093) Dividing interval by integral can result in codegen compilation error

[ https://issues.apache.org/jira/browse/SPARK-39093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-39093: -- Affects Version/s: 3.3.0 > Dividing interval by integral can result in codegen compilation err

[jira] [Assigned] (SPARK-39091) SQL Expression traits don't compose due to nodePatterns being final.

[ https://issues.apache.org/jira/browse/SPARK-39091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39091: Assignee: Apache Spark > SQL Expression traits don't compose due to nodePatterns being fi

[jira] [Assigned] (SPARK-39091) SQL Expression traits don't compose due to nodePatterns being final.

[ https://issues.apache.org/jira/browse/SPARK-39091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39091: Assignee: (was: Apache Spark) > SQL Expression traits don't compose due to nodePatter

[jira] [Commented] (SPARK-39091) SQL Expression traits don't compose due to nodePatterns being final.

[ https://issues.apache.org/jira/browse/SPARK-39091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531285#comment-17531285 ] Apache Spark commented on SPARK-39091: -- User 'neilagupta' has created a pull reques

[jira] [Commented] (SPARK-39091) SQL Expression traits don't compose due to nodePatterns being final.

[ https://issues.apache.org/jira/browse/SPARK-39091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531282#comment-17531282 ] Neil Gupta commented on SPARK-39091: Tried to implement a fix - [https://github.com/

[jira] [Comment Edited] (SPARK-39091) SQL Expression traits don't compose due to nodePatterns being final.

[ https://issues.apache.org/jira/browse/SPARK-39091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531282#comment-17531282 ] Neil Gupta edited comment on SPARK-39091 at 5/3/22 4:28 PM:

[jira] [Updated] (SPARK-39093) Dividing interval by integral can result in codegen compilation error

[ https://issues.apache.org/jira/browse/SPARK-39093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-39093: -- Description: Assume this data: {noformat} create or replace temp view v1 as select * FROM VALU

[jira] [Created] (SPARK-39093) Dividing interval by integral can result in codegen compilation error

Bruce Robbins created SPARK-39093: - Summary: Dividing interval by integral can result in codegen compilation error Key: SPARK-39093 URL: https://issues.apache.org/jira/browse/SPARK-39093 Project: Spar

[jira] [Commented] (SPARK-37259) JDBC read is always going to wrap the query in a select statement

[ https://issues.apache.org/jira/browse/SPARK-37259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531185#comment-17531185 ] Apache Spark commented on SPARK-37259: -- User 'peter-toth' has created a pull reques

[jira] [Commented] (SPARK-39087) Improve messages of error classes

[ https://issues.apache.org/jira/browse/SPARK-39087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531179#comment-17531179 ] Apache Spark commented on SPARK-39087: -- User 'MaxGekk' has created a pull request f

[jira] [Commented] (SPARK-39058) Add `getInputSignature` and `getOutputSignature` APIs for spark ML models/transformers

[ https://issues.apache.org/jira/browse/SPARK-39058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531100#comment-17531100 ] zhengruifeng commented on SPARK-39058: -- [~weichenxu123]  I can help reivew. BTW, is

[jira] [Assigned] (SPARK-39092) Propagate Empty Partitions

[ https://issues.apache.org/jira/browse/SPARK-39092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39092: Assignee: Apache Spark > Propagate Empty Partitions > -- > >

[jira] [Commented] (SPARK-39092) Propagate Empty Partitions

[ https://issues.apache.org/jira/browse/SPARK-39092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531099#comment-17531099 ] Apache Spark commented on SPARK-39092: -- User 'zhengruifeng' has created a pull requ

[jira] [Commented] (SPARK-39092) Propagate Empty Partitions

[ https://issues.apache.org/jira/browse/SPARK-39092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531098#comment-17531098 ] Apache Spark commented on SPARK-39092: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-39092) Propagate Empty Partitions

[ https://issues.apache.org/jira/browse/SPARK-39092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39092: Assignee: (was: Apache Spark) > Propagate Empty Partitions >

[jira] [Created] (SPARK-39092) Propagate Empty Partitions

zhengruifeng created SPARK-39092: Summary: Propagate Empty Partitions Key: SPARK-39092 URL: https://issues.apache.org/jira/browse/SPARK-39092 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-30661) KMeans blockify input vectors

[ https://issues.apache.org/jira/browse/SPARK-30661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-30661: - Affects Version/s: 3.4.0 (was: 3.0.0) > KMeans blockify input vectors

[jira] [Updated] (SPARK-30661) KMeans blockify input vectors

[ https://issues.apache.org/jira/browse/SPARK-30661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-30661: - Priority: Major (was: Minor) > KMeans blockify input vectors > - >

[jira] [Commented] (SPARK-30661) KMeans blockify input vectors

[ https://issues.apache.org/jira/browse/SPARK-30661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531092#comment-17531092 ] Apache Spark commented on SPARK-30661: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-30661) KMeans blockify input vectors

[ https://issues.apache.org/jira/browse/SPARK-30661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-30661: Assignee: zhengruifeng (was: Apache Spark) > KMeans blockify input vectors > ---

[jira] [Assigned] (SPARK-30661) KMeans blockify input vectors

[ https://issues.apache.org/jira/browse/SPARK-30661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-30661: Assignee: Apache Spark (was: zhengruifeng) > KMeans blockify input vectors > ---

[jira] [Assigned] (SPARK-30661) KMeans blockify input vectors

[ https://issues.apache.org/jira/browse/SPARK-30661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-30661: Assignee: Apache Spark (was: zhengruifeng) > KMeans blockify input vectors > ---

[jira] [Commented] (SPARK-35320) from_json cannot parse maps with timestamp as key

[ https://issues.apache.org/jira/browse/SPARK-35320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531083#comment-17531083 ] Apache Spark commented on SPARK-35320: -- User 'itholic' has created a pull request f