[jira] [Commented] (SPARK-42714) Sparksql temporary file conflict

2023-03-07 Thread hao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697760#comment-17697760 ] hao commented on SPARK-42714: - This problem will cause the task to throw the problem of deleting the current

[jira] [Created] (SPARK-42714) Sparksql temporary file conflict

2023-03-07 Thread hao (Jira)
hao created SPARK-42714: --- Summary: Sparksql temporary file conflict Key: SPARK-42714 URL: https://issues.apache.org/jira/browse/SPARK-42714 Project: Spark Issue Type: Bug Components: SQL

[jira] [Commented] (SPARK-42713) Add '__getattr__' and '__getitem__' of DataFrame and Column to API reference

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697756#comment-17697756 ] Apache Spark commented on SPARK-42713: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-42713) Add '__getattr__' and '__getitem__' of DataFrame and Column to API reference

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42713: Assignee: Apache Spark > Add '__getattr__' and '__getitem__' of DataFrame and Column to

[jira] [Assigned] (SPARK-42713) Add '__getattr__' and '__getitem__' of DataFrame and Column to API reference

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42713: Assignee: (was: Apache Spark) > Add '__getattr__' and '__getitem__' of DataFrame and

[jira] [Commented] (SPARK-42713) Add '__getattr__' and '__getitem__' of DataFrame and Column to API reference

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697755#comment-17697755 ] Apache Spark commented on SPARK-42713: -- User 'zhengruifeng' has created a pull request for this

[jira] [Created] (SPARK-42713) Add '__getattr__' and '__getitem__' of DataFrame and Column to API reference

2023-03-07 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-42713: - Summary: Add '__getattr__' and '__getitem__' of DataFrame and Column to API reference Key: SPARK-42713 URL: https://issues.apache.org/jira/browse/SPARK-42713

[jira] [Assigned] (SPARK-42712) Improve docstring of mapInPandas and mapInArrow

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42712: Assignee: Apache Spark > Improve docstring of mapInPandas and mapInArrow >

[jira] [Commented] (SPARK-42712) Improve docstring of mapInPandas and mapInArrow

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697752#comment-17697752 ] Apache Spark commented on SPARK-42712: -- User 'xinrong-meng' has created a pull request for this

[jira] [Assigned] (SPARK-42712) Improve docstring of mapInPandas and mapInArrow

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42712: Assignee: (was: Apache Spark) > Improve docstring of mapInPandas and mapInArrow >

[jira] [Assigned] (SPARK-42710) Rename FrameMap proto to MapPartitions

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42710: Assignee: Apache Spark > Rename FrameMap proto to MapPartitions >

[jira] [Assigned] (SPARK-42710) Rename FrameMap proto to MapPartitions

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42710: Assignee: (was: Apache Spark) > Rename FrameMap proto to MapPartitions >

[jira] [Commented] (SPARK-42710) Rename FrameMap proto to MapPartitions

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697748#comment-17697748 ] Apache Spark commented on SPARK-42710: -- User 'xinrong-meng' has created a pull request for this

[jira] [Commented] (SPARK-42711) build/sbt usage error messages about java-home

2023-03-07 Thread Liang Yan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697745#comment-17697745 ] Liang Yan commented on SPARK-42711: --- I am preparing my PR for this issue. Please assign it to me. >

[jira] [Created] (SPARK-42712) Improve docstring of mapInPandas and mapInArrow

2023-03-07 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42712: Summary: Improve docstring of mapInPandas and mapInArrow Key: SPARK-42712 URL: https://issues.apache.org/jira/browse/SPARK-42712 Project: Spark Issue Type:

[jira] [Created] (SPARK-42711) build/sbt usage error messages about java-home

2023-03-07 Thread Liang Yan (Jira)
Liang Yan created SPARK-42711: - Summary: build/sbt usage error messages about java-home Key: SPARK-42711 URL: https://issues.apache.org/jira/browse/SPARK-42711 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-42710) Rename FrameMap proto to MapPartitions

2023-03-07 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42710: Summary: Rename FrameMap proto to MapPartitions Key: SPARK-42710 URL: https://issues.apache.org/jira/browse/SPARK-42710 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-42643) Register Java (aggregate) user-defined functions

2023-03-07 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-42643. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 40244

[jira] [Assigned] (SPARK-42643) Register Java (aggregate) user-defined functions

2023-03-07 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng reassigned SPARK-42643: Assignee: Xinrong Meng > Register Java (aggregate) user-defined functions >

[jira] [Comment Edited] (SPARK-42411) Better support for Istio service mesh while running Spark on Kubernetes

2023-03-07 Thread Puneet (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697737#comment-17697737 ] Puneet edited comment on SPARK-42411 at 3/8/23 6:20 AM: True, though there might

[jira] [Commented] (SPARK-42411) Better support for Istio service mesh while running Spark on Kubernetes

2023-03-07 Thread Puneet (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697737#comment-17697737 ] Puneet commented on SPARK-42411: True, though there might be value in having an option to delegate spark

[jira] [Assigned] (SPARK-42709) Do not rely on __file__

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42709: Assignee: (was: Apache Spark) > Do not rely on __file__ > --- >

[jira] [Assigned] (SPARK-42709) Do not rely on __file__

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42709: Assignee: Apache Spark > Do not rely on __file__ > --- > >

[jira] [Commented] (SPARK-42709) Do not rely on __file__

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697734#comment-17697734 ] Apache Spark commented on SPARK-42709: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Created] (SPARK-42709) Do not rely on __file__

2023-03-07 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42709: Summary: Do not rely on __file__ Key: SPARK-42709 URL: https://issues.apache.org/jira/browse/SPARK-42709 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-42266) Local mode should work with IPython

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697727#comment-17697727 ] Apache Spark commented on SPARK-42266: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-42266) Local mode should work with IPython

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42266: Assignee: Apache Spark > Local mode should work with IPython >

[jira] [Assigned] (SPARK-42266) Local mode should work with IPython

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42266: Assignee: (was: Apache Spark) > Local mode should work with IPython >

[jira] [Resolved] (SPARK-42707) Remove experimental warning in developer documentation

2023-03-07 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-42707. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 40325

[jira] [Assigned] (SPARK-42707) Remove experimental warning in developer documentation

2023-03-07 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-42707: - Assignee: Hyukjin Kwon > Remove experimental warning in developer documentation >

[jira] [Resolved] (SPARK-42700) Add h2 as test dependency of connect-server module

2023-03-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42700. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 40317

[jira] [Assigned] (SPARK-42700) Add h2 as test dependency of connect-server module

2023-03-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42700: Assignee: Yang Jie > Add h2 as test dependency of connect-server module >

[jira] [Updated] (SPARK-42623) parameter markers not blocked in DDL

2023-03-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-42623: Fix Version/s: (was: 3.4.0) > parameter markers not blocked in DDL >

[jira] [Commented] (SPARK-42623) parameter markers not blocked in DDL

2023-03-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697721#comment-17697721 ] Wenchen Fan commented on SPARK-42623: - I think the problem occurs when we store the original SQL

[jira] [Assigned] (SPARK-42708) The generated protobuf java file is too large

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42708: Assignee: (was: Apache Spark) > The generated protobuf java file is too large >

[jira] [Commented] (SPARK-42708) The generated protobuf java file is too large

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697719#comment-17697719 ] Apache Spark commented on SPARK-42708: -- User 'Hisoka-X' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42708) The generated protobuf java file is too large

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42708: Assignee: Apache Spark > The generated protobuf java file is too large >

[jira] [Assigned] (SPARK-42681) Relax ordering constraint for ALTER TABLE ADD|REPLACE column options

2023-03-07 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-42681: -- Assignee: Vitalii Li > Relax ordering constraint for ALTER TABLE ADD|REPLACE column

[jira] [Resolved] (SPARK-42681) Relax ordering constraint for ALTER TABLE ADD|REPLACE column options

2023-03-07 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-42681. Resolution: Fixed Issue resolved by pull request 40295

[jira] [Created] (SPARK-42708) The generated protobuf java file is too large

2023-03-07 Thread Jia Fan (Jira)
Jia Fan created SPARK-42708: --- Summary: The generated protobuf java file is too large Key: SPARK-42708 URL: https://issues.apache.org/jira/browse/SPARK-42708 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-42707) Remove experimental warning in developer documentation

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697717#comment-17697717 ] Apache Spark commented on SPARK-42707: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-42707) Remove experimental warning in developer documentation

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42707: Assignee: Apache Spark > Remove experimental warning in developer documentation >

[jira] [Assigned] (SPARK-42707) Remove experimental warning in developer documentation

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42707: Assignee: (was: Apache Spark) > Remove experimental warning in developer

[jira] [Assigned] (SPARK-39399) proxy-user not working for Spark on k8s in cluster deploy mode

2023-03-07 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-39399: Assignee: Shrikant Prasad > proxy-user not working for Spark on k8s in cluster deploy mode >

[jira] [Resolved] (SPARK-39399) proxy-user not working for Spark on k8s in cluster deploy mode

2023-03-07 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-39399. -- Fix Version/s: 3.3.3 3.2.4 3.4.0 Resolution: Fixed Issue

[jira] [Created] (SPARK-42707) Remove experimental warning in developer documentation

2023-03-07 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42707: Summary: Remove experimental warning in developer documentation Key: SPARK-42707 URL: https://issues.apache.org/jira/browse/SPARK-42707 Project: Spark Issue

[jira] [Assigned] (SPARK-42705) SparkSession.sql doesn't return values from commands.

2023-03-07 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-42705: - Assignee: Takuya Ueshin > SparkSession.sql doesn't return values from commands. >

[jira] [Resolved] (SPARK-42705) SparkSession.sql doesn't return values from commands.

2023-03-07 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-42705. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 40323

[jira] [Updated] (SPARK-42650) insert overwrite table will casue table location lost if java.lang.ArithmeticException is thrown

2023-03-07 Thread kevinshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kevinshin updated SPARK-42650: -- Summary: insert overwrite table will casue table location lost if java.lang.ArithmeticException is

[jira] [Commented] (SPARK-42496) Introduction Spark Connect at main page.

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697699#comment-17697699 ] Apache Spark commented on SPARK-42496: -- User 'allanf-db' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-42703) How to use Fair Scheduler Pools

2023-03-07 Thread LiJie2023 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697679#comment-17697679 ] LiJie2023 edited comment on SPARK-42703 at 3/8/23 2:22 AM: --- [~bjornjorgensen] 

[jira] [Commented] (SPARK-42694) Data duplication and loss occur after executing 'insert overwrite...' in Spark 3.1.1

2023-03-07 Thread FengZhou (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697682#comment-17697682 ] FengZhou commented on SPARK-42694: -- [~yumwang]  This version has been running in production environment

[jira] [Commented] (SPARK-42694) Data duplication and loss occur after executing 'insert overwrite...' in Spark 3.1.1

2023-03-07 Thread FengZhou (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697683#comment-17697683 ] FengZhou commented on SPARK-42694: -- [~bjornjorgensen]  As the current upgrade would have a significant

[jira] [Comment Edited] (SPARK-42703) How to use Fair Scheduler Pools

2023-03-07 Thread LiJie2023 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697679#comment-17697679 ] LiJie2023 edited comment on SPARK-42703 at 3/8/23 1:57 AM: --- Sorry, I can't

[jira] [Commented] (SPARK-42703) How to use Fair Scheduler Pools

2023-03-07 Thread LiJie2023 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697679#comment-17697679 ] LiJie2023 commented on SPARK-42703: --- Sorry, I can't find the answer in StackOverflow and Mailing

[jira] [Updated] (SPARK-42703) How to use Fair Scheduler Pools

2023-03-07 Thread LiJie2023 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LiJie2023 updated SPARK-42703: -- Attachment: image-2023-03-08-09-53-35-867.png > How to use Fair Scheduler Pools >

[jira] [Commented] (SPARK-42706) List the error class to user-facing documentation.

2023-03-07 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697665#comment-17697665 ] Haejoon Lee commented on SPARK-42706: - I'm working on this > List the error class to user-facing

[jira] [Created] (SPARK-42706) List the error class to user-facing documentation.

2023-03-07 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-42706: --- Summary: List the error class to user-facing documentation. Key: SPARK-42706 URL: https://issues.apache.org/jira/browse/SPARK-42706 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-42022) createDataFrame should autogenerate missing column names

2023-03-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42022: Assignee: Hyukjin Kwon > createDataFrame should autogenerate missing column names >

[jira] [Resolved] (SPARK-42022) createDataFrame should autogenerate missing column names

2023-03-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42022. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 40310

[jira] [Assigned] (SPARK-42705) SparkSession.sql doesn't return values from commands.

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42705: Assignee: Apache Spark > SparkSession.sql doesn't return values from commands. >

[jira] [Commented] (SPARK-42705) SparkSession.sql doesn't return values from commands.

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697660#comment-17697660 ] Apache Spark commented on SPARK-42705: -- User 'ueshin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42705) SparkSession.sql doesn't return values from commands.

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42705: Assignee: (was: Apache Spark) > SparkSession.sql doesn't return values from

[jira] [Commented] (SPARK-41775) Implement training functions as input

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697658#comment-17697658 ] Apache Spark commented on SPARK-41775: -- User 'rithwik-db' has created a pull request for this

[jira] [Created] (SPARK-42705) SparkSession.sql doesn't return values from commands.

2023-03-07 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-42705: - Summary: SparkSession.sql doesn't return values from commands. Key: SPARK-42705 URL: https://issues.apache.org/jira/browse/SPARK-42705 Project: Spark

[jira] [Commented] (SPARK-42702) Support parameterized CTE

2023-03-07 Thread Entong Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697624#comment-17697624 ] Entong Shen commented on SPARK-42702: - And subqueries > Support parameterized CTE >

[jira] [Assigned] (SPARK-42591) Document SS guide doc for introducing watermark propagation among operators

2023-03-07 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-42591: Assignee: Jungtaek Lim > Document SS guide doc for introducing watermark propagation

[jira] [Resolved] (SPARK-42591) Document SS guide doc for introducing watermark propagation among operators

2023-03-07 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-42591. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40215

[jira] [Commented] (SPARK-42694) Data duplication and loss occur after executing 'insert overwrite...' in Spark 3.1.1

2023-03-07 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-42694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697614#comment-17697614 ] Bjørn Jørgensen commented on SPARK-42694: - Spark 3.1 [is

[jira] [Commented] (SPARK-42703) How to use Fair Scheduler Pools

2023-03-07 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-42703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697612#comment-17697612 ] Bjørn Jørgensen commented on SPARK-42703: - Hi [~lijie1912] we have a mail list

[jira] [Assigned] (SPARK-42704) SubqueryAlias should propagate metadata columns its child already selects

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42704: Assignee: Apache Spark > SubqueryAlias should propagate metadata columns its child

[jira] [Commented] (SPARK-42704) SubqueryAlias should propagate metadata columns its child already selects

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697565#comment-17697565 ] Apache Spark commented on SPARK-42704: -- User 'ryan-johnson-databricks' has created a pull request

[jira] [Assigned] (SPARK-42704) SubqueryAlias should propagate metadata columns its child already selects

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42704: Assignee: (was: Apache Spark) > SubqueryAlias should propagate metadata columns its

[jira] [Updated] (SPARK-42704) SubqueryAlias should propagate metadata columns its child already selects

2023-03-07 Thread Ryan Johnson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Johnson updated SPARK-42704: - Description: The `AddMetadataColumns` analyzer rule intends to make resolve available metadata

[jira] [Created] (SPARK-42704) SubqueryAlias should propagate metadata columns its child already selects

2023-03-07 Thread Ryan Johnson (Jira)
Ryan Johnson created SPARK-42704: Summary: SubqueryAlias should propagate metadata columns its child already selects Key: SPARK-42704 URL: https://issues.apache.org/jira/browse/SPARK-42704 Project:

[jira] [Updated] (SPARK-42683) Automatically rename metadata columns that conflict with data schema columns

2023-03-07 Thread Ryan Johnson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Johnson updated SPARK-42683: - Target Version/s: 3.5.0 (was: 3.4.0) > Automatically rename metadata columns that conflict

[jira] [Commented] (SPARK-42690) Implement CSV/JSON parsing funcions

2023-03-07 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697489#comment-17697489 ] Yang Jie commented on SPARK-42690: -- {code:java} message Parse { // (Required) Input relation to

[jira] [Updated] (SPARK-42703) How to use Fair Scheduler Pools

2023-03-07 Thread LiJie2023 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LiJie2023 updated SPARK-42703: -- Description: I have two questions to ask: #   I wrote a demo referring to the official website, but

[jira] [Created] (SPARK-42703) How to use Fair Scheduler Pools

2023-03-07 Thread LiJie2023 (Jira)
LiJie2023 created SPARK-42703: - Summary: How to use Fair Scheduler Pools Key: SPARK-42703 URL: https://issues.apache.org/jira/browse/SPARK-42703 Project: Spark Issue Type: Question

[jira] [Assigned] (SPARK-42692) Implement Dataset.toJson

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42692: Assignee: Apache Spark > Implement Dataset.toJson > > >

[jira] [Commented] (SPARK-42692) Implement Dataset.toJson

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697467#comment-17697467 ] Apache Spark commented on SPARK-42692: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-42692) Implement Dataset.toJson

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42692: Assignee: (was: Apache Spark) > Implement Dataset.toJson >

[jira] [Assigned] (SPARK-42702) Support parameterized CTE

2023-03-07 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-42702: Assignee: Max Gekk > Support parameterized CTE > - > >

[jira] [Created] (SPARK-42702) Support parameterized CTE

2023-03-07 Thread Max Gekk (Jira)
Max Gekk created SPARK-42702: Summary: Support parameterized CTE Key: SPARK-42702 URL: https://issues.apache.org/jira/browse/SPARK-42702 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-42551) Support subexpression elimination in FilterExec and JoinExec

2023-03-07 Thread Wan Kun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wan Kun updated SPARK-42551: Summary: Support subexpression elimination in FilterExec and JoinExec (was: Support subexpression

[jira] [Commented] (SPARK-42701) Add the try_aes_decrypt() function

2023-03-07 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697429#comment-17697429 ] Max Gekk commented on SPARK-42701: -- [~panbingkun] [~ivoson] [~xleesf] [~YActs] [~lvshaokang] 

[jira] [Commented] (SPARK-42579) Extend function.lit() to match Literal.apply()

2023-03-07 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697426#comment-17697426 ] Yang Jie commented on SPARK-42579: -- [~hvanhovell] should we make this one as `Fixed`,  Other work

[jira] [Updated] (SPARK-42701) Add the try_aes_decrypt() function

2023-03-07 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-42701: - Labels: starter (was: ) > Add the try_aes_decrypt() function > -- > >

[jira] [Commented] (SPARK-42656) Spark Connect Scala Client Shell Script

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697423#comment-17697423 ] Apache Spark commented on SPARK-42656: -- User 'LuciferYang' has created a pull request for this

[jira] [Created] (SPARK-42701) Add the try_aes_decrypt() function

2023-03-07 Thread Max Gekk (Jira)
Max Gekk created SPARK-42701: Summary: Add the try_aes_decrypt() function Key: SPARK-42701 URL: https://issues.apache.org/jira/browse/SPARK-42701 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-42700) Add h2 as test dependency of connect-server module

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697417#comment-17697417 ] Apache Spark commented on SPARK-42700: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-42700) Add h2 as test dependency of connect-server module

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42700: Assignee: (was: Apache Spark) > Add h2 as test dependency of connect-server module >

[jira] [Assigned] (SPARK-42700) Add h2 as test dependency of connect-server module

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42700: Assignee: Apache Spark > Add h2 as test dependency of connect-server module >

[jira] [Assigned] (SPARK-42679) createDataFrame doesn't work with non-nullable schema.

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42679: Assignee: Apache Spark > createDataFrame doesn't work with non-nullable schema. >

[jira] [Commented] (SPARK-42679) createDataFrame doesn't work with non-nullable schema.

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697412#comment-17697412 ] Apache Spark commented on SPARK-42679: -- User 'panbingkun' has created a pull request for this

[jira] [Assigned] (SPARK-42679) createDataFrame doesn't work with non-nullable schema.

2023-03-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42679: Assignee: (was: Apache Spark) > createDataFrame doesn't work with non-nullable

[jira] [Created] (SPARK-42700) Add h2 as test dependency of connect-server module

2023-03-07 Thread Yang Jie (Jira)
Yang Jie created SPARK-42700: Summary: Add h2 as test dependency of connect-server module Key: SPARK-42700 URL: https://issues.apache.org/jira/browse/SPARK-42700 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-42356) Cannot resolve orderby attributes in DISTINCT

2023-03-07 Thread Zhou Chong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697390#comment-17697390 ] Zhou Chong commented on SPARK-42356: Hi [~blackpig]  could I know more detail reasons about this

[jira] [Assigned] (SPARK-42696) Speed up parquet reading with Java Vector API

2023-03-07 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-42696: --- Assignee: jiangjiguang0719 (was: Yuming Wang) > Speed up parquet reading with Java Vector

[jira] [Assigned] (SPARK-42696) Speed up parquet reading with Java Vector API

2023-03-07 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-42696: --- Assignee: Yuming Wang > Speed up parquet reading with Java Vector API >

  1   2   >