[jira] [Commented] (SPARK-27442) ParquetFileFormat fails to read column named with invalid characters

2022-01-17 Thread Pietro Cerutti (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477632#comment-17477632 ] Pietro Cerutti commented on SPARK-27442: Using the Apache Arrow C++ API, see

[jira] [Commented] (SPARK-37951) Refactor ImageFileFormatSuite

2022-01-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477619#comment-17477619 ] Apache Spark commented on SPARK-37951: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-37951) Refactor ImageFileFormatSuite

2022-01-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37951: Assignee: (was: Apache Spark) > Refactor ImageFileFormatSuite >

[jira] [Assigned] (SPARK-37951) Refactor ImageFileFormatSuite

2022-01-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37951: Assignee: Apache Spark > Refactor ImageFileFormatSuite > - >

[jira] [Commented] (SPARK-37154) Inline type hints for python/pyspark/rdd.py

2022-01-17 Thread dch nguyen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477611#comment-17477611 ] dch nguyen commented on SPARK-37154: Can I go for this? [~byronhsu]  > Inline type hints for

[jira] [Updated] (SPARK-37951) Refactor ImageFileFormatSuite

2022-01-17 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-37951: -- Description: Not use standard API, sometimes failed, optimize it. > Refactor ImageFileFormatSuite >

[jira] [Created] (SPARK-37951) Refactor ImageFileFormatSuite

2022-01-17 Thread angerszhu (Jira)
angerszhu created SPARK-37951: - Summary: Refactor ImageFileFormatSuite Key: SPARK-37951 URL: https://issues.apache.org/jira/browse/SPARK-37951 Project: Spark Issue Type: Task

[jira] [Commented] (SPARK-37939) Use error classes in the parsing errors of properties

2022-01-17 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477592#comment-17477592 ] Max Gekk commented on SPARK-37939: -- [~senthh] Please, finish one task before grabbing all of them. >

[jira] [Resolved] (SPARK-37768) Schema pruning for the metadata struct

2022-01-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37768. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35147

[jira] [Assigned] (SPARK-37768) Schema pruning for the metadata struct

2022-01-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37768: --- Assignee: Yaohua Zhao > Schema pruning for the metadata struct >

[jira] [Comment Edited] (SPARK-37930) Fix DataFrame select subset with duplicated columns

2022-01-17 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477581#comment-17477581 ] Yikun Jiang edited comment on SPARK-37930 at 1/18/22, 5:46 AM: --- [~dchvn]

[jira] [Assigned] (SPARK-37906) spark-sql should not pass last simple comment to backend

2022-01-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37906: --- Assignee: angerszhu > spark-sql should not pass last simple comment to backend >

[jira] [Resolved] (SPARK-37906) spark-sql should not pass last simple comment to backend

2022-01-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37906. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35206

[jira] [Comment Edited] (SPARK-37930) Fix DataFrame select subset with duplicated columns

2022-01-17 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477581#comment-17477581 ] Yikun Jiang edited comment on SPARK-37930 at 1/18/22, 5:41 AM: --- [~dchvn]

[jira] [Comment Edited] (SPARK-37930) Fix DataFrame select subset with duplicated columns

2022-01-17 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477581#comment-17477581 ] Yikun Jiang edited comment on SPARK-37930 at 1/18/22, 5:34 AM: --- [~dchvn]

[jira] [Comment Edited] (SPARK-37930) Fix DataFrame select subset with duplicated columns

2022-01-17 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477581#comment-17477581 ] Yikun Jiang edited comment on SPARK-37930 at 1/18/22, 5:30 AM: --- [~dchvn]

[jira] [Commented] (SPARK-37941) Use error classes in the compilation errors of casting

2022-01-17 Thread huangtengfei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477582#comment-17477582 ] huangtengfei commented on SPARK-37941: -- I will work on this. Thanks [~maxgekk] > Use error classes

[jira] [Commented] (SPARK-37930) Fix DataFrame select subset with duplicated columns

2022-01-17 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477581#comment-17477581 ] Yikun Jiang commented on SPARK-37930: - [~dchvn] Thanks for investigation Also update the pandas

[jira] [Assigned] (SPARK-37878) Migrate SHOW CREATE TABLE to use v2 command by default

2022-01-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37878: --- Assignee: PengLei > Migrate SHOW CREATE TABLE to use v2 command by default >

[jira] [Resolved] (SPARK-37878) Migrate SHOW CREATE TABLE to use v2 command by default

2022-01-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37878. - Resolution: Fixed Issue resolved by pull request 35204

[jira] (SPARK-37930) Fix DataFrame select subset with duplicated columns

2022-01-17 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37930 ] Yikun Jiang deleted comment on SPARK-37930: - was (Author: yikunkero): https://github.com/apache/spark/blob/df7447bc62052e3d7391ba23d7220fb8c9b923fd/python/pyspark/pandas/frame.py#L12268

[jira] [Comment Edited] (SPARK-37930) Fix DataFrame select subset with duplicated columns

2022-01-17 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477561#comment-17477561 ] Yikun Jiang edited comment on SPARK-37930 at 1/18/22, 4:31 AM: ---

[jira] [Commented] (SPARK-37930) Fix DataFrame select subset with duplicated columns

2022-01-17 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477561#comment-17477561 ] Yikun Jiang commented on SPARK-37930: -

[jira] [Commented] (SPARK-37910) Spark executor self-exiting due to driver disassociated in Kubernetes with client deploy-mode

2022-01-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477560#comment-17477560 ] Dongjoon Hyun commented on SPARK-37910: --- Did you check the official Apache Spark K8s

[jira] [Commented] (SPARK-37903) Replace string_typehints with get_type_hints.

2022-01-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477559#comment-17477559 ] Apache Spark commented on SPARK-37903: -- User 'dchvn' has created a pull request for this issue:

[jira] [Created] (SPARK-37950) Take EXTERNAL as a reserved table property

2022-01-17 Thread PengLei (Jira)
PengLei created SPARK-37950: --- Summary: Take EXTERNAL as a reserved table property Key: SPARK-37950 URL: https://issues.apache.org/jira/browse/SPARK-37950 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-37949) Improve Rebalance statistics estimation

2022-01-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477556#comment-17477556 ] Apache Spark commented on SPARK-37949: -- User 'ulysses-you' has created a pull request for this

[jira] [Assigned] (SPARK-37949) Improve Rebalance statistics estimation

2022-01-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37949: Assignee: (was: Apache Spark) > Improve Rebalance statistics estimation >

[jira] [Assigned] (SPARK-37949) Improve Rebalance statistics estimation

2022-01-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37949: Assignee: Apache Spark > Improve Rebalance statistics estimation >

[jira] [Commented] (SPARK-37949) Improve Rebalance statistics estimation

2022-01-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477555#comment-17477555 ] Apache Spark commented on SPARK-37949: -- User 'ulysses-you' has created a pull request for this

[jira] [Created] (SPARK-37949) Improve Rebalance statistics estimation

2022-01-17 Thread XiDuo You (Jira)
XiDuo You created SPARK-37949: - Summary: Improve Rebalance statistics estimation Key: SPARK-37949 URL: https://issues.apache.org/jira/browse/SPARK-37949 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-37944) Use error classes in the execution errors of casting

2022-01-17 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477551#comment-17477551 ] Senthil Kumar commented on SPARK-37944: --- I will work on this > Use error classes in the execution

[jira] [Commented] (SPARK-37945) Use error classes in the execution errors of arithmetic ops

2022-01-17 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477550#comment-17477550 ] Senthil Kumar commented on SPARK-37945: --- I will work on this > Use error classes in the execution

[jira] [Commented] (SPARK-37940) Use error classes in the compilation errors of partitions

2022-01-17 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477549#comment-17477549 ] Senthil Kumar commented on SPARK-37940: --- I will work on this > Use error classes in the

[jira] [Commented] (SPARK-37939) Use error classes in the parsing errors of properties

2022-01-17 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477548#comment-17477548 ] Senthil Kumar commented on SPARK-37939: --- I will work on this > Use error classes in the parsing

[jira] [Commented] (SPARK-27337) QueryExecutionListener never cleans up listeners from the bus after SparkSession is cleared

2022-01-17 Thread Stanislav Savulchik (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477546#comment-17477546 ] Stanislav Savulchik commented on SPARK-27337: - [~vinooganesh] thanks for the response. I

[jira] [Commented] (SPARK-37936) Use error classes in the parsing errors of intervals

2022-01-17 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477544#comment-17477544 ] Senthil Kumar commented on SPARK-37936: --- Working on this > Use error classes in the parsing

[jira] [Commented] (SPARK-37930) Fix DataFrame select subset with duplicated columns

2022-01-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477543#comment-17477543 ] Hyukjin Kwon commented on SPARK-37930: -- Thanks [~dchvn]! > Fix DataFrame select subset with

[jira] [Commented] (SPARK-37938) Use error classes in the parsing errors of partitions

2022-01-17 Thread leesf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477538#comment-17477538 ] leesf commented on SPARK-37938: --- [~maxgekk] would you please assign the ticket to me, I can work on this

[jira] [Updated] (SPARK-37948) Disable mapreduce.fileoutputcommitter.algorithm.version=2 by default

2022-01-17 Thread hujiahua (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hujiahua updated SPARK-37948: - Summary: Disable mapreduce.fileoutputcommitter.algorithm.version=2 by default (was: Disable

[jira] [Created] (SPARK-37948) Disable spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=2 by default

2022-01-17 Thread hujiahua (Jira)
hujiahua created SPARK-37948: Summary: Disable spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=2 by default Key: SPARK-37948 URL: https://issues.apache.org/jira/browse/SPARK-37948 Project:

[jira] [Commented] (SPARK-37930) Fix DataFrame select subset with duplicated columns

2022-01-17 Thread dch nguyen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477529#comment-17477529 ] dch nguyen commented on SPARK-37930: I'm working on this. Thanks > Fix DataFrame select subset with

[jira] [Commented] (SPARK-37930) Fix DataFrame select subset with duplicated columns

2022-01-17 Thread dch nguyen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477528#comment-17477528 ] dch nguyen commented on SPARK-37930: {code:java} >>> import pandas as pd >>> pdf =

[jira] [Commented] (SPARK-37930) Fix DataFrame select subset with duplicated columns

2022-01-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477520#comment-17477520 ] Hyukjin Kwon commented on SPARK-37930: -- are you working on this? [~dchvn] [~yikunkero]? > Fix

[jira] [Commented] (SPARK-37930) Fix DataFrame select subset with duplicated columns

2022-01-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477519#comment-17477519 ] Hyukjin Kwon commented on SPARK-37930: -- Interesting. we should fix this. cc

[jira] [Commented] (SPARK-37926) toPandas presicion doesn't match pandas, and would cause error on some case

2022-01-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477517#comment-17477517 ] Hyukjin Kwon commented on SPARK-37926: -- [~mithril], the results might be slightly different between

[jira] [Assigned] (SPARK-37712) Spark request yarn cluster metrics slow and cause unnecessary delay

2022-01-17 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-37712: --- Assignee: angerszhu > Spark request yarn cluster metrics slow and cause

[jira] [Resolved] (SPARK-37712) Spark request yarn cluster metrics slow and cause unnecessary delay

2022-01-17 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-37712. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Commented] (SPARK-27442) ParquetFileFormat fails to read column named with invalid characters

2022-01-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477493#comment-17477493 ] Hyukjin Kwon commented on SPARK-27442: -- I commented in the PR, see

[jira] [Commented] (SPARK-32165) SessionState leaks SparkListener with multiple SparkSession

2022-01-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477488#comment-17477488 ] Apache Spark commented on SPARK-32165: -- User 'dnskr' has created a pull request for this issue:

[jira] [Resolved] (SPARK-37498) test_reuse_worker_of_parallelize_range is flaky

2022-01-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37498. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35228

[jira] [Assigned] (SPARK-37498) test_reuse_worker_of_parallelize_range is flaky

2022-01-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-37498: Assignee: Yikun Jiang > test_reuse_worker_of_parallelize_range is flaky >

[jira] [Commented] (SPARK-37857) Any depth search not working in get_json_object ($..foo)

2022-01-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477484#comment-17477484 ] Hyukjin Kwon commented on SPARK-37857: -- Hm, if there's nothing breaking, or no performance penalty,

[jira] [Commented] (SPARK-27337) QueryExecutionListener never cleans up listeners from the bus after SparkSession is cleared

2022-01-17 Thread Vinoo Ganesh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477479#comment-17477479 ] Vinoo Ganesh commented on SPARK-27337: -- Hi [~savulchik] - are you able to reproduce this on a Spark

[jira] [Comment Edited] (SPARK-37857) Any depth search not working in get_json_object ($..foo)

2022-01-17 Thread Franck Thang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477433#comment-17477433 ] Franck Thang edited comment on SPARK-37857 at 1/17/22, 10:47 PM: - As

[jira] [Commented] (SPARK-37290) Exponential planning time in case of non-deterministic function

2022-01-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477453#comment-17477453 ] Apache Spark commented on SPARK-37290: -- User 'Stelyus' has created a pull request for this issue:

[jira] [Commented] (SPARK-37857) Any depth search not working in get_json_object ($..foo)

2022-01-17 Thread Franck Thang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477433#comment-17477433 ] Franck Thang commented on SPARK-37857: -- As [~venk] is not answering, I can start taking a look if

[jira] [Assigned] (SPARK-37947) Cannot use _outer generators in a lateral view

2022-01-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37947: Assignee: (was: Apache Spark) > Cannot use _outer generators in a lateral view >

[jira] [Commented] (SPARK-37947) Cannot use _outer generators in a lateral view

2022-01-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477424#comment-17477424 ] Apache Spark commented on SPARK-37947: -- User 'bersprockets' has created a pull request for this

[jira] [Assigned] (SPARK-37947) Cannot use _outer generators in a lateral view

2022-01-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37947: Assignee: Apache Spark > Cannot use _outer generators in a lateral view >

[jira] [Commented] (SPARK-37947) Cannot use _outer generators in a lateral view

2022-01-17 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477412#comment-17477412 ] Bruce Robbins commented on SPARK-37947: --- While a minor issue (you could always use "{{outer }}"

[jira] [Created] (SPARK-37947) Cannot use _outer generators in a lateral view

2022-01-17 Thread Bruce Robbins (Jira)
Bruce Robbins created SPARK-37947: - Summary: Cannot use _outer generators in a lateral view Key: SPARK-37947 URL: https://issues.apache.org/jira/browse/SPARK-37947 Project: Spark Issue Type:

[jira] [Commented] (SPARK-37937) Use error classes in the parsing errors of lateral join

2022-01-17 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477384#comment-17477384 ] Terry Kim commented on SPARK-37937: --- I will work on this. Thanks [~maxgekk] for organizing this! >

[jira] [Assigned] (SPARK-37290) Exponential planning time in case of non-deterministic function

2022-01-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37290: Assignee: Apache Spark > Exponential planning time in case of non-deterministic function

[jira] [Assigned] (SPARK-37290) Exponential planning time in case of non-deterministic function

2022-01-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37290: Assignee: (was: Apache Spark) > Exponential planning time in case of

[jira] [Commented] (SPARK-37290) Exponential planning time in case of non-deterministic function

2022-01-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477375#comment-17477375 ] Apache Spark commented on SPARK-37290: -- User 'Stelyus' has created a pull request for this issue:

[jira] [Created] (SPARK-37946) Use error classes in the execution errors related to partitions

2022-01-17 Thread Max Gekk (Jira)
Max Gekk created SPARK-37946: Summary: Use error classes in the execution errors related to partitions Key: SPARK-37946 URL: https://issues.apache.org/jira/browse/SPARK-37946 Project: Spark

[jira] [Created] (SPARK-37945) Use error classes in the execution errors of arithmetic ops

2022-01-17 Thread Max Gekk (Jira)
Max Gekk created SPARK-37945: Summary: Use error classes in the execution errors of arithmetic ops Key: SPARK-37945 URL: https://issues.apache.org/jira/browse/SPARK-37945 Project: Spark Issue

[jira] [Created] (SPARK-37944) Use error classes in the execution errors of casting

2022-01-17 Thread Max Gekk (Jira)
Max Gekk created SPARK-37944: Summary: Use error classes in the execution errors of casting Key: SPARK-37944 URL: https://issues.apache.org/jira/browse/SPARK-37944 Project: Spark Issue Type:

[jira] [Created] (SPARK-37943) Use error classes in the compilation errors of grouping

2022-01-17 Thread Max Gekk (Jira)
Max Gekk created SPARK-37943: Summary: Use error classes in the compilation errors of grouping Key: SPARK-37943 URL: https://issues.apache.org/jira/browse/SPARK-37943 Project: Spark Issue Type:

[jira] [Created] (SPARK-37942) Use error classes in the compilation errors of properties

2022-01-17 Thread Max Gekk (Jira)
Max Gekk created SPARK-37942: Summary: Use error classes in the compilation errors of properties Key: SPARK-37942 URL: https://issues.apache.org/jira/browse/SPARK-37942 Project: Spark Issue

[jira] [Created] (SPARK-37941) Use error classes in the compilation errors of casting

2022-01-17 Thread Max Gekk (Jira)
Max Gekk created SPARK-37941: Summary: Use error classes in the compilation errors of casting Key: SPARK-37941 URL: https://issues.apache.org/jira/browse/SPARK-37941 Project: Spark Issue Type:

[jira] [Created] (SPARK-37940) Use error classes in the compilation errors of partitions

2022-01-17 Thread Max Gekk (Jira)
Max Gekk created SPARK-37940: Summary: Use error classes in the compilation errors of partitions Key: SPARK-37940 URL: https://issues.apache.org/jira/browse/SPARK-37940 Project: Spark Issue

[jira] [Updated] (SPARK-37939) Use error classes in the parsing errors of properties

2022-01-17 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-37939: - Description: Migrate the following errors in QueryParsingErrors: *

[jira] [Created] (SPARK-37939) Use error classes in the parsing errors of properties

2022-01-17 Thread Max Gekk (Jira)
Max Gekk created SPARK-37939: Summary: Use error classes in the parsing errors of properties Key: SPARK-37939 URL: https://issues.apache.org/jira/browse/SPARK-37939 Project: Spark Issue Type:

[jira] [Created] (SPARK-37938) Use error classes in the parsing errors of partitions

2022-01-17 Thread Max Gekk (Jira)
Max Gekk created SPARK-37938: Summary: Use error classes in the parsing errors of partitions Key: SPARK-37938 URL: https://issues.apache.org/jira/browse/SPARK-37938 Project: Spark Issue Type:

[jira] [Created] (SPARK-37937) Use error classes in the parsing errors of lateral join

2022-01-17 Thread Max Gekk (Jira)
Max Gekk created SPARK-37937: Summary: Use error classes in the parsing errors of lateral join Key: SPARK-37937 URL: https://issues.apache.org/jira/browse/SPARK-37937 Project: Spark Issue Type:

[jira] [Updated] (SPARK-37936) Use error classes in the parsing errors of intervals

2022-01-17 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-37936: - Description: Modify the following methods in QueryParsingErrors: *

[jira] [Updated] (SPARK-37936) Use error classes in the parsing errors of intervals

2022-01-17 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-37936: - Description: Modify the following methods in QueryParsingErrors: *

[jira] [Updated] (SPARK-37936) Use error classes in the parsing errors of intervals

2022-01-17 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-37936: - Description: Modify the following methods in QueryParsingErrors: *

[jira] [Created] (SPARK-37936) Use error classes in the parsing errors of intervals

2022-01-17 Thread Max Gekk (Jira)
Max Gekk created SPARK-37936: Summary: Use error classes in the parsing errors of intervals Key: SPARK-37936 URL: https://issues.apache.org/jira/browse/SPARK-37936 Project: Spark Issue Type:

[jira] [Updated] (SPARK-37858) Throw Spark exceptions from AES functions

2022-01-17 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-37858: - Parent: SPARK-37935 Issue Type: Sub-task (was: Improvement) > Throw Spark exceptions from AES

[jira] [Created] (SPARK-37935) Migrate onto error classes

2022-01-17 Thread Max Gekk (Jira)
Max Gekk created SPARK-37935: Summary: Migrate onto error classes Key: SPARK-37935 URL: https://issues.apache.org/jira/browse/SPARK-37935 Project: Spark Issue Type: Umbrella

[jira] [Assigned] (SPARK-37934) Upgrade Jetty version to 9.4.44

2022-01-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37934: Assignee: (was: Apache Spark) > Upgrade Jetty version to 9.4.44 >

[jira] [Assigned] (SPARK-37934) Upgrade Jetty version to 9.4.44

2022-01-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37934: Assignee: Apache Spark > Upgrade Jetty version to 9.4.44 >

[jira] [Commented] (SPARK-37934) Upgrade Jetty version to 9.4.44

2022-01-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477260#comment-17477260 ] Apache Spark commented on SPARK-37934: -- User 'this' has created a pull request for this issue:

[jira] [Created] (SPARK-37934) Upgrade Jetty version to 9.4.44

2022-01-17 Thread Sajith A (Jira)
Sajith A created SPARK-37934: Summary: Upgrade Jetty version to 9.4.44 Key: SPARK-37934 URL: https://issues.apache.org/jira/browse/SPARK-37934 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-31726) Make spark.files available in driver with cluster deploy mode on kubernetes

2022-01-17 Thread Martin Andersson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477197#comment-17477197 ] Martin Andersson commented on SPARK-31726: -- [~koert] this works well when you need to read a

[jira] [Updated] (SPARK-37932) Analyzer can fail when join left side and right side are the same view

2022-01-17 Thread Feng Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Zhu updated SPARK-37932: - Description: See the attachment for details, including SQL and the exception information. * sql1,

[jira] [Updated] (SPARK-37932) Analyzer can fail when join left side and right side are the same view

2022-01-17 Thread Feng Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Zhu updated SPARK-37932: - Description: See the attachment for details, including SQL and the exception information. * sql1,

[jira] [Updated] (SPARK-37932) Analyzer can fail when join left side and right side are the same view

2022-01-17 Thread Feng Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Zhu updated SPARK-37932: - Description: See the attachment for details, including SQL and the exception information. * sql1,

[jira] [Updated] (SPARK-37932) Analyzer can fail when join left side and right side are the same view

2022-01-17 Thread Feng Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Zhu updated SPARK-37932: - Description: See the attachment for details, including SQL and the exception information. sql1 

[jira] [Updated] (SPARK-37932) Analyzer can fail when join left side and right side are the same view

2022-01-17 Thread Feng Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Zhu updated SPARK-37932: - Attachment: (was: sql_and_exception) > Analyzer can fail when join left side and right side are the

[jira] [Updated] (SPARK-37932) Analyzer can fail when join left side and right side are the same view

2022-01-17 Thread Feng Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Zhu updated SPARK-37932: - Attachment: sql_and_exception > Analyzer can fail when join left side and right side are the same view

[jira] [Created] (SPARK-37933) Limit push down for parquet datasource v2

2022-01-17 Thread Jackey Lee (Jira)
Jackey Lee created SPARK-37933: -- Summary: Limit push down for parquet datasource v2 Key: SPARK-37933 URL: https://issues.apache.org/jira/browse/SPARK-37933 Project: Spark Issue Type:

[jira] [Commented] (SPARK-37690) Recursive view `df` detected (cycle: `df` -> `df`)

2022-01-17 Thread Daniel Darabos (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477141#comment-17477141 ] Daniel Darabos commented on SPARK-37690: We've hit this too with Spark 3.2.0. Could this be

[jira] [Updated] (SPARK-37932) Analyzer can fail when join left side and right side are the same view

2022-01-17 Thread Feng Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Zhu updated SPARK-37932: - Description: See the attachment for details, including SQL and the exception information (was: with

[jira] [Updated] (SPARK-37932) Analyzer can fail when join left side and right side are the same view

2022-01-17 Thread Feng Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Zhu updated SPARK-37932: - Attachment: sql_and_exception > Analyzer can fail when join left side and right side are the same view

[jira] [Assigned] (SPARK-27442) ParquetFileFormat fails to read column named with invalid characters

2022-01-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27442: Assignee: (was: Apache Spark) > ParquetFileFormat fails to read column named with

[jira] [Commented] (SPARK-27442) ParquetFileFormat fails to read column named with invalid characters

2022-01-17 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477122#comment-17477122 ] Apache Spark commented on SPARK-27442: -- User 'AngersZh' has created a pull request for this

  1   2   >