[jira] [Commented] (SPARK-37265) Support Java 17 in `dev/test-dependencies.sh`

2021-11-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441523#comment-17441523 ] Dongjoon Hyun commented on SPARK-37265: --- Shall we close this, [~sarutak]? > Support Java 17 in

[jira] [Resolved] (SPARK-37264) [SPARK-37264][BUILD] Exclude hadoop-client-api transitive dependency from orc-core

2021-11-09 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-37264. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved in

[jira] [Updated] (SPARK-37264) [SPARK-37264][BUILD] Exclude hadoop-client-api transitive dependency from orc-core

2021-11-09 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-37264: --- Description: Like hadoop-common and hadoop-hdfs, this PR proposes to exclude

[jira] [Updated] (SPARK-37264) Cut the transitive dependency on hadoop-client-api which orc-shims depends on only for Java 17 with hadoop-2.7

2021-11-09 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-37264: --- Description: In the current master, `run-tests.py` fails on Java 17 due to

[jira] [Updated] (SPARK-37264) [SPARK-37264][BUILD] Exclude hadoop-client-api transitive dependency from orc-core

2021-11-09 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-37264: --- Summary: [SPARK-37264][BUILD] Exclude hadoop-client-api transitive dependency from orc-core

[jira] [Updated] (SPARK-37264) Cut the transitive dependency on hadoop-client-api which orc-shims depends on only for Java 17 with hadoop-2.7

2021-11-09 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-37264: --- Summary: Cut the transitive dependency on hadoop-client-api which orc-shims depends on only

[jira] [Assigned] (SPARK-36575) Executor lost may cause spark stage to hang

2021-11-09 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi reassigned SPARK-36575: Assignee: hujiahua > Executor lost may cause spark stage to hang >

[jira] [Commented] (SPARK-34230) Let AQE determine the right parallelism in DistributionAndOrderingUtils

2021-11-09 Thread Anton Okolnychyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441509#comment-17441509 ] Anton Okolnychyi commented on SPARK-34230: -- We should also double check what impact SPARK-36315

[jira] [Commented] (SPARK-37201) Spark SQL reads unnecessary nested fields (filter after explode)

2021-11-09 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441507#comment-17441507 ] angerszhu commented on SPARK-37201: --- [~Kotlov] For case two, caused by SchemaPruning executed before

[jira] [Assigned] (SPARK-37264) Skip dependency testing on Java 17 temporarily

2021-11-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37264: Assignee: Kousuke Saruta (was: Apache Spark) > Skip dependency testing on Java 17

[jira] [Commented] (SPARK-37264) Skip dependency testing on Java 17 temporarily

2021-11-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441505#comment-17441505 ] Apache Spark commented on SPARK-37264: -- User 'sarutak' has created a pull request for this issue:

[jira] [Commented] (SPARK-37264) Skip dependency testing on Java 17 temporarily

2021-11-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441504#comment-17441504 ] Apache Spark commented on SPARK-37264: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37264) Skip dependency testing on Java 17 temporarily

2021-11-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37264: Assignee: Apache Spark (was: Kousuke Saruta) > Skip dependency testing on Java 17

[jira] [Updated] (SPARK-37264) Skip dependency testing on Java 17 temporarily

2021-11-09 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-37264: --- Description: In the current master, `run-tests.py` fails on Java 17 due to

[jira] [Created] (SPARK-37265) Support Java 17 in `dev/test-dependencies.sh`

2021-11-09 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-37265: -- Summary: Support Java 17 in `dev/test-dependencies.sh` Key: SPARK-37265 URL: https://issues.apache.org/jira/browse/SPARK-37265 Project: Spark Issue

[jira] [Created] (SPARK-37264) Skip dependency testing on Java 17 temporarily

2021-11-09 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-37264: -- Summary: Skip dependency testing on Java 17 temporarily Key: SPARK-37264 URL: https://issues.apache.org/jira/browse/SPARK-37264 Project: Spark Issue

[jira] [Comment Edited] (SPARK-37260) PYSPARK Arrow 3.2.0 docs link invalid

2021-11-09 Thread dch nguyen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441494#comment-17441494 ] dch nguyen edited comment on SPARK-37260 at 11/10/21, 4:03 AM: --- ping

[jira] [Commented] (SPARK-37260) PYSPARK Arrow 3.2.0 docs link invalid

2021-11-09 Thread dch nguyen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441494#comment-17441494 ] dch nguyen commented on SPARK-37260: [~hyukjin.kwon] , is this issue resolved by

[jira] [Assigned] (SPARK-36575) Executor lost may cause spark stage to hang

2021-11-09 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi reassigned SPARK-36575: Assignee: (was: wuyi) > Executor lost may cause spark stage to hang >

[jira] [Commented] (SPARK-36575) Executor lost may cause spark stage to hang

2021-11-09 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441486#comment-17441486 ] wuyi commented on SPARK-36575: -- Issue resolved by [https://github.com/apache/spark/pull/33872.]   To

[jira] [Updated] (SPARK-36575) Executor lost may cause spark stage to hang

2021-11-09 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-36575: - Issue Type: Improvement (was: Bug) > Executor lost may cause spark stage to hang >

[jira] [Updated] (SPARK-36575) Executor lost may cause spark stage to hang

2021-11-09 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-36575: - Fix Version/s: 3.3.0 > Executor lost may cause spark stage to hang >

[jira] [Assigned] (SPARK-36575) Executor lost may cause spark stage to hang

2021-11-09 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi reassigned SPARK-36575: Assignee: wuyi > Executor lost may cause spark stage to hang >

[jira] [Resolved] (SPARK-36575) Executor lost may cause spark stage to hang

2021-11-09 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi resolved SPARK-36575. -- Resolution: Fixed > Executor lost may cause spark stage to hang > ---

[jira] [Created] (SPARK-37263) Reduce pandas-on-Spark warning for internal usage.

2021-11-09 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-37263: --- Summary: Reduce pandas-on-Spark warning for internal usage. Key: SPARK-37263 URL: https://issues.apache.org/jira/browse/SPARK-37263 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-37262) Not log empty aggregate and group by in JDBCScan

2021-11-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37262: Assignee: (was: Apache Spark) > Not log empty aggregate and group by in JDBCScan >

[jira] [Assigned] (SPARK-37262) Not log empty aggregate and group by in JDBCScan

2021-11-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37262: Assignee: Apache Spark > Not log empty aggregate and group by in JDBCScan >

[jira] [Commented] (SPARK-37262) Not log empty aggregate and group by in JDBCScan

2021-11-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441429#comment-17441429 ] Apache Spark commented on SPARK-37262: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Commented] (SPARK-37120) Add Java17 GitHub Action build and test job

2021-11-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441425#comment-17441425 ] Apache Spark commented on SPARK-37120: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Updated] (SPARK-37262) Not log empty aggregate and group by in JDBCScan

2021-11-09 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-37262: --- Description: Address the comment https://github.com/apache/spark/pull/34451#discussion_r740220800

[jira] [Created] (SPARK-37262) Not log empty aggregate and group by in JDBCScan

2021-11-09 Thread Huaxin Gao (Jira)
Huaxin Gao created SPARK-37262: -- Summary: Not log empty aggregate and group by in JDBCScan Key: SPARK-37262 URL: https://issues.apache.org/jira/browse/SPARK-37262 Project: Spark Issue Type:

[jira] [Commented] (SPARK-37221) The collect-like API in SparkPlan should support columnar output

2021-11-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441339#comment-17441339 ] Apache Spark commented on SPARK-37221: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37261) Check adding partitions with ANSI intervals

2021-11-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37261: Assignee: Apache Spark (was: Max Gekk) > Check adding partitions with ANSI intervals >

[jira] [Commented] (SPARK-37261) Check adding partitions with ANSI intervals

2021-11-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441313#comment-17441313 ] Apache Spark commented on SPARK-37261: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37261) Check adding partitions with ANSI intervals

2021-11-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37261: Assignee: Max Gekk (was: Apache Spark) > Check adding partitions with ANSI intervals >

[jira] [Created] (SPARK-37261) Check adding partitions with ANSI intervals

2021-11-09 Thread Max Gekk (Jira)
Max Gekk created SPARK-37261: Summary: Check adding partitions with ANSI intervals Key: SPARK-37261 URL: https://issues.apache.org/jira/browse/SPARK-37261 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-37260) PYSPARK Arrow 3.2.0 docs link invalid

2021-11-09 Thread Thomas Graves (Jira)
Thomas Graves created SPARK-37260: - Summary: PYSPARK Arrow 3.2.0 docs link invalid Key: SPARK-37260 URL: https://issues.apache.org/jira/browse/SPARK-37260 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-37246) Run PyArrow tests on Python 3.10

2021-11-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37246: -- Issue Type: Bug (was: Umbrella) > Run PyArrow tests on Python 3.10 >

[jira] [Commented] (SPARK-37246) Run PyArrow tests on Python 3.10

2021-11-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441257#comment-17441257 ] Dongjoon Hyun commented on SPARK-37246: --- Never mind. This is also valuable. I converted this

[jira] [Reopened] (SPARK-37246) Run PyArrow tests on Python 3.10

2021-11-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reopened SPARK-37246: --- > Run PyArrow tests on Python 3.10 > > > Key:

[jira] [Closed] (SPARK-37247) Failed test_create_nan_decimal_dataframe (pyspark.sql.tests.test_dataframe.DataFrameTests)

2021-11-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-37247. - > Failed test_create_nan_decimal_dataframe > (pyspark.sql.tests.test_dataframe.DataFrameTests) >

[jira] [Updated] (SPARK-37246) Run PyArrow tests on Python 3.10

2021-11-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37246: -- Summary: Run PyArrow tests on Python 3.10 (was: Support Python 3.10 in PySpark) > Run

[jira] [Updated] (SPARK-37247) Failed test_create_nan_decimal_dataframe (pyspark.sql.tests.test_dataframe.DataFrameTests)

2021-11-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37247: -- Parent: (was: SPARK-37246) Issue Type: Bug (was: Sub-task) > Failed

[jira] [Updated] (SPARK-37248) Failed test_make_date (pyspark.sql.tests.test_functions.FunctionsTests)

2021-11-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37248: -- Parent: (was: SPARK-37246) Issue Type: Bug (was: Sub-task) > Failed

[jira] [Closed] (SPARK-37248) Failed test_make_date (pyspark.sql.tests.test_functions.FunctionsTests)

2021-11-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-37248. - > Failed test_make_date (pyspark.sql.tests.test_functions.FunctionsTests) >

[jira] [Updated] (SPARK-37249) ImportError: cannot import name 'Callable' from 'collections'

2021-11-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37249: -- Parent: (was: SPARK-37246) Issue Type: Bug (was: Sub-task) > ImportError: cannot

[jira] [Updated] (SPARK-37250) Failed test_capture_user_friendly_exception (pyspark.sql.tests.test_utils.UtilsTests)

2021-11-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37250: -- Parent: (was: SPARK-37246) Issue Type: Bug (was: Sub-task) > Failed

[jira] [Closed] (SPARK-37251) Failed _joinAsOf doctest

2021-11-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-37251. - > Failed _joinAsOf doctest > > > Key: SPARK-37251 >

[jira] [Updated] (SPARK-37251) Failed _joinAsOf doctest

2021-11-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37251: -- Parent: (was: SPARK-37246) Issue Type: Bug (was: Sub-task) > Failed _joinAsOf

[jira] [Closed] (SPARK-37250) Failed test_capture_user_friendly_exception (pyspark.sql.tests.test_utils.UtilsTests)

2021-11-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-37250. - > Failed test_capture_user_friendly_exception > (pyspark.sql.tests.test_utils.UtilsTests) >

[jira] [Resolved] (SPARK-37257) Update setup.py for Python 3.10

2021-11-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37257. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34533

[jira] [Assigned] (SPARK-37257) Update setup.py for Python 3.10

2021-11-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-37257: - Assignee: Hyukjin Kwon > Update setup.py for Python 3.10 >

[jira] [Commented] (SPARK-35011) Avoid Block Manager registerations when StopExecutor msg is in-flight.

2021-11-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441242#comment-17441242 ] Apache Spark commented on SPARK-35011: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Updated] (SPARK-37259) JDBC read is always going to wrap the query in a select statement

2021-11-09 Thread Kevin Appel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Appel updated SPARK-37259: Description: The read jdbc is wrapping the query it sends to the database server inside a select

[jira] [Commented] (SPARK-35557) Adapt uses of JDK 17 Internal APIs

2021-11-09 Thread Olivier Peyrusse (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441227#comment-17441227 ] Olivier Peyrusse commented on SPARK-35557: -- Hello, everyone, I see a similar issue when

[jira] [Created] (SPARK-37259) JDBC read is always going to wrap the query in a select statement

2021-11-09 Thread Kevin Appel (Jira)
Kevin Appel created SPARK-37259: --- Summary: JDBC read is always going to wrap the query in a select statement Key: SPARK-37259 URL: https://issues.apache.org/jira/browse/SPARK-37259 Project: Spark

[jira] [Commented] (SPARK-37201) Spark SQL reads unnecessary nested fields (filter after explode)

2021-11-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441205#comment-17441205 ] Apache Spark commented on SPARK-37201: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-37201) Spark SQL reads unnecessary nested fields (filter after explode)

2021-11-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37201: Assignee: (was: Apache Spark) > Spark SQL reads unnecessary nested fields (filter

[jira] [Assigned] (SPARK-37201) Spark SQL reads unnecessary nested fields (filter after explode)

2021-11-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37201: Assignee: Apache Spark > Spark SQL reads unnecessary nested fields (filter after

[jira] [Updated] (SPARK-37258) Add Volcano support in kubernetes-client

2021-11-09 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-37258: Description: We need to add a volcano in [1]. There were some comments on k8s-client repo to

[jira] [Created] (SPARK-37258) Add Volcano support in kubernetes-client

2021-11-09 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-37258: --- Summary: Add Volcano support in kubernetes-client Key: SPARK-37258 URL: https://issues.apache.org/jira/browse/SPARK-37258 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-37201) Spark SQL reads unnecessary nested fields (filter after explode)

2021-11-09 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441189#comment-17441189 ] angerszhu commented on SPARK-37201: --- Working on this > Spark SQL reads unnecessary nested fields

[jira] [Commented] (SPARK-37257) Update setup.py for Python 3.10

2021-11-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441180#comment-17441180 ] Apache Spark commented on SPARK-37257: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-37257) Update setup.py for Python 3.10

2021-11-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37257: Assignee: Apache Spark > Update setup.py for Python 3.10 >

[jira] [Commented] (SPARK-37257) Update setup.py for Python 3.10

2021-11-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441178#comment-17441178 ] Apache Spark commented on SPARK-37257: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-37257) Update setup.py for Python 3.10

2021-11-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37257: Assignee: (was: Apache Spark) > Update setup.py for Python 3.10 >

[jira] [Created] (SPARK-37257) Update setup.py for Python 3.10

2021-11-09 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-37257: Summary: Update setup.py for Python 3.10 Key: SPARK-37257 URL: https://issues.apache.org/jira/browse/SPARK-37257 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-37256) Replace `ScalaObjectMapper` with `ClassTagExtensions` to fix compilation warnings

2021-11-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37256: Assignee: Apache Spark > Replace `ScalaObjectMapper` with `ClassTagExtensions` to fix

[jira] [Assigned] (SPARK-37256) Replace `ScalaObjectMapper` with `ClassTagExtensions` to fix compilation warnings

2021-11-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37256: Assignee: (was: Apache Spark) > Replace `ScalaObjectMapper` with

[jira] [Commented] (SPARK-37256) Replace `ScalaObjectMapper` with `ClassTagExtensions` to fix compilation warnings

2021-11-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441142#comment-17441142 ] Apache Spark commented on SPARK-37256: -- User 'LuciferYang' has created a pull request for this

[jira] [Commented] (SPARK-37256) Replace `ScalaObjectMapper` with `ClassTagExtensions` to fix compilation warnings

2021-11-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441141#comment-17441141 ] Apache Spark commented on SPARK-37256: -- User 'LuciferYang' has created a pull request for this

[jira] [Created] (SPARK-37256) Replace `ScalaObjectMapper` with `ClassTagExtensions` to fix compilation warnings

2021-11-09 Thread Yang Jie (Jira)
Yang Jie created SPARK-37256: Summary: Replace `ScalaObjectMapper` with `ClassTagExtensions` to fix compilation warnings Key: SPARK-37256 URL: https://issues.apache.org/jira/browse/SPARK-37256 Project:

[jira] [Commented] (SPARK-25224) Improvement of Spark SQL ThriftServer memory management

2021-11-09 Thread ramakrishna chilaka (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441131#comment-17441131 ] ramakrishna chilaka commented on SPARK-25224: - can anyone please confirm, if there are any

[jira] [Resolved] (SPARK-37253) try_simplify_traceback should not fail when tb_frame.f_lineno is None

2021-11-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37253. -- Fix Version/s: 3.3.0 3.2.1 3.1.3 Resolution:

[jira] [Assigned] (SPARK-37253) try_simplify_traceback should not fail when tb_frame.f_lineno is None

2021-11-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-37253: Assignee: Apache Spark > try_simplify_traceback should not fail when tb_frame.f_lineno

[jira] [Created] (SPARK-37255) When Used with PyHive (by dropbox) query timeout doesn't result in propagation to the UI

2021-11-09 Thread ramakrishna chilaka (Jira)
ramakrishna chilaka created SPARK-37255: --- Summary: When Used with PyHive (by dropbox) query timeout doesn't result in propagation to the UI Key: SPARK-37255 URL:

[jira] [Updated] (SPARK-37254) 100% CPU usage on Spark Thrift Server.

2021-11-09 Thread ramakrishna chilaka (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramakrishna chilaka updated SPARK-37254: Description: We are trying to use Spark thrift server as a distributed sql query

[jira] [Created] (SPARK-37254) 100% CPU usage on Spark Thrift Server.

2021-11-09 Thread ramakrishna chilaka (Jira)
ramakrishna chilaka created SPARK-37254: --- Summary: 100% CPU usage on Spark Thrift Server. Key: SPARK-37254 URL: https://issues.apache.org/jira/browse/SPARK-37254 Project: Spark Issue

[jira] [Commented] (SPARK-37239) Avoid unnecessary `setReplication` in Yarn mode

2021-11-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441038#comment-17441038 ] Apache Spark commented on SPARK-37239: -- User 'LuciferYang' has created a pull request for this

[jira] [Updated] (SPARK-37253) try_simplify_traceback should not fail when tb_frame.f_lineno is None

2021-11-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37253: -- Description: {code} Traceback (most recent call last): File

[jira] [Updated] (SPARK-37253) try_simplify_traceback should not fail when tb_frame.f_lineno is None

2021-11-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37253: -- Description: {code} During handling of the above exception, another exception occurred:

[jira] [Updated] (SPARK-37253) try_simplify_traceback should not fail when tb_frame.f_lineno is None

2021-11-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37253: -- Affects Version/s: 3.2.0 3.1.2 > try_simplify_traceback should not

[jira] [Commented] (SPARK-37253) try_simplify_traceback should not fail when tb_frame.f_lineno is None

2021-11-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17440998#comment-17440998 ] Apache Spark commented on SPARK-37253: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-37253) try_simplify_traceback should not fail when tb_frame.f_lineno is None

2021-11-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37253: Assignee: (was: Apache Spark) > try_simplify_traceback should not fail when

[jira] [Assigned] (SPARK-37253) try_simplify_traceback should not fail when tb_frame.f_lineno is None

2021-11-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37253: Assignee: Apache Spark > try_simplify_traceback should not fail when tb_frame.f_lineno

[jira] [Commented] (SPARK-37253) try_simplify_traceback should not fail when tb_frame.f_lineno is None

2021-11-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17440996#comment-17440996 ] Apache Spark commented on SPARK-37253: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Created] (SPARK-37253) try_simplify_traceback should not fail when tb_frame.f_lineno is None

2021-11-09 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-37253: - Summary: try_simplify_traceback should not fail when tb_frame.f_lineno is None Key: SPARK-37253 URL: https://issues.apache.org/jira/browse/SPARK-37253 Project: