[jira] [Assigned] (SPARK-37904) Improve RebalancePartitions in rules of Optimizer

[ https://issues.apache.org/jira/browse/SPARK-37904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37904: Assignee: Apache Spark > Improve RebalancePartitions in rules of Optimizer >

[jira] [Commented] (SPARK-37904) Improve RebalancePartitions in rules of Optimizer

[ https://issues.apache.org/jira/browse/SPARK-37904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17476006#comment-17476006 ] Apache Spark commented on SPARK-37904: -- User 'ulysses-you' has created a pull request for this

[jira] [Assigned] (SPARK-37904) Improve RebalancePartitions in rules of Optimizer

[ https://issues.apache.org/jira/browse/SPARK-37904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37904: Assignee: (was: Apache Spark) > Improve RebalancePartitions in rules of Optimizer >

[jira] [Assigned] (SPARK-36967) Report accurate shuffle block size if its skewed

[ https://issues.apache.org/jira/browse/SPARK-36967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros reassigned SPARK-36967: -- Assignee: Wan Kun (was: Apache Spark) > Report accurate shuffle block size

[jira] [Resolved] (SPARK-36967) Report accurate shuffle block size if its skewed

[ https://issues.apache.org/jira/browse/SPARK-36967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros resolved SPARK-36967. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-37907) StaticInvoke should support ConstantFolding

[ https://issues.apache.org/jira/browse/SPARK-37907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37907: Assignee: (was: Apache Spark) > StaticInvoke should support ConstantFolding >

[jira] [Assigned] (SPARK-37907) StaticInvoke should support ConstantFolding

[ https://issues.apache.org/jira/browse/SPARK-37907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37907: Assignee: Apache Spark > StaticInvoke should support ConstantFolding >

[jira] [Commented] (SPARK-37907) StaticInvoke should support ConstantFolding

[ https://issues.apache.org/jira/browse/SPARK-37907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475994#comment-17475994 ] Apache Spark commented on SPARK-37907: -- User 'AngersZh' has created a pull request for this

[jira] [Updated] (SPARK-37873) SQL Syntax links are broken

[ https://issues.apache.org/jira/browse/SPARK-37873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Ott updated SPARK-37873: - Attachment: Screenshot 2022-01-14 at 08.07.24.png > SQL Syntax links are broken >

[jira] [Commented] (SPARK-37873) SQL Syntax links are broken

[ https://issues.apache.org/jira/browse/SPARK-37873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475993#comment-17475993 ] Alex Ott commented on SPARK-37873: -- If you click on any: * [DDL

[jira] [Commented] (SPARK-37907) StaticInvoke should support ConstantFolding

[ https://issues.apache.org/jira/browse/SPARK-37907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475980#comment-17475980 ] angerszhu commented on SPARK-37907: --- Raise a pr soon > StaticInvoke should support ConstantFolding >

[jira] [Created] (SPARK-37907) StaticInvoke should support ConstantFolding

angerszhu created SPARK-37907: - Summary: StaticInvoke should support ConstantFolding Key: SPARK-37907 URL: https://issues.apache.org/jira/browse/SPARK-37907 Project: Spark Issue Type: Task

[jira] [Commented] (SPARK-37906) spark-sql should not pass last simple comment to backend

[ https://issues.apache.org/jira/browse/SPARK-37906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475974#comment-17475974 ] Apache Spark commented on SPARK-37906: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-37906) spark-sql should not pass last simple comment to backend

[ https://issues.apache.org/jira/browse/SPARK-37906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37906: Assignee: Apache Spark > spark-sql should not pass last simple comment to backend >

[jira] [Assigned] (SPARK-37906) spark-sql should not pass last simple comment to backend

[ https://issues.apache.org/jira/browse/SPARK-37906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37906: Assignee: (was: Apache Spark) > spark-sql should not pass last simple comment to

[jira] [Commented] (SPARK-37873) SQL Syntax links are broken

[ https://issues.apache.org/jira/browse/SPARK-37873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475969#comment-17475969 ] Hyukjin Kwon commented on SPARK-37873: -- [~alexott] which syntax is broken? > SQL Syntax links are

[jira] [Commented] (SPARK-37872) [SQL] Some classes are move from org.codehaus.janino:janino to org.codehaus.janino:common-compiler after version 3.1.x

[ https://issues.apache.org/jira/browse/SPARK-37872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475971#comment-17475971 ] Hyukjin Kwon commented on SPARK-37872: -- Spark 2.4 is EOL. Is it still valid for Spark 3+? > [SQL]

[jira] [Resolved] (SPARK-37874) Link to Pandas UDF documentation is broken

[ https://issues.apache.org/jira/browse/SPARK-37874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37874. -- Resolution: Fixed > Link to Pandas UDF documentation is broken >

[jira] [Commented] (SPARK-37874) Link to Pandas UDF documentation is broken

[ https://issues.apache.org/jira/browse/SPARK-37874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475968#comment-17475968 ] Hyukjin Kwon commented on SPARK-37874: -- Fixed in https://github.com/apache/spark/pull/34475 > Link

[jira] [Updated] (SPARK-37874) Link to Pandas UDF documentation is broken

[ https://issues.apache.org/jira/browse/SPARK-37874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37874: - Fix Version/s: 3.2.1 3.3.0 > Link to Pandas UDF documentation is broken >

[jira] [Commented] (SPARK-37882) pyarrow.lib.ArrowInvalid: Can only convert 1-dimensional array values

[ https://issues.apache.org/jira/browse/SPARK-37882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475967#comment-17475967 ] Hyukjin Kwon commented on SPARK-37882: -- [~mattvan83] mind providing self-contained reproducer? >

[jira] [Resolved] (SPARK-37883) log4j update to 2.17.1 in spark-core 3.2

[ https://issues.apache.org/jira/browse/SPARK-37883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37883. -- Resolution: Won't Fix > log4j update to 2.17.1 in spark-core 3.2 >

[jira] [Commented] (SPARK-37883) log4j update to 2.17.1 in spark-core 3.2

[ https://issues.apache.org/jira/browse/SPARK-37883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475966#comment-17475966 ] Hyukjin Kwon commented on SPARK-37883: -- you should either upgrade to the latest Spark 3.3 when it's

[jira] [Commented] (SPARK-37883) log4j update to 2.17.1 in spark-core 3.2

[ https://issues.apache.org/jira/browse/SPARK-37883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475965#comment-17475965 ] Hyukjin Kwon commented on SPARK-37883: -- We upgraded the log4J in the latest master. Old Spark 3.2

[jira] [Updated] (SPARK-37883) log4j update to 2.17.1 in spark-core 3.2

[ https://issues.apache.org/jira/browse/SPARK-37883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37883: - Priority: Major (was: Blocker) > log4j update to 2.17.1 in spark-core 3.2 >

[jira] [Created] (SPARK-37906) spark-sql should not pass last simple comment to backend

angerszhu created SPARK-37906: - Summary: spark-sql should not pass last simple comment to backend Key: SPARK-37906 URL: https://issues.apache.org/jira/browse/SPARK-37906 Project: Spark Issue

[jira] [Assigned] (SPARK-37905) Make `merge_spark_pr.py` set primary author from the first commit in case of ties

[ https://issues.apache.org/jira/browse/SPARK-37905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-37905: - Assignee: Dongjoon Hyun > Make `merge_spark_pr.py` set primary author from the first

[jira] [Resolved] (SPARK-37905) Make `merge_spark_pr.py` set primary author from the first commit in case of ties

[ https://issues.apache.org/jira/browse/SPARK-37905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37905. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35205

[jira] [Assigned] (SPARK-37905) Make `merge_spark_pr.py` set primary author from the first commit in case of ties

[ https://issues.apache.org/jira/browse/SPARK-37905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37905: Assignee: (was: Apache Spark) > Make `merge_spark_pr.py` set primary author from the

[jira] [Assigned] (SPARK-37905) Make `merge_spark_pr.py` set primary author from the first commit in case of ties

[ https://issues.apache.org/jira/browse/SPARK-37905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37905: Assignee: Apache Spark > Make `merge_spark_pr.py` set primary author from the first

[jira] [Commented] (SPARK-37905) Make `merge_spark_pr.py` set primary author from the first commit in case of ties

[ https://issues.apache.org/jira/browse/SPARK-37905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475923#comment-17475923 ] Apache Spark commented on SPARK-37905: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Updated] (SPARK-37905) Make `merge_spark_pr.py` set primary author from the first commit in case of ties

[ https://issues.apache.org/jira/browse/SPARK-37905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37905: -- Summary: Make `merge_spark_pr.py` set primary author from the first commit in case of ties

[jira] [Created] (SPARK-37905) Fix `merge_spark_pr.py` to consider the first commit author as the primary author in case of ties

Dongjoon Hyun created SPARK-37905: - Summary: Fix `merge_spark_pr.py` to consider the first commit author as the primary author in case of ties Key: SPARK-37905 URL:

[jira] [Assigned] (SPARK-37880) Upgrade Scala to 2.13.8

[ https://issues.apache.org/jira/browse/SPARK-37880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-37880: - Assignee: Yang Jie > Upgrade Scala to 2.13.8 > --- > >

[jira] [Resolved] (SPARK-37880) Upgrade Scala to 2.13.8

[ https://issues.apache.org/jira/browse/SPARK-37880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37880. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35181

[jira] [Assigned] (SPARK-37878) Migrate SHOW CREATE TABLE to use v2 command by default

[ https://issues.apache.org/jira/browse/SPARK-37878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37878: Assignee: (was: Apache Spark) > Migrate SHOW CREATE TABLE to use v2 command by

[jira] [Commented] (SPARK-37878) Migrate SHOW CREATE TABLE to use v2 command by default

[ https://issues.apache.org/jira/browse/SPARK-37878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475911#comment-17475911 ] Apache Spark commented on SPARK-37878: -- User 'Peng-Lei' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37878) Migrate SHOW CREATE TABLE to use v2 command by default

[ https://issues.apache.org/jira/browse/SPARK-37878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37878: Assignee: Apache Spark > Migrate SHOW CREATE TABLE to use v2 command by default >

[jira] [Resolved] (SPARK-37893) Fix flaky test: AdaptiveQueryExecSuite with Scala 2.13

[ https://issues.apache.org/jira/browse/SPARK-37893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37893. --- Fix Version/s: 3.3.0 Assignee: Yang Jie Resolution: Fixed This is resolved

[jira] [Assigned] (SPARK-37859) SQL tables created with JDBC with Spark 3.1 are not readable with 3.2

[ https://issues.apache.org/jira/browse/SPARK-37859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37859: --- Assignee: Karen Feng > SQL tables created with JDBC with Spark 3.1 are not readable with

[jira] [Resolved] (SPARK-37859) SQL tables created with JDBC with Spark 3.1 are not readable with 3.2

[ https://issues.apache.org/jira/browse/SPARK-37859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37859. - Fix Version/s: 3.3.0 3.2.1 Resolution: Fixed Issue resolved by pull

[jira] [Created] (SPARK-37904) Improve RebalancePartitions in rules of Optimizer

XiDuo You created SPARK-37904: - Summary: Improve RebalancePartitions in rules of Optimizer Key: SPARK-37904 URL: https://issues.apache.org/jira/browse/SPARK-37904 Project: Spark Issue Type:

[jira] [Commented] (SPARK-37886) Use ComparisonTestBase to reduce redundant test code

[ https://issues.apache.org/jira/browse/SPARK-37886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475902#comment-17475902 ] Apache Spark commented on SPARK-37886: -- User 'Yikun' has created a pull request for this issue:

[jira] [Commented] (SPARK-27442) ParquetFileFormat fails to read column named with invalid characters

[ https://issues.apache.org/jira/browse/SPARK-27442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475894#comment-17475894 ] Wenchen Fan commented on SPARK-27442: - I think we should fix this. It's OK for Spark to forbid

[jira] [Commented] (SPARK-37479) Migrate DROP NAMESPACE to use V2 command by default

[ https://issues.apache.org/jira/browse/SPARK-37479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475889#comment-17475889 ] Apache Spark commented on SPARK-37479: -- User 'dchvn' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37479) Migrate DROP NAMESPACE to use V2 command by default

[ https://issues.apache.org/jira/browse/SPARK-37479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37479: Assignee: Apache Spark > Migrate DROP NAMESPACE to use V2 command by default >

[jira] [Assigned] (SPARK-37479) Migrate DROP NAMESPACE to use V2 command by default

[ https://issues.apache.org/jira/browse/SPARK-37479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37479: Assignee: (was: Apache Spark) > Migrate DROP NAMESPACE to use V2 command by default

[jira] [Commented] (SPARK-37479) Migrate DROP NAMESPACE to use V2 command by default

[ https://issues.apache.org/jira/browse/SPARK-37479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475888#comment-17475888 ] Apache Spark commented on SPARK-37479: -- User 'dchvn' has created a pull request for this issue:

[jira] [Commented] (SPARK-36885) Inline type hints for python/pyspark/sql/dataframe.py

[ https://issues.apache.org/jira/browse/SPARK-36885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475883#comment-17475883 ] Apache Spark commented on SPARK-36885: -- User 'Yikun' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37903) Replace string_typehints with get_type_hints.

[ https://issues.apache.org/jira/browse/SPARK-37903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-37903: Assignee: Takuya Ueshin > Replace string_typehints with get_type_hints. >

[jira] [Resolved] (SPARK-37903) Replace string_typehints with get_type_hints.

[ https://issues.apache.org/jira/browse/SPARK-37903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37903. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35200

[jira] [Comment Edited] (SPARK-37154) Inline type hints for python/pyspark/rdd.py

[ https://issues.apache.org/jira/browse/SPARK-37154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17460243#comment-17460243 ] Byron Hsu edited comment on SPARK-37154 at 1/14/22, 12:41 AM: -- I am looking

[jira] (SPARK-37154) Inline type hints for python/pyspark/rdd.py

[ https://issues.apache.org/jira/browse/SPARK-37154 ] Byron Hsu deleted comment on SPARK-37154: --- was (Author: byronhsu): I am looking into this one > Inline type hints for python/pyspark/rdd.py > --- > >

[jira] [Commented] (SPARK-37903) Replace string_typehints with get_type_hints.

[ https://issues.apache.org/jira/browse/SPARK-37903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475858#comment-17475858 ] Apache Spark commented on SPARK-37903: -- User 'ueshin' has created a pull request for this issue:

[jira] [Commented] (SPARK-37903) Replace string_typehints with get_type_hints.

[ https://issues.apache.org/jira/browse/SPARK-37903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475857#comment-17475857 ] Apache Spark commented on SPARK-37903: -- User 'ueshin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37903) Replace string_typehints with get_type_hints.

[ https://issues.apache.org/jira/browse/SPARK-37903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37903: Assignee: Apache Spark > Replace string_typehints with get_type_hints. >

[jira] [Assigned] (SPARK-37903) Replace string_typehints with get_type_hints.

[ https://issues.apache.org/jira/browse/SPARK-37903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37903: Assignee: (was: Apache Spark) > Replace string_typehints with get_type_hints. >

[jira] [Resolved] (SPARK-37879) Show test report in GitHub Actions builds from PRs

[ https://issues.apache.org/jira/browse/SPARK-37879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37879. -- Resolution: Fixed Issue resolved by pull request 35193

[jira] [Assigned] (SPARK-37879) Show test report in GitHub Actions builds from PRs

[ https://issues.apache.org/jira/browse/SPARK-37879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-37879: Assignee: Hyukjin Kwon > Show test report in GitHub Actions builds from PRs >

[jira] [Created] (SPARK-37903) Replace string_typehints with get_type_hints.

Takuya Ueshin created SPARK-37903: - Summary: Replace string_typehints with get_type_hints. Key: SPARK-37903 URL: https://issues.apache.org/jira/browse/SPARK-37903 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-37887) PySpark shell sets log level to INFO by default

[ https://issues.apache.org/jira/browse/SPARK-37887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37887. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35198

[jira] [Resolved] (SPARK-37900) Use SparkMasterRegex.KUBERNETES_REGEX in SecurityManager

[ https://issues.apache.org/jira/browse/SPARK-37900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37900. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35195

[jira] [Assigned] (SPARK-37900) Use SparkMasterRegex.KUBERNETES_REGEX in SecurityManager

[ https://issues.apache.org/jira/browse/SPARK-37900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-37900: - Assignee: Dongjoon Hyun > Use SparkMasterRegex.KUBERNETES_REGEX in SecurityManager >

[jira] [Assigned] (SPARK-37902) Update annotations to resolve issues detected with mypy==0.931

[ https://issues.apache.org/jira/browse/SPARK-37902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37902: Assignee: (was: Apache Spark) > Update annotations to resolve issues detected with

[jira] [Assigned] (SPARK-37902) Update annotations to resolve issues detected with mypy==0.931

[ https://issues.apache.org/jira/browse/SPARK-37902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37902: Assignee: Apache Spark > Update annotations to resolve issues detected with mypy==0.931

[jira] [Commented] (SPARK-37902) Update annotations to resolve issues detected with mypy==0.931

[ https://issues.apache.org/jira/browse/SPARK-37902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475767#comment-17475767 ] Apache Spark commented on SPARK-37902: -- User 'zero323' has created a pull request for this issue:

[jira] [Updated] (SPARK-36404) Support nested columns in ORC vectorized reader for data source v2

[ https://issues.apache.org/jira/browse/SPARK-36404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36404: -- Labels: releasenotes (was: ) > Support nested columns in ORC vectorized reader for data

[jira] [Commented] (SPARK-36649) Support Trigger.AvailableNow on Kafka data source

[ https://issues.apache.org/jira/browse/SPARK-36649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475761#comment-17475761 ] Boyang Jerry Peng commented on SPARK-36649: --- i'm working on it > Support Trigger.AvailableNow

[jira] [Created] (SPARK-37902) Update annotations to resolve issues detected with mypy==0.931

Maciej Szymkiewicz created SPARK-37902: -- Summary: Update annotations to resolve issues detected with mypy==0.931 Key: SPARK-37902 URL: https://issues.apache.org/jira/browse/SPARK-37902 Project:

[jira] [Assigned] (SPARK-37887) PySpark shell sets log level to INFO by default

[ https://issues.apache.org/jira/browse/SPARK-37887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37887: Assignee: Apache Spark (was: L. C. Hsieh) > PySpark shell sets log level to INFO by

[jira] [Assigned] (SPARK-37887) PySpark shell sets log level to INFO by default

[ https://issues.apache.org/jira/browse/SPARK-37887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37887: Assignee: L. C. Hsieh (was: Apache Spark) > PySpark shell sets log level to INFO by

[jira] [Commented] (SPARK-37887) PySpark shell sets log level to INFO by default

[ https://issues.apache.org/jira/browse/SPARK-37887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475740#comment-17475740 ] Apache Spark commented on SPARK-37887: -- User 'viirya' has created a pull request for this issue:

[jira] [Updated] (SPARK-37900) Use SparkMasterRegex.KUBERNETES_REGEX in SecurityManager

[ https://issues.apache.org/jira/browse/SPARK-37900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37900: -- Component/s: Spark Core (was: Kubernetes) > Use

[jira] [Commented] (SPARK-37887) PySpark shell sets log level to INFO by default

[ https://issues.apache.org/jira/browse/SPARK-37887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475713#comment-17475713 ] L. C. Hsieh commented on SPARK-37887: - I know the root cause. I will submit a PR later. > PySpark

[jira] [Assigned] (SPARK-37887) PySpark shell sets log level to INFO by default

[ https://issues.apache.org/jira/browse/SPARK-37887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-37887: --- Assignee: L. C. Hsieh > PySpark shell sets log level to INFO by default >

[jira] [Assigned] (SPARK-37901) Upgrade Netty from 4.1.72 to 4.1.73

[ https://issues.apache.org/jira/browse/SPARK-37901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37901: Assignee: (was: Apache Spark) > Upgrade Netty from 4.1.72 to 4.1.73 >

[jira] [Commented] (SPARK-37901) Upgrade Netty from 4.1.72 to 4.1.73

[ https://issues.apache.org/jira/browse/SPARK-37901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475672#comment-17475672 ] Apache Spark commented on SPARK-37901: -- User 'dchristle' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37901) Upgrade Netty from 4.1.72 to 4.1.73

[ https://issues.apache.org/jira/browse/SPARK-37901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37901: Assignee: Apache Spark > Upgrade Netty from 4.1.72 to 4.1.73 >

[jira] [Created] (SPARK-37901) Upgrade Netty from 4.1.72 to 4.1.73

David Christle created SPARK-37901: -- Summary: Upgrade Netty from 4.1.72 to 4.1.73 Key: SPARK-37901 URL: https://issues.apache.org/jira/browse/SPARK-37901 Project: Spark Issue Type:

[jira] [Commented] (SPARK-37900) Use SparkMasterRegex.KUBERNETES_REGEX in SecurityManager

[ https://issues.apache.org/jira/browse/SPARK-37900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475650#comment-17475650 ] Apache Spark commented on SPARK-37900: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-37900) Use SparkMasterRegex.KUBERNETES_REGEX in SecurityManager

[ https://issues.apache.org/jira/browse/SPARK-37900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475643#comment-17475643 ] Apache Spark commented on SPARK-37900: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-37900) Use SparkMasterRegex.KUBERNETES_REGEX in SecurityManager

[ https://issues.apache.org/jira/browse/SPARK-37900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37900: Assignee: (was: Apache Spark) > Use SparkMasterRegex.KUBERNETES_REGEX in

[jira] [Assigned] (SPARK-37900) Use SparkMasterRegex.KUBERNETES_REGEX in SecurityManager

[ https://issues.apache.org/jira/browse/SPARK-37900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37900: Assignee: Apache Spark > Use SparkMasterRegex.KUBERNETES_REGEX in SecurityManager >

[jira] [Created] (SPARK-37900) Use SparkMasterRegex.KUBERNETES_REGEX in SecurityManager

Dongjoon Hyun created SPARK-37900: - Summary: Use SparkMasterRegex.KUBERNETES_REGEX in SecurityManager Key: SPARK-37900 URL: https://issues.apache.org/jira/browse/SPARK-37900 Project: Spark

[jira] [Resolved] (SPARK-37864) Support Parquet v2 data page RLE encoding (for Boolean Values) for the vectorized path

[ https://issues.apache.org/jira/browse/SPARK-37864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-37864. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35163

[jira] [Assigned] (SPARK-37864) Support Parquet v2 data page RLE encoding (for Boolean Values) for the vectorized path

[ https://issues.apache.org/jira/browse/SPARK-37864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-37864: Assignee: Yang Jie > Support Parquet v2 data page RLE encoding (for Boolean Values) for the >

[jira] [Commented] (SPARK-37479) Migrate DROP NAMESPACE to use V2 command by default

[ https://issues.apache.org/jira/browse/SPARK-37479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475542#comment-17475542 ] Terry Kim commented on SPARK-37479: --- OK, thanks! > Migrate DROP NAMESPACE to use V2 command by

[jira] [Commented] (SPARK-37899) EliminateInnerJoin to support convert inner join to left semi join

[ https://issues.apache.org/jira/browse/SPARK-37899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475525#comment-17475525 ] Apache Spark commented on SPARK-37899: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37899) EliminateInnerJoin to support convert inner join to left semi join

[ https://issues.apache.org/jira/browse/SPARK-37899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37899: Assignee: (was: Apache Spark) > EliminateInnerJoin to support convert inner join to

[jira] [Assigned] (SPARK-37899) EliminateInnerJoin to support convert inner join to left semi join

[ https://issues.apache.org/jira/browse/SPARK-37899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37899: Assignee: Apache Spark > EliminateInnerJoin to support convert inner join to left semi

[jira] [Updated] (SPARK-37899) EliminateInnerJoin to support convert inner join to left semi join

[ https://issues.apache.org/jira/browse/SPARK-37899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-37899: Summary: EliminateInnerJoin to support convert inner join to left semi join (was:

[jira] [Created] (SPARK-37899) EliminateInnerJoin

Yuming Wang created SPARK-37899: --- Summary: EliminateInnerJoin Key: SPARK-37899 URL: https://issues.apache.org/jira/browse/SPARK-37899 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-37895) Error while joining two tables with non-english field names

[ https://issues.apache.org/jira/browse/SPARK-37895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475361#comment-17475361 ] Wenchen Fan commented on SPARK-37895: - [~beliefer]  can you help to fix it? also cc [~huaxingao]  >

[jira] [Commented] (SPARK-37895) Error while joining two tables with non-english field names

[ https://issues.apache.org/jira/browse/SPARK-37895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475360#comment-17475360 ] Wenchen Fan commented on SPARK-37895: - This bug is only in JDBC v2. In the v2 code path, we always

[jira] [Updated] (SPARK-37898) Error reading old dates when AQE is enabled in Spark 3.1. Works when AQE is disabled

[ https://issues.apache.org/jira/browse/SPARK-37898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gaspar Muñoz updated SPARK-37898: -- Description: Hi guys,    I was testing an spark job that fail when I encountered something

[jira] [Updated] (SPARK-37898) Error reading old dates when AQE is enabled in Spark 3.1. Works when AQE is disabled

[ https://issues.apache.org/jira/browse/SPARK-37898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gaspar Muñoz updated SPARK-37898: -- Description: Hi guys,    I was testing an spark job that fail when I encountered something

[jira] [Created] (SPARK-37898) Error reading old dates when AQE is enabled in Spark 3.1. Works when AQE is disabled

Gaspar Muñoz created SPARK-37898: - Summary: Error reading old dates when AQE is enabled in Spark 3.1. Works when AQE is disabled Key: SPARK-37898 URL: https://issues.apache.org/jira/browse/SPARK-37898

[jira] [Updated] (SPARK-37897) Filter with subexpression elimination may cause query failed

[ https://issues.apache.org/jira/browse/SPARK-37897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hujiahua updated SPARK-37897: - Description:   The following test results will fail, the root cause was that the execution order of

[jira] [Updated] (SPARK-37897) Filter with subexpression elimination may cause query failed

[ https://issues.apache.org/jira/browse/SPARK-37897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hujiahua updated SPARK-37897: - Attachment: image-2022-01-13-20-22-09-055.png > Filter with subexpression elimination may cause query

[jira] [Updated] (SPARK-37897) Filter with subexpression elimination may cause query failed

[ https://issues.apache.org/jira/browse/SPARK-37897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hujiahua updated SPARK-37897: - Description:   The following test results will fail, the root cause was that the execution order of

  1   2   >