[jira] [Created] (SPARK-37266) Optimize the analysis for view text of persist view and fix security vulnerabilities caused by sql tampering

jiaan.geng created SPARK-37266: -- Summary: Optimize the analysis for view text of persist view and fix security vulnerabilities caused by sql tampering Key: SPARK-37266 URL:

[jira] [Commented] (SPARK-37266) Optimize the analysis for view text of persistent view and fix security vulnerabilities caused by sql tampering

[ https://issues.apache.org/jira/browse/SPARK-37266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441580#comment-17441580 ] Apache Spark commented on SPARK-37266: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37266) Optimize the analysis for view text of persistent view and fix security vulnerabilities caused by sql tampering

[ https://issues.apache.org/jira/browse/SPARK-37266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37266: Assignee: Apache Spark > Optimize the analysis for view text of persistent view and fix

[jira] [Assigned] (SPARK-37266) Optimize the analysis for view text of persistent view and fix security vulnerabilities caused by sql tampering

[ https://issues.apache.org/jira/browse/SPARK-37266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37266: Assignee: (was: Apache Spark) > Optimize the analysis for view text of persistent

[jira] [Commented] (SPARK-37267) OptimizeSkewInRebalancePartitions support optimize non-root node

[ https://issues.apache.org/jira/browse/SPARK-37267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441578#comment-17441578 ] Apache Spark commented on SPARK-37267: -- User 'ulysses-you' has created a pull request for this

[jira] [Assigned] (SPARK-37267) OptimizeSkewInRebalancePartitions support optimize non-root node

[ https://issues.apache.org/jira/browse/SPARK-37267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37267: Assignee: (was: Apache Spark) > OptimizeSkewInRebalancePartitions support optimize

[jira] [Assigned] (SPARK-37267) OptimizeSkewInRebalancePartitions support optimize non-root node

[ https://issues.apache.org/jira/browse/SPARK-37267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37267: Assignee: Apache Spark > OptimizeSkewInRebalancePartitions support optimize non-root

[jira] [Updated] (SPARK-37266) Optimize the analysis for view text of persistent view and fix security vulnerabilities caused by sql tampering

[ https://issues.apache.org/jira/browse/SPARK-37266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-37266: --- Description: The current implementation of persistent view is create hive table with view text.

[jira] [Updated] (SPARK-37266) Optimize the analysis for view text of persistent view and fix security vulnerabilities caused by sql tampering

[ https://issues.apache.org/jira/browse/SPARK-37266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-37266: --- Summary: Optimize the analysis for view text of persistent view and fix security vulnerabilities

[jira] [Created] (SPARK-37267) OptimizeSkewInRebalancePartitions support optimize non-root node

XiDuo You created SPARK-37267: - Summary: OptimizeSkewInRebalancePartitions support optimize non-root node Key: SPARK-37267 URL: https://issues.apache.org/jira/browse/SPARK-37267 Project: Spark

[jira] [Commented] (SPARK-37022) Use black as a formatter for the whole PySpark codebase.

[ https://issues.apache.org/jira/browse/SPARK-37022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441612#comment-17441612 ] Apache Spark commented on SPARK-37022: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-36575) Executor lost may cause spark stage to hang

[ https://issues.apache.org/jira/browse/SPARK-36575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441796#comment-17441796 ] wuyi commented on SPARK-36575: -- FYI: the fix is reverted due to test issues. > Executor lost may cause

[jira] [Commented] (SPARK-37271) Spark OOM issue

[ https://issues.apache.org/jira/browse/SPARK-37271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441805#comment-17441805 ] M Shadab commented on SPARK-37271: -- Memory increased for the container > Spark OOM issue >

[jira] [Resolved] (SPARK-37271) Spark OOM issue

[ https://issues.apache.org/jira/browse/SPARK-37271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] M Shadab resolved SPARK-37271. -- Resolution: Fixed done > Spark OOM issue > --- > > Key: SPARK-37271 >

[jira] [Created] (SPARK-37271) Spark OOM issue

M Shadab created SPARK-37271: Summary: Spark OOM issue Key: SPARK-37271 URL: https://issues.apache.org/jira/browse/SPARK-37271 Project: Spark Issue Type: Bug Components: Spark Submit

[jira] [Updated] (SPARK-37271) Spark OOM issue

[ https://issues.apache.org/jira/browse/SPARK-37271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] M Shadab updated SPARK-37271: - Shepherd: M Shadab > Spark OOM issue > --- > > Key: SPARK-37271 >

[jira] [Created] (SPARK-37268) Remove unused method call in FileScanRDD

Junfan Zhang created SPARK-37268: Summary: Remove unused method call in FileScanRDD Key: SPARK-37268 URL: https://issues.apache.org/jira/browse/SPARK-37268 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-37022) Use black as a formatter for the whole PySpark codebase.

[ https://issues.apache.org/jira/browse/SPARK-37022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-37022: Assignee: Maciej Szymkiewicz > Use black as a formatter for the whole PySpark codebase.

[jira] [Resolved] (SPARK-37022) Use black as a formatter for the whole PySpark codebase.

[ https://issues.apache.org/jira/browse/SPARK-37022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37022. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34297

[jira] [Resolved] (SPARK-37261) Check adding partitions with ANSI intervals

[ https://issues.apache.org/jira/browse/SPARK-37261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-37261. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34537

[jira] [Commented] (SPARK-37045) Unify v1 and v2 ALTER TABLE .. ADD COLUMNS tests

[ https://issues.apache.org/jira/browse/SPARK-37045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441755#comment-17441755 ] Max Gekk commented on SPARK-37045: -- I am working on this. > Unify v1 and v2 ALTER TABLE .. ADD COLUMNS

[jira] [Assigned] (SPARK-37045) Unify v1 and v2 ALTER TABLE .. ADD COLUMNS tests

[ https://issues.apache.org/jira/browse/SPARK-37045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-37045: Assignee: Max Gekk > Unify v1 and v2 ALTER TABLE .. ADD COLUMNS tests >

[jira] [Commented] (SPARK-37022) Use black as a formatter for the whole PySpark codebase.

[ https://issues.apache.org/jira/browse/SPARK-37022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441610#comment-17441610 ] Apache Spark commented on SPARK-37022: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Created] (SPARK-37269) The partitionOverwriteMode option is not respected when using insertInto

David Szakallas created SPARK-37269: --- Summary: The partitionOverwriteMode option is not respected when using insertInto Key: SPARK-37269 URL: https://issues.apache.org/jira/browse/SPARK-37269

[jira] [Created] (SPARK-37270) Incorect result of filter using isNull condition

Tomasz Kus created SPARK-37270: -- Summary: Incorect result of filter using isNull condition Key: SPARK-37270 URL: https://issues.apache.org/jira/browse/SPARK-37270 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-37268) Remove unused method call in FileScanRDD

[ https://issues.apache.org/jira/browse/SPARK-37268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37268: Assignee: (was: Apache Spark) > Remove unused method call in FileScanRDD >

[jira] [Assigned] (SPARK-37268) Remove unused method call in FileScanRDD

[ https://issues.apache.org/jira/browse/SPARK-37268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37268: Assignee: Apache Spark > Remove unused method call in FileScanRDD >

[jira] [Commented] (SPARK-37268) Remove unused method call in FileScanRDD

[ https://issues.apache.org/jira/browse/SPARK-37268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441669#comment-17441669 ] Apache Spark commented on SPARK-37268: -- User 'zuston' has created a pull request for this issue:

[jira] [Updated] (SPARK-37270) Incorect result of filter using isNull condition

[ https://issues.apache.org/jira/browse/SPARK-37270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomasz Kus updated SPARK-37270: --- Component/s: SQL > Incorect result of filter using isNull condition >

[jira] [Resolved] (SPARK-37236) Inline type hints for KernelDensity.pyi, test.py in python/pyspark/mllib/stat/

[ https://issues.apache.org/jira/browse/SPARK-37236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37236. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-37236) Inline type hints for KernelDensity.pyi, test.py in python/pyspark/mllib/stat/

[ https://issues.apache.org/jira/browse/SPARK-37236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37236: -- Assignee: dch nguyen > Inline type hints for KernelDensity.pyi, test.py in

[jira] [Comment Edited] (SPARK-33502) Large number of SELECT columns causes StackOverflowError

[ https://issues.apache.org/jira/browse/SPARK-33502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17236434#comment-17236434 ] Arwin S Tio edited comment on SPARK-33502 at 11/10/21, 7:22 PM: Note,

[jira] [Resolved] (SPARK-35557) Adapt uses of JDK 17 Internal APIs

[ https://issues.apache.org/jira/browse/SPARK-35557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35557. --- Resolution: Duplicate This is superseded by SPARK-36796 via adding `--add-open` options. >

[jira] [Resolved] (SPARK-37265) Support Java 17 in `dev/test-dependencies.sh`

[ https://issues.apache.org/jira/browse/SPARK-37265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37265. --- Resolution: Invalid Let me close this Invalid. > Support Java 17 in

[jira] [Created] (SPARK-37273) Hidden File Metadata Support for Spark SQL

Yaohua Zhao created SPARK-37273: --- Summary: Hidden File Metadata Support for Spark SQL Key: SPARK-37273 URL: https://issues.apache.org/jira/browse/SPARK-37273 Project: Spark Issue Type:

[jira] [Commented] (SPARK-37272) Add ExtendedRocksDBTest

[ https://issues.apache.org/jira/browse/SPARK-37272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17442020#comment-17442020 ] Apache Spark commented on SPARK-37272: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-37272) Add ExtendedRocksDBTest

[ https://issues.apache.org/jira/browse/SPARK-37272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37272: Assignee: (was: Apache Spark) > Add ExtendedRocksDBTest > --- >

[jira] [Assigned] (SPARK-37272) Add ExtendedRocksDBTest

[ https://issues.apache.org/jira/browse/SPARK-37272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37272: Assignee: Apache Spark > Add ExtendedRocksDBTest > --- > >

[jira] [Created] (SPARK-37272) Add ExtendedRocksDBTest

Dongjoon Hyun created SPARK-37272: - Summary: Add ExtendedRocksDBTest Key: SPARK-37272 URL: https://issues.apache.org/jira/browse/SPARK-37272 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-37233) Inline type hints for files in python/pyspark/mllib

[ https://issues.apache.org/jira/browse/SPARK-37233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-37233: Assignee: dch nguyen > Inline type hints for files in python/pyspark/mllib >

[jira] [Commented] (SPARK-37254) 100% CPU usage on Spark Thrift Server.

[ https://issues.apache.org/jira/browse/SPARK-37254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17442046#comment-17442046 ] Hyukjin Kwon commented on SPARK-37254: -- it would be much easier to investigate the issue if

[jira] [Updated] (SPARK-37272) Add `ExtendedRocksDBTest` and disable RocksDB tests on Apple Silicon

[ https://issues.apache.org/jira/browse/SPARK-37272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37272: -- Parent: SPARK-33772 Issue Type: Sub-task (was: Improvement) > Add

[jira] [Updated] (SPARK-37272) Add `ExtendedRocksDBTest` and disable RocksDB tests on Apple Silicon

[ https://issues.apache.org/jira/browse/SPARK-37272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37272: -- Description: Java 17 officially support Apple Silicon - JEP 391: macOS/AArch64 Port -

[jira] [Updated] (SPARK-37272) Add `ExtendedRocksDBTest` and disable RocksDB tests on Apple Silicon

[ https://issues.apache.org/jira/browse/SPARK-37272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37272: -- Description: Javava 17 officially support Apple Silicon - JEP 391: macOS/AArch64 Port -

[jira] [Created] (SPARK-37275) Support ANSI intervals in PySpark

Hyukjin Kwon created SPARK-37275: Summary: Support ANSI intervals in PySpark Key: SPARK-37275 URL: https://issues.apache.org/jira/browse/SPARK-37275 Project: Spark Issue Type: Umbrella

[jira] [Created] (SPARK-37276) Support YearMonthIntervalType in Arrow

Hyukjin Kwon created SPARK-37276: Summary: Support YearMonthIntervalType in Arrow Key: SPARK-37276 URL: https://issues.apache.org/jira/browse/SPARK-37276 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-37277) Support DayTimeIntervalType in Arrow

Hyukjin Kwon created SPARK-37277: Summary: Support DayTimeIntervalType in Arrow Key: SPARK-37277 URL: https://issues.apache.org/jira/browse/SPARK-37277 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-37270) Incorect result of filter using isNull condition

[ https://issues.apache.org/jira/browse/SPARK-37270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17442074#comment-17442074 ] Hyukjin Kwon commented on SPARK-37270: -- Hm, I can't reproduce this locally. Are you able to

[jira] [Resolved] (SPARK-36799) Pass queryExecution name in CLI when only select query

[ https://issues.apache.org/jira/browse/SPARK-36799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-36799. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34041

[jira] [Assigned] (SPARK-36799) Pass queryExecution name in CLI when only select query

[ https://issues.apache.org/jira/browse/SPARK-36799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-36799: --- Assignee: dzcxzl > Pass queryExecution name in CLI when only select query >

[jira] [Updated] (SPARK-36799) Pass queryExecution name in CLI

[ https://issues.apache.org/jira/browse/SPARK-36799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dzcxzl updated SPARK-36799: --- Summary: Pass queryExecution name in CLI (was: Pass queryExecution name in CLI when only select query) >

[jira] [Updated] (SPARK-37276) Support YearMonthIntervalType in Arrow

[ https://issues.apache.org/jira/browse/SPARK-37276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37276: - Description: Implements the support of YearMonthIntervalType in Arrow code path: - pandas UDFs

[jira] [Updated] (SPARK-37278) Support YearMonthIntervalType in createDataFrame/toPandas and Python UDFs

[ https://issues.apache.org/jira/browse/SPARK-37278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37278: - Description: Implements the support of YearMonthIntervalType in: - Python UDFs -

[jira] [Updated] (SPARK-37263) Create an option to silence advice for pandas API on Spark.

[ https://issues.apache.org/jira/browse/SPARK-37263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-37263: Summary: Create an option to silence advice for pandas API on Spark. (was: Reduce

[jira] [Updated] (SPARK-37263) Add an option to silence advice for pandas API on Spark.

[ https://issues.apache.org/jira/browse/SPARK-37263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-37263: Summary: Add an option to silence advice for pandas API on Spark. (was: Create an option to

[jira] [Updated] (SPARK-37263) Add PandasAPIOnSparkAdviceWarning class

[ https://issues.apache.org/jira/browse/SPARK-37263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-37263: Description: Raised from comment

[jira] [Commented] (SPARK-37260) PYSPARK Arrow 3.2.0 docs link invalid

[ https://issues.apache.org/jira/browse/SPARK-37260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17442044#comment-17442044 ] Hyukjin Kwon commented on SPARK-37260: -- oh yeah. that's fixed via #34475. There are some more

[jira] [Resolved] (SPARK-37260) PYSPARK Arrow 3.2.0 docs link invalid

[ https://issues.apache.org/jira/browse/SPARK-37260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37260. -- Resolution: Fixed > PYSPARK Arrow 3.2.0 docs link invalid >

[jira] [Updated] (SPARK-37260) PYSPARK Arrow 3.2.0 docs link invalid

[ https://issues.apache.org/jira/browse/SPARK-37260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37260: - Fix Version/s: 3.2.1 > PYSPARK Arrow 3.2.0 docs link invalid >

[jira] [Updated] (SPARK-37270) Incorect result of filter using isNull condition

[ https://issues.apache.org/jira/browse/SPARK-37270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37270: - Labels: correctness (was: ) > Incorect result of filter using isNull condition >

[jira] [Closed] (SPARK-37109) Install Java 17 on all of the Jenkins workers

[ https://issues.apache.org/jira/browse/SPARK-37109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-37109. - > Install Java 17 on all of the Jenkins workers > - > >

[jira] [Assigned] (SPARK-36900) "SPARK-36464: size returns correct positive number even with over 2GB data" will oom with JDK17

[ https://issues.apache.org/jira/browse/SPARK-36900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-36900: - Assignee: Yang Jie > "SPARK-36464: size returns correct positive number even with over

[jira] [Updated] (SPARK-37109) Install Java 17 on all of the Jenkins workers

[ https://issues.apache.org/jira/browse/SPARK-37109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37109: -- Parent: (was: SPARK-33772) Issue Type: Bug (was: Sub-task) > Install Java 17 on

[jira] [Assigned] (SPARK-36073) EquivalentExpressions fixes and improvements

[ https://issues.apache.org/jira/browse/SPARK-36073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-36073: --- Assignee: Peter Toth > EquivalentExpressions fixes and improvements >

[jira] [Resolved] (SPARK-36073) EquivalentExpressions fixes and improvements

[ https://issues.apache.org/jira/browse/SPARK-36073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-36073. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33281

[jira] [Assigned] (SPARK-37274) These parameters should be of type long, not int

[ https://issues.apache.org/jira/browse/SPARK-37274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37274: Assignee: (was: Apache Spark) > These parameters should be of type long, not int >

[jira] [Assigned] (SPARK-37274) These parameters should be of type long, not int

[ https://issues.apache.org/jira/browse/SPARK-37274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37274: Assignee: Apache Spark > These parameters should be of type long, not int >

[jira] [Commented] (SPARK-37274) These parameters should be of type long, not int

[ https://issues.apache.org/jira/browse/SPARK-37274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17442102#comment-17442102 ] Apache Spark commented on SPARK-37274: -- User 'dh20' has created a pull request for this issue:

[jira] [Updated] (SPARK-37283) Don't try to store a V1 table which contains ANSI intervals in Hive compatible format

[ https://issues.apache.org/jira/browse/SPARK-37283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-37283: --- Description: If, a table being created contains a column of ANSI interval types and the

[jira] [Created] (SPARK-37283) Don't try to store a V1 table which contains ANSI intervals in Hive compatible format

Kousuke Saruta created SPARK-37283: -- Summary: Don't try to store a V1 table which contains ANSI intervals in Hive compatible format Key: SPARK-37283 URL: https://issues.apache.org/jira/browse/SPARK-37283

[jira] [Commented] (SPARK-37284) Upgrade Jekyll to 4.2.1

[ https://issues.apache.org/jira/browse/SPARK-37284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17442114#comment-17442114 ] Apache Spark commented on SPARK-37284: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37284) Upgrade Jekyll to 4.2.1

[ https://issues.apache.org/jira/browse/SPARK-37284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37284: Assignee: Apache Spark (was: Kousuke Saruta) > Upgrade Jekyll to 4.2.1 >

[jira] [Assigned] (SPARK-37284) Upgrade Jekyll to 4.2.1

[ https://issues.apache.org/jira/browse/SPARK-37284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37284: Assignee: Kousuke Saruta (was: Apache Spark) > Upgrade Jekyll to 4.2.1 >

[jira] [Assigned] (SPARK-37272) Add ExtendedRocksDBTest

[ https://issues.apache.org/jira/browse/SPARK-37272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-37272: - Assignee: Dongjoon Hyun > Add ExtendedRocksDBTest > --- > >

[jira] [Updated] (SPARK-37272) Add `ExtendedRocksDBTest` and disable RocksDB tests on Apple Silicon

[ https://issues.apache.org/jira/browse/SPARK-37272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37272: -- Summary: Add `ExtendedRocksDBTest` and disable RocksDB tests on Apple Silicon (was: Add

[jira] [Resolved] (SPARK-37272) Add ExtendedRocksDBTest

[ https://issues.apache.org/jira/browse/SPARK-37272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37272. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34547

[jira] [Resolved] (SPARK-37273) Hidden File Metadata Support for Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-37273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37273. -- Resolution: Duplicate > Hidden File Metadata Support for Spark SQL >

[jira] [Commented] (SPARK-37273) Hidden File Metadata Support for Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-37273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17442057#comment-17442057 ] Hyukjin Kwon commented on SPARK-37273: -- Don't we already have this in DSv2? e.g.) SPARK-31255 >

[jira] [Created] (SPARK-37274) These parameters should be of type long, not int

hao created SPARK-37274: --- Summary: These parameters should be of type long, not int Key: SPARK-37274 URL: https://issues.apache.org/jira/browse/SPARK-37274 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-37278) Support YearMonthIntervalType in createDataFrame/toPandas and Python UDFs

Hyukjin Kwon created SPARK-37278: Summary: Support YearMonthIntervalType in createDataFrame/toPandas and Python UDFs Key: SPARK-37278 URL: https://issues.apache.org/jira/browse/SPARK-37278 Project:

[jira] [Created] (SPARK-37280) Support YearMonthIntervalType in Py4J

Hyukjin Kwon created SPARK-37280: Summary: Support YearMonthIntervalType in Py4J Key: SPARK-37280 URL: https://issues.apache.org/jira/browse/SPARK-37280 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-37281) Support DayTimeIntervalType in Py4J

[ https://issues.apache.org/jira/browse/SPARK-37281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37281: - Description: This PR adds the support of YearMonthIntervalType in Py4J. For example,

[jira] [Commented] (SPARK-37275) Support ANSI intervals in PySpark

[ https://issues.apache.org/jira/browse/SPARK-37275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17442066#comment-17442066 ] Hyukjin Kwon commented on SPARK-37275: -- cc [~maxgekk] FYI > Support ANSI intervals in PySpark >

[jira] [Created] (SPARK-37282) Add ExtendedLevelDBTest and disable LevelDB tests on Apple Silicon

Dongjoon Hyun created SPARK-37282: - Summary: Add ExtendedLevelDBTest and disable LevelDB tests on Apple Silicon Key: SPARK-37282 URL: https://issues.apache.org/jira/browse/SPARK-37282 Project: Spark

[jira] [Updated] (SPARK-37278) Support YearMonthIntervalType in createDataFrame/toPandas and Python UDFs

[ https://issues.apache.org/jira/browse/SPARK-37278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37278: - Description: Implements the support of YearMonthIntervalType in: - Python UDFs -

[jira] [Updated] (SPARK-37276) Support YearMonthIntervalType in Arrow

[ https://issues.apache.org/jira/browse/SPARK-37276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37276: - Description: Implements the support of YearMonthIntervalType in Arrow code path: - pandas UDFs

[jira] [Commented] (SPARK-37283) Don't try to store a V1 table which contains ANSI intervals in Hive compatible format

[ https://issues.apache.org/jira/browse/SPARK-37283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17442111#comment-17442111 ] Apache Spark commented on SPARK-37283: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37283) Don't try to store a V1 table which contains ANSI intervals in Hive compatible format

[ https://issues.apache.org/jira/browse/SPARK-37283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37283: Assignee: Apache Spark (was: Kousuke Saruta) > Don't try to store a V1 table which

[jira] [Assigned] (SPARK-37263) Add PandasAPIOnSparkAdviceWarning class

[ https://issues.apache.org/jira/browse/SPARK-37263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37263: Assignee: (was: Apache Spark) > Add PandasAPIOnSparkAdviceWarning class >

[jira] [Assigned] (SPARK-37283) Don't try to store a V1 table which contains ANSI intervals in Hive compatible format

[ https://issues.apache.org/jira/browse/SPARK-37283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37283: Assignee: Kousuke Saruta (was: Apache Spark) > Don't try to store a V1 table which

[jira] [Created] (SPARK-37284) Upgrade Jekyll to 4.2.1

Kousuke Saruta created SPARK-37284: -- Summary: Upgrade Jekyll to 4.2.1 Key: SPARK-37284 URL: https://issues.apache.org/jira/browse/SPARK-37284 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-37263) Add PandasAPIOnSparkAdviceWarning class

[ https://issues.apache.org/jira/browse/SPARK-37263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17442112#comment-17442112 ] Apache Spark commented on SPARK-37263: -- User 'itholic' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37263) Add PandasAPIOnSparkAdviceWarning class

[ https://issues.apache.org/jira/browse/SPARK-37263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37263: Assignee: Apache Spark > Add PandasAPIOnSparkAdviceWarning class >

[jira] [Resolved] (SPARK-37282) Add ExtendedLevelDBTest and disable LevelDB tests on Apple Silicon

[ https://issues.apache.org/jira/browse/SPARK-37282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37282. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34548

[jira] [Assigned] (SPARK-37282) Add ExtendedLevelDBTest and disable LevelDB tests on Apple Silicon

[ https://issues.apache.org/jira/browse/SPARK-37282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-37282: - Assignee: Dongjoon Hyun > Add ExtendedLevelDBTest and disable LevelDB tests on Apple

[jira] [Updated] (SPARK-37264) Exclude hadoop-client-api transitive dependency from orc-core

[ https://issues.apache.org/jira/browse/SPARK-37264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37264: -- Summary: Exclude hadoop-client-api transitive dependency from orc-core (was:

[jira] [Resolved] (SPARK-37255) When Used with PyHive (by dropbox) query timeout doesn't result in propagation to the UI

[ https://issues.apache.org/jira/browse/SPARK-37255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37255. -- Resolution: Invalid > When Used with PyHive (by dropbox) query timeout doesn't result in >

[jira] [Commented] (SPARK-37255) When Used with PyHive (by dropbox) query timeout doesn't result in propagation to the UI

[ https://issues.apache.org/jira/browse/SPARK-37255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17442058#comment-17442058 ] Hyukjin Kwon commented on SPARK-37255: -- That's very likely an issue in PyHive. > When Used with

[jira] [Updated] (SPARK-37278) Support YearMonthIntervalType in createDataFrame/toPandas and Python UDFs

[ https://issues.apache.org/jira/browse/SPARK-37278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37278: - Description: Implements the support of YearMonthIntervalType in: - Python UDFs -

[jira] [Created] (SPARK-37279) Support DayTimeIntervalType in createDataFrame/toPandas and Python UDFs

Hyukjin Kwon created SPARK-37279: Summary: Support DayTimeIntervalType in createDataFrame/toPandas and Python UDFs Key: SPARK-37279 URL: https://issues.apache.org/jira/browse/SPARK-37279 Project:

  1   2   >