[jira] [Assigned] (SPARK-34567) CreateTableAsSelect should have metrics update too

2021-02-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34567: Assignee: (was: Apache Spark) > CreateTableAsSelect should have metrics update too >

[jira] [Assigned] (SPARK-34567) CreateTableAsSelect should have metrics update too

2021-02-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34567: Assignee: Apache Spark > CreateTableAsSelect should have metrics update too >

[jira] [Commented] (SPARK-34567) CreateTableAsSelect should have metrics update too

2021-02-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17292325#comment-17292325 ] Apache Spark commented on SPARK-34567: -- User 'AngersZh' has created a pull request for this

[jira] [Commented] (SPARK-34567) CreateTableAsSelect should have metrics update too

2021-02-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17292324#comment-17292324 ] Apache Spark commented on SPARK-34567: -- User 'AngersZh' has created a pull request for this

[jira] [Created] (SPARK-34567) CreateTableAsSelect should have metrics update too

2021-02-27 Thread angerszhu (Jira)
angerszhu created SPARK-34567: - Summary: CreateTableAsSelect should have metrics update too Key: SPARK-34567 URL: https://issues.apache.org/jira/browse/SPARK-34567 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-34567) CreateTableAsSelect should have metrics update too

2021-02-27 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17292301#comment-17292301 ] angerszhu commented on SPARK-34567: --- raise a pr soon > CreateTableAsSelect should have metrics update

[jira] [Updated] (SPARK-34543) Respect case sensitivity in V1 ALTER TABLE .. SET LOCATION

2021-02-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34543: -- Fix Version/s: 3.0.3 > Respect case sensitivity in V1 ALTER TABLE .. SET LOCATION >

[jira] [Assigned] (SPARK-34566) Fix typo error of `spark.launcher.childConectionTimeout`

2021-02-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34566: Assignee: Apache Spark > Fix typo error of `spark.launcher.childConectionTimeout` >

[jira] [Commented] (SPARK-34566) Fix typo error of `spark.launcher.childConectionTimeout`

2021-02-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17292290#comment-17292290 ] Apache Spark commented on SPARK-34566: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-34566) Fix typo error of `spark.launcher.childConectionTimeout`

2021-02-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34566: Assignee: (was: Apache Spark) > Fix typo error of

[jira] [Created] (SPARK-34566) Fix typo error of `spark.launcher.childConectionTimeout`

2021-02-27 Thread angerszhu (Jira)
angerszhu created SPARK-34566: - Summary: Fix typo error of `spark.launcher.childConectionTimeout` Key: SPARK-34566 URL: https://issues.apache.org/jira/browse/SPARK-34566 Project: Spark Issue

[jira] [Commented] (SPARK-34562) Leverage parquet bloom filters

2021-02-27 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17292256#comment-17292256 ] Yuming Wang commented on SPARK-34562: - [~h-vetinari] This is an example to build bloom filter:

[jira] [Updated] (SPARK-34562) Leverage parquet bloom filters

2021-02-27 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-34562: Component/s: (was: Input/Output) SQL > Leverage parquet bloom filters >

[jira] [Commented] (SPARK-34565) Collapse Window nodes with Project between them

2021-02-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17292251#comment-17292251 ] Apache Spark commented on SPARK-34565: -- User 'tanelk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34565) Collapse Window nodes with Project between them

2021-02-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34565: Assignee: Apache Spark > Collapse Window nodes with Project between them >

[jira] [Commented] (SPARK-34565) Collapse Window nodes with Project between them

2021-02-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17292250#comment-17292250 ] Apache Spark commented on SPARK-34565: -- User 'tanelk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34565) Collapse Window nodes with Project between them

2021-02-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34565: Assignee: (was: Apache Spark) > Collapse Window nodes with Project between them >

[jira] [Created] (SPARK-34565) Collapse Window nodes with Project between them

2021-02-27 Thread Tanel Kiis (Jira)
Tanel Kiis created SPARK-34565: -- Summary: Collapse Window nodes with Project between them Key: SPARK-34565 URL: https://issues.apache.org/jira/browse/SPARK-34565 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-34542) Upgrade Parquet to 1.12.0

2021-02-27 Thread H. Vetinari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17292239#comment-17292239 ] H. Vetinari commented on SPARK-34542: - Would be amazing if this could make it into 3.2, given all

[jira] [Updated] (SPARK-34564) DateTimeUtils.fromJavaDate fails for very late dates during casting to Int

2021-02-27 Thread kondziolka9ld (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kondziolka9ld updated SPARK-34564: -- Description: Please consider a following scenario on *spark-3.0.1*:  {code:java} scala>

[jira] [Updated] (SPARK-34564) DateTimeUtils.fromJavaDate fails for very late dates during casting to Int

2021-02-27 Thread kondziolka9ld (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kondziolka9ld updated SPARK-34564: -- Description: Please consider a following scenario on *spark-3.0.1*:  {code:java} scala>

[jira] [Created] (SPARK-34564) DateTimeUtils.fromJavaDate fails for very late dates during casting to Int

2021-02-27 Thread kondziolka9ld (Jira)
kondziolka9ld created SPARK-34564: - Summary: DateTimeUtils.fromJavaDate fails for very late dates during casting to Int Key: SPARK-34564 URL: https://issues.apache.org/jira/browse/SPARK-34564

[jira] [Updated] (SPARK-34563) Checkpointing a union with another checkpoint fails

2021-02-27 Thread Michael Kamprath (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Kamprath updated SPARK-34563: - Description: I have some PySpark code that periodically checkpoints a data frame  that

[jira] [Created] (SPARK-34563) Checkpointing a union with another checkpoint fails

2021-02-27 Thread Michael Kamprath (Jira)
Michael Kamprath created SPARK-34563: Summary: Checkpointing a union with another checkpoint fails Key: SPARK-34563 URL: https://issues.apache.org/jira/browse/SPARK-34563 Project: Spark

[jira] [Assigned] (SPARK-34479) Add zstandard codec to spark.sql.avro.compression.codec

2021-02-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34479: - Assignee: Yuming Wang > Add zstandard codec to spark.sql.avro.compression.codec >

[jira] [Resolved] (SPARK-34479) Add zstandard codec to spark.sql.avro.compression.codec

2021-02-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34479. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31673

[jira] [Updated] (SPARK-34562) Leverage parquet bloom filters

2021-02-27 Thread H. Vetinari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] H. Vetinari updated SPARK-34562: Description: The currently in-progress SPARK-34542 brings in parquet 1.12, which contains

[jira] [Updated] (SPARK-34562) Leverage parquet bloom filters

2021-02-27 Thread H. Vetinari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] H. Vetinari updated SPARK-34562: Issue Type: Improvement (was: Task) > Leverage parquet bloom filters >

[jira] [Created] (SPARK-34562) Leverage parquet bloom filters

2021-02-27 Thread H. Vetinari (Jira)
H. Vetinari created SPARK-34562: --- Summary: Leverage parquet bloom filters Key: SPARK-34562 URL: https://issues.apache.org/jira/browse/SPARK-34562 Project: Spark Issue Type: Task

[jira] [Commented] (SPARK-34561) Cannot drop/add columns from/to a dataset of v2 `DESCRIBE TABLE`

2021-02-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17292192#comment-17292192 ] Apache Spark commented on SPARK-34561: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34561) Cannot drop/add columns from/to a dataset of v2 `DESCRIBE TABLE`

2021-02-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34561: Assignee: Apache Spark > Cannot drop/add columns from/to a dataset of v2 `DESCRIBE

[jira] [Assigned] (SPARK-34561) Cannot drop/add columns from/to a dataset of v2 `DESCRIBE TABLE`

2021-02-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34561: Assignee: (was: Apache Spark) > Cannot drop/add columns from/to a dataset of v2

[jira] [Created] (SPARK-34561) Cannot drop/add columns from/to a dataset of v2 `DESCRIBE TABLE`

2021-02-27 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34561: -- Summary: Cannot drop/add columns from/to a dataset of v2 `DESCRIBE TABLE` Key: SPARK-34561 URL: https://issues.apache.org/jira/browse/SPARK-34561 Project: Spark

[jira] [Resolved] (SPARK-34415) Use randomization as a possibly better technique than grid search in optimizing hyperparameters

2021-02-27 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-34415. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31535

[jira] [Assigned] (SPARK-34415) Use randomization as a possibly better technique than grid search in optimizing hyperparameters

2021-02-27 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-34415: Assignee: Phillip Henry > Use randomization as a possibly better technique than grid

[jira] [Updated] (SPARK-34392) Invalid ID for offset-based ZoneId since Spark 3.0

2021-02-27 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-34392: - Fix Version/s: 3.1.2 > Invalid ID for offset-based ZoneId since Spark 3.0 >

[jira] [Assigned] (SPARK-34560) Cannot join datasets of SHOW TABLES

2021-02-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34560: Assignee: (was: Apache Spark) > Cannot join datasets of SHOW TABLES >

[jira] [Commented] (SPARK-34560) Cannot join datasets of SHOW TABLES

2021-02-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17292128#comment-17292128 ] Apache Spark commented on SPARK-34560: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-34560) Cannot join datasets of SHOW TABLES

2021-02-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17292129#comment-17292129 ] Apache Spark commented on SPARK-34560: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34560) Cannot join datasets of SHOW TABLES

2021-02-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34560: Assignee: Apache Spark > Cannot join datasets of SHOW TABLES >

[jira] [Created] (SPARK-34560) Cannot join datasets of SHOW TABLES

2021-02-27 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34560: -- Summary: Cannot join datasets of SHOW TABLES Key: SPARK-34560 URL: https://issues.apache.org/jira/browse/SPARK-34560 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-34559) Upgrade to ZSTD JNI 1.4.8-6

2021-02-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34559. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31674

[jira] [Assigned] (SPARK-34559) Upgrade to ZSTD JNI 1.4.8-6

2021-02-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34559: - Assignee: Dongjoon Hyun > Upgrade to ZSTD JNI 1.4.8-6 > --- >

[jira] [Updated] (SPARK-34447) Refactor the unified v1 and v2 command tests

2021-02-27 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-34447: --- Description: The ticket aims to gather potential improvements for the unified tests. 1. Remove

[jira] [Updated] (SPARK-34559) Upgrade to ZSTD JNI 1.4.8-6

2021-02-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34559: -- Summary: Upgrade to ZSTD JNI 1.4.8-6 (was: Upgrade to ZSTD JNI 1.4.6) > Upgrade to ZSTD JNI

[jira] [Assigned] (SPARK-34559) Upgrade to ZSTD JNI 1.4.6

2021-02-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34559: Assignee: Apache Spark > Upgrade to ZSTD JNI 1.4.6 > - > >

[jira] [Commented] (SPARK-34559) Upgrade to ZSTD JNI 1.4.6

2021-02-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17292076#comment-17292076 ] Apache Spark commented on SPARK-34559: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-34559) Upgrade to ZSTD JNI 1.4.6

2021-02-27 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34559: Assignee: (was: Apache Spark) > Upgrade to ZSTD JNI 1.4.6 >

[jira] [Created] (SPARK-34559) Upgrade to ZSTD JNI 1.4.6

2021-02-27 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-34559: - Summary: Upgrade to ZSTD JNI 1.4.6 Key: SPARK-34559 URL: https://issues.apache.org/jira/browse/SPARK-34559 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-34557) Exclude Avro's transitive zstd-jni dependency

2021-02-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34557: - Assignee: Dongjoon Hyun > Exclude Avro's transitive zstd-jni dependency >

[jira] [Resolved] (SPARK-34557) Exclude Avro's transitive zstd-jni dependency

2021-02-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34557. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31670