[jira] [Created] (SPARK-33849) Unify v1 and v2 DROP TABLE tests

2020-12-19 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33849: -- Summary: Unify v1 and v2 DROP TABLE tests Key: SPARK-33849 URL: https://issues.apache.org/jira/browse/SPARK-33849 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-33838) Add comments about `PURGE` in DropTable and in AlterTableDropPartition

2020-12-18 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-33838: --- Summary: Add comments about `PURGE` in DropTable and in AlterTableDropPartition (was: Add comment

[jira] [Created] (SPARK-33838) Add comment about `PURGE` in DropTable and in AlterTableDropPartition

2020-12-18 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33838: -- Summary: Add comment about `PURGE` in DropTable and in AlterTableDropPartition Key: SPARK-33838 URL: https://issues.apache.org/jira/browse/SPARK-33838 Project: Spark

[jira] [Updated] (SPARK-33830) Describe the PURGE option of ALTER TABLE .. DROP PARTITION

2020-12-17 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-33830: --- Summary: Describe the PURGE option of ALTER TABLE .. DROP PARTITION (was: Describe the PURGE

[jira] [Created] (SPARK-33830) Describe the PURGE option of ALTER TABLE .. DROP PATITION

2020-12-17 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33830: -- Summary: Describe the PURGE option of ALTER TABLE .. DROP PATITION Key: SPARK-33830 URL: https://issues.apache.org/jira/browse/SPARK-33830 Project: Spark Issue

[jira] [Created] (SPARK-33789) Refactor unified V1 and V2 datasource tests

2020-12-15 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33789: -- Summary: Refactor unified V1 and V2 datasource tests Key: SPARK-33789 URL: https://issues.apache.org/jira/browse/SPARK-33789 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-33788) Throw NoSuchPartitionsException from HiveExternalCatalog.dropPartitions()

2020-12-15 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-33788: --- Description: HiveExternalCatalog.dropPartitions and as a consequence of that V1 Hive catalog throws

[jira] [Created] (SPARK-33788) Throw NoSuchPartitionsException from HiveExternalCatalog.dropPartitions()

2020-12-15 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33788: -- Summary: Throw NoSuchPartitionsException from HiveExternalCatalog.dropPartitions() Key: SPARK-33788 URL: https://issues.apache.org/jira/browse/SPARK-33788 Project: Spark

[jira] [Created] (SPARK-33787) Add `purge` to `dropPartition` in `SupportsPartitionManagement`

2020-12-14 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33787: -- Summary: Add `purge` to `dropPartition` in `SupportsPartitionManagement` Key: SPARK-33787 URL: https://issues.apache.org/jira/browse/SPARK-33787 Project: Spark

[jira] [Commented] (SPARK-33777) Sort output of V2 SHOW PARTITIONS

2020-12-14 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249120#comment-17249120 ] Maxim Gekk commented on SPARK-33777: I am working on this. > Sort output of V2 SHOW PARTITIONS >

[jira] [Created] (SPARK-33777) Sort output of SHOW PARTITIONS V2

2020-12-14 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33777: -- Summary: Sort output of SHOW PARTITIONS V2 Key: SPARK-33777 URL: https://issues.apache.org/jira/browse/SPARK-33777 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-33777) Sort output of V2 SHOW PARTITIONS

2020-12-14 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-33777: --- Summary: Sort output of V2 SHOW PARTITIONS (was: Sort output of SHOW PARTITIONS V2) > Sort output

[jira] [Created] (SPARK-33770) Test failures: ALTER TABLE .. DROP PARTITION tries to delete files out of partition path

2020-12-13 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33770: -- Summary: Test failures: ALTER TABLE .. DROP PARTITION tries to delete files out of partition path Key: SPARK-33770 URL: https://issues.apache.org/jira/browse/SPARK-33770

[jira] [Created] (SPARK-33768) Remove unused parameter `retainData` from AlterTableDropPartition

2020-12-12 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33768: -- Summary: Remove unused parameter `retainData` from AlterTableDropPartition Key: SPARK-33768 URL: https://issues.apache.org/jira/browse/SPARK-33768 Project: Spark

[jira] [Updated] (SPARK-33767) Unify v1 and v2 ALTER TABLE .. DROP PARTITION tests

2020-12-12 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-33767: --- Description: Extract ALTER TABLE .. DROP PARTITION tests to the common place to run them for V1 and

[jira] [Created] (SPARK-33767) Unify v1 and v2 ALTER TABLE .. DROP PARTITION tests

2020-12-12 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33767: -- Summary: Unify v1 and v2 ALTER TABLE .. DROP PARTITION tests Key: SPARK-33767 URL: https://issues.apache.org/jira/browse/SPARK-33767 Project: Spark Issue Type:

[jira] [Created] (SPARK-33742) Throw PartitionsAlreadyExistException from HiveExternalCatalog.createPartitions()

2020-12-10 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33742: -- Summary: Throw PartitionsAlreadyExistException from HiveExternalCatalog.createPartitions() Key: SPARK-33742 URL: https://issues.apache.org/jira/browse/SPARK-33742

[jira] [Commented] (SPARK-33571) Handling of hybrid to proleptic calendar when reading and writing Parquet data not working correctly

2020-12-09 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17246717#comment-17246717 ] Maxim Gekk commented on SPARK-33571: > The behavior of the to be introduced in Spark 3.1 >

[jira] [Updated] (SPARK-33558) Unify v1 and v2 ALTER TABLE .. ADD PARTITION tests

2020-12-09 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-33558: --- Description: Extract ALTER TABLE .. ADD PARTITION tests to the common place to run them for V1 and

[jira] [Updated] (SPARK-33558) Unify v1 and v2 ALTER TABLE .. ADD PARTITION tests

2020-12-09 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-33558: --- Summary: Unify v1 and v2 ALTER TABLE .. ADD PARTITION tests (was: Unify v1 and v2 ALTER TABLE ..

[jira] [Created] (SPARK-33706) Require fully specified partition identifier in partitionExists()

2020-12-07 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33706: -- Summary: Require fully specified partition identifier in partitionExists() Key: SPARK-33706 URL: https://issues.apache.org/jira/browse/SPARK-33706 Project: Spark

[jira] [Commented] (SPARK-33441) Add "unused-import" compile arg to scalac and remove all unused imports in Scala code

2020-12-07 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245723#comment-17245723 ] Maxim Gekk commented on SPARK-33441: Would it be possible to check unused imports in Java code? >

[jira] [Commented] (SPARK-33670) Verify the partition provider is Hive in v1 SHOW TABLE EXTENDED

2020-12-07 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245174#comment-17245174 ] Maxim Gekk commented on SPARK-33670: [~hyukjin.kwon] Just in case, which "Affects Version" should be

[jira] [Commented] (SPARK-33688) Migrate SHOW TABLE EXTENDED to new resolution framework

2020-12-07 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245143#comment-17245143 ] Maxim Gekk commented on SPARK-33688: I am working on this. > Migrate SHOW TABLE EXTENDED to new

[jira] [Created] (SPARK-33688) Migrate SHOW TABLE EXTENDED to new resolution framework

2020-12-07 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33688: -- Summary: Migrate SHOW TABLE EXTENDED to new resolution framework Key: SPARK-33688 URL: https://issues.apache.org/jira/browse/SPARK-33688 Project: Spark Issue

[jira] [Created] (SPARK-33676) Require exact matched partition spec to schema in ADD/DROP PARTITION

2020-12-06 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33676: -- Summary: Require exact matched partition spec to schema in ADD/DROP PARTITION Key: SPARK-33676 URL: https://issues.apache.org/jira/browse/SPARK-33676 Project: Spark

[jira] [Commented] (SPARK-33672) Check SQLContext.tables() for V2 session catalog

2020-12-05 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244542#comment-17244542 ] Maxim Gekk commented on SPARK-33672: [~cloud_fan] FYI > Check SQLContext.tables() for V2 session

[jira] [Created] (SPARK-33672) Check SQLContext.tables() for V2 session catalog

2020-12-05 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33672: -- Summary: Check SQLContext.tables() for V2 session catalog Key: SPARK-33672 URL: https://issues.apache.org/jira/browse/SPARK-33672 Project: Spark Issue Type:

[jira] [Created] (SPARK-33671) Remove VIEW checks from V1 table commands

2020-12-05 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33671: -- Summary: Remove VIEW checks from V1 table commands Key: SPARK-33671 URL: https://issues.apache.org/jira/browse/SPARK-33671 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-33670) Verify the partition provider is Hive in v1 SHOW TABLE EXTENDED

2020-12-05 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33670: -- Summary: Verify the partition provider is Hive in v1 SHOW TABLE EXTENDED Key: SPARK-33670 URL: https://issues.apache.org/jira/browse/SPARK-33670 Project: Spark

[jira] [Updated] (SPARK-33667) Respect case sensitivity in V1 SHOW PARTITIONS

2020-12-04 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-33667: --- Description: SHOW PARTITIONS is case sensitive, and doesn't respect the SQL config

[jira] [Created] (SPARK-33667) Respect case sensitivity in V1 SHOW PARTITIONS

2020-12-04 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33667: -- Summary: Respect case sensitivity in V1 SHOW PARTITIONS Key: SPARK-33667 URL: https://issues.apache.org/jira/browse/SPARK-33667 Project: Spark Issue Type:

[jira] [Commented] (SPARK-33571) Handling of hybrid to proleptic calendar when reading and writing Parquet data not working correctly

2020-12-03 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17243441#comment-17243441 ] Maxim Gekk commented on SPARK-33571: I opened the PR [https://github.com/apache/spark/pull/30596] 

[jira] [Created] (SPARK-33650) Misleading error from ALTER TABLE .. PARTITION for non-supported partition management table

2020-12-03 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33650: -- Summary: Misleading error from ALTER TABLE .. PARTITION for non-supported partition management table Key: SPARK-33650 URL: https://issues.apache.org/jira/browse/SPARK-33650

[jira] [Commented] (SPARK-33571) Handling of hybrid to proleptic calendar when reading and writing Parquet data not working correctly

2020-12-01 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17241408#comment-17241408 ] Maxim Gekk commented on SPARK-33571: [~simonvanderveldt] Looking at the dates, you tested, both

[jira] [Commented] (SPARK-33571) Handling of hybrid to proleptic calendar when reading and writing Parquet data not working correctly

2020-12-01 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17241400#comment-17241400 ] Maxim Gekk commented on SPARK-33571: Spark 3.0.1 shows different results as well: {code:scala}

[jira] [Commented] (SPARK-33571) Handling of hybrid to proleptic calendar when reading and writing Parquet data not working correctly

2020-12-01 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17241379#comment-17241379 ] Maxim Gekk commented on SPARK-33571: I have tried to reproduce the issue on the master branch by

[jira] [Commented] (SPARK-33571) Handling of hybrid to proleptic calendar when reading and writing Parquet data not working correctly

2020-12-01 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17241339#comment-17241339 ] Maxim Gekk commented on SPARK-33571: [~simonvanderveldt] Thank you for the detailed description and

[jira] [Updated] (SPARK-33591) NULL is recognized as the "null" string in partition specs

2020-11-29 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-33591: --- Issue Type: Bug (was: Improvement) > NULL is recognized as the "null" string in partition specs >

[jira] [Updated] (SPARK-33591) NULL is recognized as the "null" string in partition specs

2020-11-29 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-33591: --- Issue Type: Improvement (was: Bug) > NULL is recognized as the "null" string in partition specs >

[jira] [Created] (SPARK-33591) NULL is recognized as the "null" string in partition specs

2020-11-29 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33591: -- Summary: NULL is recognized as the "null" string in partition specs Key: SPARK-33591 URL: https://issues.apache.org/jira/browse/SPARK-33591 Project: Spark Issue

[jira] [Updated] (SPARK-33585) The comment for SQLContext.tables() doesn't mention the `database` column

2020-11-28 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-33585: --- Affects Version/s: (was: 2.4.7) (was: 3.0.1)

[jira] [Created] (SPARK-33588) Partition spec in SHOW TABLE EXTENDED doesn't respect `spark.sql.caseSensitive`

2020-11-28 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33588: -- Summary: Partition spec in SHOW TABLE EXTENDED doesn't respect `spark.sql.caseSensitive` Key: SPARK-33588 URL: https://issues.apache.org/jira/browse/SPARK-33588 Project:

[jira] [Created] (SPARK-33585) The comment for SQLContext.tables() doesn't mention the `database` column

2020-11-28 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33585: -- Summary: The comment for SQLContext.tables() doesn't mention the `database` column Key: SPARK-33585 URL: https://issues.apache.org/jira/browse/SPARK-33585 Project: Spark

[jira] [Created] (SPARK-33569) Remove getting partitions by only ident

2020-11-26 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33569: -- Summary: Remove getting partitions by only ident Key: SPARK-33569 URL: https://issues.apache.org/jira/browse/SPARK-33569 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-33558) Unify v1 and v2 ALTER TABLE .. PARTITION tests

2020-11-25 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33558: -- Summary: Unify v1 and v2 ALTER TABLE .. PARTITION tests Key: SPARK-33558 URL: https://issues.apache.org/jira/browse/SPARK-33558 Project: Spark Issue Type:

[jira] [Updated] (SPARK-33529) Resolver of V2 partition specs doesn't handle __HIVE_DEFAULT_PARTITION__

2020-11-24 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-33529: --- Description: The partition value '__HIVE_DEFAULT_PARTITION__' should be handled as null - the same

[jira] [Created] (SPARK-33529) Resolver of V2 partition specs doesn't handle __HIVE_DEFAULT_PARTITION__

2020-11-24 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33529: -- Summary: Resolver of V2 partition specs doesn't handle __HIVE_DEFAULT_PARTITION__ Key: SPARK-33529 URL: https://issues.apache.org/jira/browse/SPARK-33529 Project: Spark

[jira] [Created] (SPARK-33521) Universal type conversion of V2 partition values

2020-11-23 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33521: -- Summary: Universal type conversion of V2 partition values Key: SPARK-33521 URL: https://issues.apache.org/jira/browse/SPARK-33521 Project: Spark Issue Type:

[jira] [Created] (SPARK-33511) Respect case sensitivity in resolving partition specs V2

2020-11-21 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33511: -- Summary: Respect case sensitivity in resolving partition specs V2 Key: SPARK-33511 URL: https://issues.apache.org/jira/browse/SPARK-33511 Project: Spark Issue

[jira] [Created] (SPARK-33509) List partition by names from V2 tables that support partition management

2020-11-21 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33509: -- Summary: List partition by names from V2 tables that support partition management Key: SPARK-33509 URL: https://issues.apache.org/jira/browse/SPARK-33509 Project: Spark

[jira] [Created] (SPARK-33505) Fix insert into `InMemoryPartitionTable`

2020-11-20 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33505: -- Summary: Fix insert into `InMemoryPartitionTable` Key: SPARK-33505 URL: https://issues.apache.org/jira/browse/SPARK-33505 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-33453) Unify v1 and v2 SHOW PARTITIONS tests

2020-11-14 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33453: -- Summary: Unify v1 and v2 SHOW PARTITIONS tests Key: SPARK-33453 URL: https://issues.apache.org/jira/browse/SPARK-33453 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-33452) Create a V2 SHOW PARTITIONS execution node

2020-11-14 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33452: -- Summary: Create a V2 SHOW PARTITIONS execution node Key: SPARK-33452 URL: https://issues.apache.org/jira/browse/SPARK-33452 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-33452) Create a V2 SHOW PARTITIONS execution node

2020-11-14 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17232095#comment-17232095 ] Maxim Gekk commented on SPARK-33452: I plan to work on this soon. > Create a V2 SHOW PARTITIONS

[jira] [Commented] (SPARK-33393) Support SHOW TABLE EXTENDED in DSv2

2020-11-14 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17232094#comment-17232094 ] Maxim Gekk commented on SPARK-33393: I plan to work on this soon. > Support SHOW TABLE EXTENDED in

[jira] [Commented] (SPARK-33430) Support namespaces in JDBC v2 Table Catalog

2020-11-12 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17230514#comment-17230514 ] Maxim Gekk commented on SPARK-33430: [~cloud_fan] [~huaxingao] It would be nice to support

[jira] [Created] (SPARK-33430) Support namespaces in JDBC v2 Table Catalog

2020-11-12 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33430: -- Summary: Support namespaces in JDBC v2 Table Catalog Key: SPARK-33430 URL: https://issues.apache.org/jira/browse/SPARK-33430 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-33426) Unify Hive SHOW TABLES tests

2020-11-11 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33426: -- Summary: Unify Hive SHOW TABLES tests Key: SPARK-33426 URL: https://issues.apache.org/jira/browse/SPARK-33426 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-33403) DSv2 SHOW TABLES doesn't show `default`

2020-11-09 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33403: -- Summary: DSv2 SHOW TABLES doesn't show `default` Key: SPARK-33403 URL: https://issues.apache.org/jira/browse/SPARK-33403 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-33394) Throw `NoSuchDatabaseException` for not existing namespace in DSv2 SHOW TABLES

2020-11-09 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33394: -- Summary: Throw `NoSuchDatabaseException` for not existing namespace in DSv2 SHOW TABLES Key: SPARK-33394 URL: https://issues.apache.org/jira/browse/SPARK-33394 Project:

[jira] [Created] (SPARK-33393) Support SHOW TABLE EXTENDED in DSv2

2020-11-09 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33393: -- Summary: Support SHOW TABLE EXTENDED in DSv2 Key: SPARK-33393 URL: https://issues.apache.org/jira/browse/SPARK-33393 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-33364) Expose purge option in TableCatalog.dropTable

2020-11-09 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-33364: --- Parent: SPARK-33392 Issue Type: Sub-task (was: New Feature) > Expose purge option in

[jira] [Updated] (SPARK-33305) DSv2: DROP TABLE command should also invalidate cache

2020-11-09 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-33305: --- Parent: SPARK-33392 Issue Type: Sub-task (was: Bug) > DSv2: DROP TABLE command should also

[jira] [Created] (SPARK-33392) Align DSv2 commands to DSv1 implementation

2020-11-09 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33392: -- Summary: Align DSv2 commands to DSv1 implementation Key: SPARK-33392 URL: https://issues.apache.org/jira/browse/SPARK-33392 Project: Spark Issue Type: Umbrella

[jira] [Created] (SPARK-33382) Unify v1 and v2 SHOW TABLES tests

2020-11-07 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33382: -- Summary: Unify v1 and v2 SHOW TABLES tests Key: SPARK-33382 URL: https://issues.apache.org/jira/browse/SPARK-33382 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-33381) Unify dsv1 and dsv2 command tests

2020-11-07 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33381: -- Summary: Unify dsv1 and dsv2 command tests Key: SPARK-33381 URL: https://issues.apache.org/jira/browse/SPARK-33381 Project: Spark Issue Type: Test

[jira] [Updated] (SPARK-33381) Unify DSv1 and DSv2 command tests

2020-11-07 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-33381: --- Summary: Unify DSv1 and DSv2 command tests (was: Unify dsv1 and dsv2 command tests) > Unify DSv1

[jira] [Created] (SPARK-33299) Unify schema parsing in from_json/from_csv across all APIs

2020-10-30 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33299: -- Summary: Unify schema parsing in from_json/from_csv across all APIs Key: SPARK-33299 URL: https://issues.apache.org/jira/browse/SPARK-33299 Project: Spark Issue

[jira] [Created] (SPARK-33286) Misleading error messages of `from_json`

2020-10-29 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33286: -- Summary: Misleading error messages of `from_json` Key: SPARK-33286 URL: https://issues.apache.org/jira/browse/SPARK-33286 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-33281) Return SQL schema instead of Catalog string from the `SchemaOfCsv` expression

2020-10-29 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33281: -- Summary: Return SQL schema instead of Catalog string from the `SchemaOfCsv` expression Key: SPARK-33281 URL: https://issues.apache.org/jira/browse/SPARK-33281 Project:

[jira] [Created] (SPARK-33270) from_json() cannot parse schema from schema_of_json()

2020-10-28 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33270: -- Summary: from_json() cannot parse schema from schema_of_json() Key: SPARK-33270 URL: https://issues.apache.org/jira/browse/SPARK-33270 Project: Spark Issue

[jira] [Commented] (SPARK-31342) Fail by default if Parquet DATE or TIMESTAMP data is before October 15, 1582

2020-10-25 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220274#comment-17220274 ] Maxim Gekk commented on SPARK-31342: [~cloud_fan] I think we can close this because it has been

[jira] [Created] (SPARK-33210) Set the rebasing mode for parquet INT96 type to `EXCEPTION` by default

2020-10-21 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33210: -- Summary: Set the rebasing mode for parquet INT96 type to `EXCEPTION` by default Key: SPARK-33210 URL: https://issues.apache.org/jira/browse/SPARK-33210 Project: Spark

[jira] [Created] (SPARK-33169) Check propagation of datasource options to underlying file system for built-in file-based datasources

2020-10-16 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33169: -- Summary: Check propagation of datasource options to underlying file system for built-in file-based datasources Key: SPARK-33169 URL: https://issues.apache.org/jira/browse/SPARK-33169

[jira] [Created] (SPARK-33163) Check the metadata key 'org.apache.spark.legacyDateTime' in Avro/Parquet files

2020-10-15 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33163: -- Summary: Check the metadata key 'org.apache.spark.legacyDateTime' in Avro/Parquet files Key: SPARK-33163 URL: https://issues.apache.org/jira/browse/SPARK-33163 Project:

[jira] [Commented] (SPARK-33160) Allow saving/loading INT96 in parquet w/o rebasing

2020-10-15 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17214643#comment-17214643 ] Maxim Gekk commented on SPARK-33160: I am working on this. > Allow saving/loading INT96 in parquet

[jira] [Created] (SPARK-33160) Allow saving/loading INT96 in parquet w/o rebasing

2020-10-15 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33160: -- Summary: Allow saving/loading INT96 in parquet w/o rebasing Key: SPARK-33160 URL: https://issues.apache.org/jira/browse/SPARK-33160 Project: Spark Issue Type:

[jira] [Updated] (SPARK-33134) Incorrect nested complex JSON fields raise an exception

2020-10-13 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-33134: --- Description: The code below: {code:scala} val pokerhand_raw = Seq("""[{"cards": [19], "playerId":

[jira] [Updated] (SPARK-33134) Incorrect nested complex JSON fields raise an exception

2020-10-13 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-33134: --- Description: The code below: {code:scala} val pokerhand_raw = Seq("""[{"cards": [11],

[jira] [Created] (SPARK-33134) Incorrect nested complex JSON fields raise an exception

2020-10-13 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33134: -- Summary: Incorrect nested complex JSON fields raise an exception Key: SPARK-33134 URL: https://issues.apache.org/jira/browse/SPARK-33134 Project: Spark Issue

[jira] [Commented] (SPARK-31294) Benchmark the performance regression

2020-10-12 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17212199#comment-17212199 ] Maxim Gekk commented on SPARK-31294: [~cloud_fan] Could you close this ticket since the PR was

[jira] [Updated] (SPARK-33101) LibSVM format does not propagate Hadoop config from DS options to underlying HDFS file system

2020-10-09 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-33101: --- Description: When running: {code:java} spark.read.format("libsvm").options(conf).load(path) {code}

[jira] [Created] (SPARK-33101) LibSVM format does not propagate Hadoop config from DS options to underlying HDFS file system

2020-10-09 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33101: -- Summary: LibSVM format does not propagate Hadoop config from DS options to underlying HDFS file system Key: SPARK-33101 URL: https://issues.apache.org/jira/browse/SPARK-33101

[jira] [Created] (SPARK-33094) ORC format does not propagate Hadoop config from DS options to underlying HDFS file system

2020-10-08 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33094: -- Summary: ORC format does not propagate Hadoop config from DS options to underlying HDFS file system Key: SPARK-33094 URL: https://issues.apache.org/jira/browse/SPARK-33094

[jira] [Updated] (SPARK-33094) ORC format does not propagate Hadoop config from DS options to underlying HDFS file system

2020-10-08 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-33094: --- Description: When running: {code:java} spark.read.format("orc").options(conf).load(path) {code} The

[jira] [Updated] (SPARK-33074) Classify dialect exceptions in JDBC v2 Table Catalog

2020-10-07 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-33074: --- Parent: SPARK-24907 Issue Type: Sub-task (was: Improvement) > Classify dialect exceptions

[jira] [Created] (SPARK-33074) Classify dialect exceptions in JDBC v2 Table Catalog

2020-10-06 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33074: -- Summary: Classify dialect exceptions in JDBC v2 Table Catalog Key: SPARK-33074 URL: https://issues.apache.org/jira/browse/SPARK-33074 Project: Spark Issue Type:

[jira] [Created] (SPARK-33067) Add negative checks to JDBC v2 Table Catalog tests

2020-10-05 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33067: -- Summary: Add negative checks to JDBC v2 Table Catalog tests Key: SPARK-33067 URL: https://issues.apache.org/jira/browse/SPARK-33067 Project: Spark Issue Type:

[jira] [Created] (SPARK-33066) Port docker integration tests to JDBC v2

2020-10-05 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33066: -- Summary: Port docker integration tests to JDBC v2 Key: SPARK-33066 URL: https://issues.apache.org/jira/browse/SPARK-33066 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-33034) Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns (Oracle dialect)

2020-09-30 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-33034: --- Summary: Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns

[jira] [Commented] (SPARK-33034) Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns

2020-09-30 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204672#comment-17204672 ] Maxim Gekk commented on SPARK-33034: I am working on this. > Support ALTER TABLE in JDBC v2 Table

[jira] [Created] (SPARK-33034) Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns

2020-09-30 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33034: -- Summary: Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns Key: SPARK-33034 URL: https://issues.apache.org/jira/browse/SPARK-33034

[jira] [Created] (SPARK-33015) Compute the current date only once

2020-09-28 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33015: -- Summary: Compute the current date only once Key: SPARK-33015 URL: https://issues.apache.org/jira/browse/SPARK-33015 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-33015) Compute the current date only once

2020-09-28 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203084#comment-17203084 ] Maxim Gekk commented on SPARK-33015: I am working on this > Compute the current date only once >

[jira] [Commented] (SPARK-32306) `approx_percentile` in Spark SQL gives incorrect results

2020-09-22 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199952#comment-17199952 ] Maxim Gekk commented on SPARK-32306: I opened PR https://github.com/apache/spark/pull/29835 with

[jira] [Commented] (SPARK-32306) `approx_percentile` in Spark SQL gives incorrect results

2020-09-22 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199839#comment-17199839 ] Maxim Gekk commented on SPARK-32306: The function returns an element of the input sequence, see

[jira] [Commented] (SPARK-27345) Expose approx_percentile in python api

2020-09-21 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199618#comment-17199618 ] Maxim Gekk commented on SPARK-27345: The function was exposed recently by

[jira] [Commented] (SPARK-32306) `approx_percentile` in Spark SQL gives incorrect results

2020-09-21 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199615#comment-17199615 ] Maxim Gekk commented on SPARK-32306: [~seanmalory] Which result do you expect? 8? >

[jira] [Commented] (SPARK-30882) Inaccurate results with higher precision ApproximatePercentile results

2020-09-21 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199602#comment-17199602 ] Maxim Gekk commented on SPARK-30882: This PR [https://github.com/apache/spark/pull/29784] solves the

<    1   2   3   4   5   6   7   8   9   10   >