[jira] [Commented] (SPARK-40798) Alter partition should verify value
[
https://issues.apache.org/jira/browse/SPARK-40798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17634614#comment-17634614
]
Apache Spark commented on SPARK-40798:
--
User 'HyukjinKwon' has created a pull request for this issue:
https://github.com/apache/spark/pull/38667
> Alter partition should verify value
> ---
>
> Key: SPARK-40798
> URL: https://issues.apache.org/jira/browse/SPARK-40798
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: XiDuo You
>Assignee: XiDuo You
>Priority: Major
> Fix For: 3.4.0
>
>
>
> {code:java}
> CREATE TABLE t (c int) USING PARQUET PARTITIONED BY(p int);
> -- This DDL should fail but worked:
> ALTER TABLE t ADD PARTITION(p='aaa'); {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40798) Alter partition should verify value
[
https://issues.apache.org/jira/browse/SPARK-40798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17633489#comment-17633489
]
XiDuo You commented on SPARK-40798:
---
thank you [~rangareddy.av...@gmail.com]
> Alter partition should verify value
> ---
>
> Key: SPARK-40798
> URL: https://issues.apache.org/jira/browse/SPARK-40798
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: XiDuo You
>Assignee: XiDuo You
>Priority: Major
> Fix For: 3.4.0
>
>
>
> {code:java}
> CREATE TABLE t (c int) USING PARQUET PARTITIONED BY(p int);
> -- This DDL should fail but worked:
> ALTER TABLE t ADD PARTITION(p='aaa'); {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40798) Alter partition should verify value
[
https://issues.apache.org/jira/browse/SPARK-40798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17633488#comment-17633488
]
Ranga Reddy commented on SPARK-40798:
-
Hi [~ulysses]
I will go ahead and add a test case for the *insert* operation and update the
Jira description.
> Alter partition should verify value
> ---
>
> Key: SPARK-40798
> URL: https://issues.apache.org/jira/browse/SPARK-40798
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: XiDuo You
>Assignee: XiDuo You
>Priority: Major
> Fix For: 3.4.0
>
>
>
> {code:java}
> CREATE TABLE t (c int) USING PARQUET PARTITIONED BY(p int);
> -- This DDL should fail but worked:
> ALTER TABLE t ADD PARTITION(p='aaa'); {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40798) Alter partition should verify value
[
https://issues.apache.org/jira/browse/SPARK-40798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17633486#comment-17633486
]
XiDuo You commented on SPARK-40798:
---
I think it's fine, the semantics of insert is `insert data` + `add partition`.
> Alter partition should verify value
> ---
>
> Key: SPARK-40798
> URL: https://issues.apache.org/jira/browse/SPARK-40798
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: XiDuo You
>Assignee: XiDuo You
>Priority: Major
> Fix For: 3.4.0
>
>
>
> {code:java}
> CREATE TABLE t (c int) USING PARQUET PARTITIONED BY(p int);
> -- This DDL should fail but worked:
> ALTER TABLE t ADD PARTITION(p='aaa'); {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40798) Alter partition should verify value
[
https://issues.apache.org/jira/browse/SPARK-40798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17633471#comment-17633471
]
Ranga Reddy commented on SPARK-40798:
-
Hi [~ulysses]
*SPARK-40988* is created for validating the value type while inserting a
partition and *SPARK-40798* is created for validating the value while altering
a partition. *SPARK-40798* Jira will fix the *validating partition value* for
both *insert* and *alter* partitions. So, we can use the *same configuration*
parameter for both *Insert* and *Alter* partitions?
> Alter partition should verify value
> ---
>
> Key: SPARK-40798
> URL: https://issues.apache.org/jira/browse/SPARK-40798
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: XiDuo You
>Assignee: XiDuo You
>Priority: Major
> Fix For: 3.4.0
>
>
>
> {code:java}
> CREATE TABLE t (c int) USING PARQUET PARTITIONED BY(p int);
> -- This DDL should fail but worked:
> ALTER TABLE t ADD PARTITION(p='aaa'); {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40798) Alter partition should verify value
[
https://issues.apache.org/jira/browse/SPARK-40798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17633429#comment-17633429
]
XiDuo You commented on SPARK-40798:
---
I think the name should be fine without change. You are actually altering
partition when do insert a partitioned table, right ?
> Alter partition should verify value
> ---
>
> Key: SPARK-40798
> URL: https://issues.apache.org/jira/browse/SPARK-40798
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: XiDuo You
>Assignee: XiDuo You
>Priority: Major
> Fix For: 3.4.0
>
>
>
> {code:java}
> CREATE TABLE t (c int) USING PARQUET PARTITIONED BY(p int);
> -- This DDL should fail but worked:
> ALTER TABLE t ADD PARTITION(p='aaa'); {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40798) Alter partition should verify value
[
https://issues.apache.org/jira/browse/SPARK-40798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17633209#comment-17633209
]
Ranga Reddy commented on SPARK-40798:
-
Hi [~ulysses]
The current Jira will solve both the *Insert* and *Alter* partition values.
{code:java}
CREATE EXTERNAL TABLE test_partition_value_tbl ( id INT, name STRING )
PARTITIONED BY (age INT) LOCATION
'file:/tmp/spark-warehouse/test_partition_value_tbl'
-- This DDL should fail but worked:
ALTER TABLE test_partition_value_tbl ADD PARTITION(age='aaa');
-- This DML should fail but worked:
INSERT INTO test_partition_value_tbl PARTITION(age="aaa") VALUES (1,
'ABC'){code}
Can we rename *SKIP_TYPE_VALIDATION_ON_ALTER_PARTITION* variable and update the
description of this variable?
After that, I will commit my test case.
> Alter partition should verify value
> ---
>
> Key: SPARK-40798
> URL: https://issues.apache.org/jira/browse/SPARK-40798
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: XiDuo You
>Assignee: XiDuo You
>Priority: Major
> Fix For: 3.4.0
>
>
>
> {code:java}
> CREATE TABLE t (c int) USING PARQUET PARTITIONED BY(p int);
> -- This DDL should fail but worked:
> ALTER TABLE t ADD PARTITION(p='aaa'); {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40798) Alter partition should verify value
[
https://issues.apache.org/jira/browse/SPARK-40798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17632413#comment-17632413
]
XiDuo You commented on SPARK-40798:
---
[~rangareddy.av...@gmail.com] sure, just go ahead
> Alter partition should verify value
> ---
>
> Key: SPARK-40798
> URL: https://issues.apache.org/jira/browse/SPARK-40798
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: XiDuo You
>Assignee: XiDuo You
>Priority: Major
> Fix For: 3.4.0
>
>
>
> {code:java}
> CREATE TABLE t (c int) USING PARQUET PARTITIONED BY(p int);
> -- This DDL should fail but worked:
> ALTER TABLE t ADD PARTITION(p='aaa'); {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40798) Alter partition should verify value
[
https://issues.apache.org/jira/browse/SPARK-40798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17632106#comment-17632106
]
Ranga Reddy commented on SPARK-40798:
-
The below issue is addressed in the current Jira. Can I add a test case for the
below issue in InsertSuite.scala?
https://issues.apache.org/jira/browse/SPARK-40988
> Alter partition should verify value
> ---
>
> Key: SPARK-40798
> URL: https://issues.apache.org/jira/browse/SPARK-40798
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: XiDuo You
>Assignee: XiDuo You
>Priority: Major
> Fix For: 3.4.0
>
>
>
> {code:java}
> CREATE TABLE t (c int) USING PARQUET PARTITIONED BY(p int);
> -- This DDL should fail but worked:
> ALTER TABLE t ADD PARTITION(p='aaa'); {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40798) Alter partition should verify value
[
https://issues.apache.org/jira/browse/SPARK-40798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17630870#comment-17630870
]
Apache Spark commented on SPARK-40798:
--
User 'ulysses-you' has created a pull request for this issue:
https://github.com/apache/spark/pull/38580
> Alter partition should verify value
> ---
>
> Key: SPARK-40798
> URL: https://issues.apache.org/jira/browse/SPARK-40798
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: XiDuo You
>Assignee: XiDuo You
>Priority: Major
> Fix For: 3.4.0
>
>
>
> {code:java}
> CREATE TABLE t (c int) USING PARQUET PARTITIONED BY(p int);
> -- This DDL should fail but worked:
> ALTER TABLE t ADD PARTITION(p='aaa'); {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40798) Alter partition should verify value
[
https://issues.apache.org/jira/browse/SPARK-40798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17630869#comment-17630869
]
Apache Spark commented on SPARK-40798:
--
User 'ulysses-you' has created a pull request for this issue:
https://github.com/apache/spark/pull/38580
> Alter partition should verify value
> ---
>
> Key: SPARK-40798
> URL: https://issues.apache.org/jira/browse/SPARK-40798
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: XiDuo You
>Assignee: XiDuo You
>Priority: Major
> Fix For: 3.4.0
>
>
>
> {code:java}
> CREATE TABLE t (c int) USING PARQUET PARTITIONED BY(p int);
> -- This DDL should fail but worked:
> ALTER TABLE t ADD PARTITION(p='aaa'); {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40798) Alter partition should verify value
[
https://issues.apache.org/jira/browse/SPARK-40798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17630147#comment-17630147
]
Apache Spark commented on SPARK-40798:
--
User 'HyukjinKwon' has created a pull request for this issue:
https://github.com/apache/spark/pull/38547
> Alter partition should verify value
> ---
>
> Key: SPARK-40798
> URL: https://issues.apache.org/jira/browse/SPARK-40798
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: XiDuo You
>Assignee: XiDuo You
>Priority: Major
> Fix For: 3.4.0
>
>
>
> {code:java}
> CREATE TABLE t (c int) USING PARQUET PARTITIONED BY(p int);
> -- This DDL should fail but worked:
> ALTER TABLE t ADD PARTITION(p='aaa'); {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40798) Alter partition should verify value
[
https://issues.apache.org/jira/browse/SPARK-40798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17630146#comment-17630146
]
Apache Spark commented on SPARK-40798:
--
User 'HyukjinKwon' has created a pull request for this issue:
https://github.com/apache/spark/pull/38547
> Alter partition should verify value
> ---
>
> Key: SPARK-40798
> URL: https://issues.apache.org/jira/browse/SPARK-40798
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: XiDuo You
>Assignee: XiDuo You
>Priority: Major
> Fix For: 3.4.0
>
>
>
> {code:java}
> CREATE TABLE t (c int) USING PARQUET PARTITIONED BY(p int);
> -- This DDL should fail but worked:
> ALTER TABLE t ADD PARTITION(p='aaa'); {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40798) Alter partition should verify value
[
https://issues.apache.org/jira/browse/SPARK-40798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17626556#comment-17626556
]
Apache Spark commented on SPARK-40798:
--
User 'ulysses-you' has created a pull request for this issue:
https://github.com/apache/spark/pull/38449
> Alter partition should verify value
> ---
>
> Key: SPARK-40798
> URL: https://issues.apache.org/jira/browse/SPARK-40798
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: XiDuo You
>Assignee: XiDuo You
>Priority: Major
> Fix For: 3.4.0
>
>
>
> {code:java}
> CREATE TABLE t (c int) USING PARQUET PARTITIONED BY(p int);
> -- This DDL should fail but worked:
> ALTER TABLE t ADD PARTITION(p='aaa'); {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40798) Alter partition should verify value
[
https://issues.apache.org/jira/browse/SPARK-40798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17626367#comment-17626367
]
Apache Spark commented on SPARK-40798:
--
User 'HyukjinKwon' has created a pull request for this issue:
https://github.com/apache/spark/pull/38443
> Alter partition should verify value
> ---
>
> Key: SPARK-40798
> URL: https://issues.apache.org/jira/browse/SPARK-40798
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: XiDuo You
>Assignee: XiDuo You
>Priority: Major
> Fix For: 3.4.0
>
>
>
> {code:java}
> CREATE TABLE t (c int) USING PARQUET PARTITIONED BY(p int);
> -- This DDL should fail but worked:
> ALTER TABLE t ADD PARTITION(p='aaa'); {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40798) Alter partition should verify value
[
https://issues.apache.org/jira/browse/SPARK-40798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17617617#comment-17617617
]
Apache Spark commented on SPARK-40798:
--
User 'ulysses-you' has created a pull request for this issue:
https://github.com/apache/spark/pull/38257
> Alter partition should verify value
> ---
>
> Key: SPARK-40798
> URL: https://issues.apache.org/jira/browse/SPARK-40798
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: XiDuo You
>Priority: Major
>
>
> {code:java}
> CREATE TABLE t (c int) USING PARQUET PARTITIONED BY(p int);
> -- This DDL should fail but worked:
> ALTER TABLE t ADD PARTITION(p='aaa'); {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40798) Alter partition should verify value
[
https://issues.apache.org/jira/browse/SPARK-40798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17617619#comment-17617619
]
Apache Spark commented on SPARK-40798:
--
User 'ulysses-you' has created a pull request for this issue:
https://github.com/apache/spark/pull/38257
> Alter partition should verify value
> ---
>
> Key: SPARK-40798
> URL: https://issues.apache.org/jira/browse/SPARK-40798
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: XiDuo You
>Priority: Major
>
>
> {code:java}
> CREATE TABLE t (c int) USING PARQUET PARTITIONED BY(p int);
> -- This DDL should fail but worked:
> ALTER TABLE t ADD PARTITION(p='aaa'); {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
