[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[ https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=861752&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-861752 ] ASF GitHub Bot logged work on HIVE-27234: - Author: ASF GitHub Bot Created on: 12/May/23 12:32 Start Date: 12/May/23 12:32 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4216: URL: https://github.com/apache/hive/pull/4216#issuecomment-1545671693 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4216) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [2 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [27 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4216&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4216&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 861752) Time Spent: 10.5h (was: 10h 20m) > Iceberg: CREATE BRANCH SQL implementation > -- > > Key: HIVE-27234 > URL: https://issues.apache.org/jira/browse/HIVE-27234 > Project: Hive > Issue Type: Sub-task > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > Time Spent: 10.5h > Remaining Estimate: 0h > > Maybe we can follow spark sql about branch ddl implementation > [https://github.com/apache/iceberg/pull/6617] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[ https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=861660&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-861660 ] ASF GitHub Bot logged work on HIVE-27234: - Author: ASF GitHub Bot Created on: 12/May/23 03:14 Start Date: 12/May/23 03:14 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4216: URL: https://github.com/apache/hive/pull/4216#issuecomment-1545049969 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4216) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [2 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [18 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4216&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4216&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 861660) Time Spent: 10h 20m (was: 10h 10m) > Iceberg: CREATE BRANCH SQL implementation > -- > > Key: HIVE-27234 > URL: https://issues.apache.org/jira/browse/HIVE-27234 > Project: Hive > Issue Type: Sub-task > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > Time Spent: 10h 20m > Remaining Estimate: 0h > > Maybe we can follow spark sql about branch ddl implementation > [https://github.com/apache/iceberg/pull/6617] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=861649&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-861649
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 12/May/23 00:15
Start Date: 12/May/23 00:15
Worklog Time Spent: 10m
Work Description: zhangbutao opened a new pull request, #4216:
URL: https://github.com/apache/hive/pull/4216
### What changes were proposed in this pull request?
This PR refers to spark-sql about iceberg branch ddl implementation
https://github.com/apache/iceberg/pull/6617
If someone has different opinions about the sql syntax of branch, we can
discuss here.
### Why are the changes needed?
Personally, branch is more useful than snapshot in iceberg, and it is more
friendly to users. We can use branch do lots of meaningfull work.
### Does this PR introduce _any_ user-facing change?
Added a new sql syntax and hive users can create iceberg branch using the
sql.
```
ALTER TABLE tableName
{CREATE BRANCH branchName [FOR SYSTEM_VERSION AS OF {snapshotId} | FOR
SYSTEM_TIME AS OF {timestamp}]
[RETAIN interval {DAYS | HOURS | MINUTES}]
[WITH SNAPSHOT RETENTION {[num_snapshots SNAPSHOTS] [interval {DAYS | HOURS
| MINUTES}]}]}]
```
### How was this patch tested?
UT
Issue Time Tracking
---
Worklog Id: (was: 861649)
Time Spent: 10h 10m (was: 10h)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10h 10m
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[ https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=861648&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-861648 ] ASF GitHub Bot logged work on HIVE-27234: - Author: ASF GitHub Bot Created on: 12/May/23 00:14 Start Date: 12/May/23 00:14 Worklog Time Spent: 10m Work Description: zhangbutao closed pull request #4216: HIVE-27234: Iceberg: CREATE BRANCH SQL implementation URL: https://github.com/apache/hive/pull/4216 Issue Time Tracking --- Worklog Id: (was: 861648) Time Spent: 10h (was: 9h 50m) > Iceberg: CREATE BRANCH SQL implementation > -- > > Key: HIVE-27234 > URL: https://issues.apache.org/jira/browse/HIVE-27234 > Project: Hive > Issue Type: Sub-task > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > Time Spent: 10h > Remaining Estimate: 0h > > Maybe we can follow spark sql about branch ddl implementation > [https://github.com/apache/iceberg/pull/6617] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[ https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=861527&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-861527 ] ASF GitHub Bot logged work on HIVE-27234: - Author: ASF GitHub Bot Created on: 11/May/23 13:22 Start Date: 11/May/23 13:22 Worklog Time Spent: 10m Work Description: zhangbutao commented on PR #4216: URL: https://github.com/apache/hive/pull/4216#issuecomment-1544002029 Git rebase to fix code conflicts. Issue Time Tracking --- Worklog Id: (was: 861527) Time Spent: 9h 50m (was: 9h 40m) > Iceberg: CREATE BRANCH SQL implementation > -- > > Key: HIVE-27234 > URL: https://issues.apache.org/jira/browse/HIVE-27234 > Project: Hive > Issue Type: Sub-task > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > Time Spent: 9h 50m > Remaining Estimate: 0h > > Maybe we can follow spark sql about branch ddl implementation > [https://github.com/apache/iceberg/pull/6617] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[ https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=861242&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-861242 ] ASF GitHub Bot logged work on HIVE-27234: - Author: ASF GitHub Bot Created on: 09/May/23 17:32 Start Date: 09/May/23 17:32 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4216: URL: https://github.com/apache/hive/pull/4216#issuecomment-1540586902 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4216) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [2 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [11 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4216&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4216&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 861242) Time Spent: 9h 40m (was: 9.5h) > Iceberg: CREATE BRANCH SQL implementation > -- > > Key: HIVE-27234 > URL: https://issues.apache.org/jira/browse/HIVE-27234 > Project: Hive > Issue Type: Sub-task > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > Time Spent: 9h 40m > Remaining Estimate: 0h > > Maybe we can follow spark sql about branch ddl implementation > [https://github.com/apache/iceberg/pull/6617] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[ https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=861221&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-861221 ] ASF GitHub Bot logged work on HIVE-27234: - Author: ASF GitHub Bot Created on: 09/May/23 15:14 Start Date: 09/May/23 15:14 Worklog Time Spent: 10m Work Description: zhangbutao commented on code in PR #4216: URL: https://github.com/apache/hive/pull/4216#discussion_r1188764400 ## ql/src/java/org/apache/hadoop/hive/ql/ddl/DDLUtils.java: ## @@ -217,4 +218,11 @@ public static void setColumnsAndStorePartitionTransformSpecOfTable( partCols.ifPresent(tbl::setPartCols); } } + + public static void validateTableIsIceberg(org.apache.hadoop.hive.ql.metadata.Table table) Review Comment: yep, it duplicates `HiveTableOperations.validateTableIsIceberg` as DDLTask in`ql ` package can not import iceberg class. I changed the thrown exception to` Not an iceberg table: %s (type=%s)` , this means the same thing as `HiveTableOperations.validateTableIsIceberg` Issue Time Tracking --- Worklog Id: (was: 861221) Time Spent: 9.5h (was: 9h 20m) > Iceberg: CREATE BRANCH SQL implementation > -- > > Key: HIVE-27234 > URL: https://issues.apache.org/jira/browse/HIVE-27234 > Project: Hive > Issue Type: Sub-task > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > Time Spent: 9.5h > Remaining Estimate: 0h > > Maybe we can follow spark sql about branch ddl implementation > [https://github.com/apache/iceberg/pull/6617] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=861174&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-861174
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 09/May/23 10:23
Start Date: 09/May/23 10:23
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1188448686
##
ql/src/java/org/apache/hadoop/hive/ql/ddl/DDLUtils.java:
##
@@ -217,4 +218,11 @@ public static void
setColumnsAndStorePartitionTransformSpecOfTable(
partCols.ifPresent(tbl::setPartCols);
}
}
+
+ public static void
validateTableIsIceberg(org.apache.hadoop.hive.ql.metadata.Table table)
Review Comment:
doesn't it duplicate `HiveTableOperations.validateTableIsIceberg`, could we
keep just one version of it
static void validateTableIsIceberg(Table table, String fullName) {
String tableType = table.getParameters().get(TABLE_TYPE_PROP);
NoSuchIcebergTableException.check(tableType != null &&
tableType.equalsIgnoreCase(ICEBERG_TABLE_TYPE_VALUE),
"Not an iceberg table: %s (type=%s)", fullName, tableType);
}
Issue Time Tracking
---
Worklog Id: (was: 861174)
Time Spent: 9h 20m (was: 9h 10m)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 9h 20m
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=861173&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-861173
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 09/May/23 10:17
Start Date: 09/May/23 10:17
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1188441506
##
ql/src/java/org/apache/hadoop/hive/ql/parse/AlterTableCreateBranchSpec.java:
##
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.parse;
+
+import com.google.common.base.MoreObjects;
+
+public class AlterTableCreateBranchSpec {
Review Comment:
👍
Issue Time Tracking
---
Worklog Id: (was: 861173)
Time Spent: 9h 10m (was: 9h)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 9h 10m
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[ https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=861166&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-861166 ] ASF GitHub Bot logged work on HIVE-27234: - Author: ASF GitHub Bot Created on: 09/May/23 09:45 Start Date: 09/May/23 09:45 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4216: URL: https://github.com/apache/hive/pull/4216#issuecomment-1539804069 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4216) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [2 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [11 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4216&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4216&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 861166) Time Spent: 9h (was: 8h 50m) > Iceberg: CREATE BRANCH SQL implementation > -- > > Key: HIVE-27234 > URL: https://issues.apache.org/jira/browse/HIVE-27234 > Project: Hive > Issue Type: Sub-task > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > Time Spent: 9h > Remaining Estimate: 0h > > Maybe we can follow spark sql about branch ddl implementation > [https://github.com/apache/iceberg/pull/6617] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=861140&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-861140
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 09/May/23 08:08
Start Date: 09/May/23 08:08
Worklog Time Spent: 10m
Work Description: zhangbutao commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1188283923
##
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java:
##
@@ -6709,6 +6710,15 @@ public void alterTableExecuteOperation(Table table,
AlterTableExecuteSpec execut
}
}
+ public void alterTableCreateBranchOperation(Table table,
AlterTableCreateBranchSpec createBranchSpec) throws HiveException {
Review Comment:
I have rename it to 'alterTableBranchOperation' which can be used to
create/drop/replace branch. Is it ok?
Issue Time Tracking
---
Worklog Id: (was: 861140)
Time Spent: 8h 50m (was: 8h 40m)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 8h 50m
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=861138&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-861138
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 09/May/23 08:05
Start Date: 09/May/23 08:05
Worklog Time Spent: 10m
Work Description: zhangbutao commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1188281049
##
ql/src/java/org/apache/hadoop/hive/ql/parse/AlterTableCreateBranchSpec.java:
##
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.parse;
+
+import com.google.common.base.MoreObjects;
+
+public class AlterTableCreateBranchSpec {
Review Comment:
Finally, i decide not to reuse codes about `alter table execute` syntax , as
i think it is more important to make code clear and readable.
I used `AlterTableBranchSpec ` which looks like the pattern
`AlterTableExecuteSpec` , and then define inner `CreateBranchSpec` which type
is `CREATE_BRANCH`. We can continue to add other brancn operation e.g
`DROP_BRANCH`, `REPLACE_BRANCH`.
Please take a look again to see if it is make more sense. Thanks.
Issue Time Tracking
---
Worklog Id: (was: 861138)
Time Spent: 8h 40m (was: 8.5h)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 8h 40m
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=861132&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-861132
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 09/May/23 07:53
Start Date: 09/May/23 07:53
Worklog Time Spent: 10m
Work Description: zhangbutao commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1188267965
##
ql/src/java/org/apache/hadoop/hive/ql/ddl/table/branch/create/AlterTableCreateBranchAnalyzer.java:
##
@@ -0,0 +1,93 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.ddl.table.branch.create;
+
+import java.util.Locale;
+import java.util.Map;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.hadoop.hive.common.TableName;
+import org.apache.hadoop.hive.metastore.HiveMetaHook;
+import org.apache.hadoop.hive.ql.QueryState;
+import org.apache.hadoop.hive.ql.ddl.DDLSemanticAnalyzerFactory;
+import org.apache.hadoop.hive.ql.ddl.DDLWork;
+import org.apache.hadoop.hive.ql.ddl.table.AbstractAlterTableAnalyzer;
+import org.apache.hadoop.hive.ql.ddl.table.AlterTableType;
+import org.apache.hadoop.hive.ql.exec.TaskFactory;
+import org.apache.hadoop.hive.ql.hooks.ReadEntity;
+import org.apache.hadoop.hive.ql.metadata.Table;
+import org.apache.hadoop.hive.ql.parse.ASTNode;
+import org.apache.hadoop.hive.ql.parse.AlterTableCreateBranchSpec;
+import org.apache.hadoop.hive.ql.parse.HiveParser;
+import org.apache.hadoop.hive.ql.parse.SemanticException;
+
+@DDLSemanticAnalyzerFactory.DDLType(types =
HiveParser.TOK_ALTERTABLE_CREATE_BRANCH)
+public class AlterTableCreateBranchAnalyzer extends AbstractAlterTableAnalyzer
{
+
+ public AlterTableCreateBranchAnalyzer(QueryState queryState) throws
SemanticException {
+super(queryState);
+ }
+
+ @Override
+ protected void analyzeCommand(TableName tableName, Map
partitionSpec, ASTNode command)
+ throws SemanticException {
+Table table = getTable(tableName);
+validateAlterTableType(table, AlterTableType.CREATEBRANCH, false);
+if
(!HiveMetaHook.ICEBERG.equalsIgnoreCase(table.getParameters().get(HiveMetaHook.TABLE_TYPE)))
{
Review Comment:
Thanks for your tip. Added a helper method:
`DDLUtils::validateTableIsIceberg`
Issue Time Tracking
---
Worklog Id: (was: 861132)
Time Spent: 8.5h (was: 8h 20m)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 8.5h
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=861050&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-861050
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 08/May/23 16:19
Start Date: 08/May/23 16:19
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1187634091
##
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java:
##
@@ -6709,6 +6710,15 @@ public void alterTableExecuteOperation(Table table,
AlterTableExecuteSpec execut
}
}
+ public void alterTableCreateBranchOperation(Table table,
AlterTableCreateBranchSpec createBranchSpec) throws HiveException {
Review Comment:
maybe simply 'alterTableCreateBranch' ?
Issue Time Tracking
---
Worklog Id: (was: 861050)
Time Spent: 8h 20m (was: 8h 10m)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 8h 20m
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=861049&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-861049
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 08/May/23 16:14
Start Date: 08/May/23 16:14
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1187629739
##
ql/src/java/org/apache/hadoop/hive/ql/ddl/table/branch/create/AlterTableCreateBranchAnalyzer.java:
##
@@ -0,0 +1,93 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.ddl.table.branch.create;
+
+import java.util.Locale;
+import java.util.Map;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.hadoop.hive.common.TableName;
+import org.apache.hadoop.hive.metastore.HiveMetaHook;
+import org.apache.hadoop.hive.ql.QueryState;
+import org.apache.hadoop.hive.ql.ddl.DDLSemanticAnalyzerFactory;
+import org.apache.hadoop.hive.ql.ddl.DDLWork;
+import org.apache.hadoop.hive.ql.ddl.table.AbstractAlterTableAnalyzer;
+import org.apache.hadoop.hive.ql.ddl.table.AlterTableType;
+import org.apache.hadoop.hive.ql.exec.TaskFactory;
+import org.apache.hadoop.hive.ql.hooks.ReadEntity;
+import org.apache.hadoop.hive.ql.metadata.Table;
+import org.apache.hadoop.hive.ql.parse.ASTNode;
+import org.apache.hadoop.hive.ql.parse.AlterTableCreateBranchSpec;
+import org.apache.hadoop.hive.ql.parse.HiveParser;
+import org.apache.hadoop.hive.ql.parse.SemanticException;
+
+@DDLSemanticAnalyzerFactory.DDLType(types =
HiveParser.TOK_ALTERTABLE_CREATE_BRANCH)
+public class AlterTableCreateBranchAnalyzer extends AbstractAlterTableAnalyzer
{
+
+ public AlterTableCreateBranchAnalyzer(QueryState queryState) throws
SemanticException {
+super(queryState);
+ }
+
+ @Override
+ protected void analyzeCommand(TableName tableName, Map
partitionSpec, ASTNode command)
+ throws SemanticException {
+Table table = getTable(tableName);
+validateAlterTableType(table, AlterTableType.CREATEBRANCH, false);
+if
(!HiveMetaHook.ICEBERG.equalsIgnoreCase(table.getParameters().get(HiveMetaHook.TABLE_TYPE)))
{
Review Comment:
would be great to have a short helper method like `isIceberg()` instead of
this long construct (used in multiple places).
Found similar thing in `HiveTableOperations#validateTableIsIceberg(Table
table, String fullName)`
Issue Time Tracking
---
Worklog Id: (was: 861049)
Time Spent: 8h 10m (was: 8h)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 8h 10m
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=861048&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-861048
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 08/May/23 16:14
Start Date: 08/May/23 16:14
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1187629739
##
ql/src/java/org/apache/hadoop/hive/ql/ddl/table/branch/create/AlterTableCreateBranchAnalyzer.java:
##
@@ -0,0 +1,93 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.ddl.table.branch.create;
+
+import java.util.Locale;
+import java.util.Map;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.hadoop.hive.common.TableName;
+import org.apache.hadoop.hive.metastore.HiveMetaHook;
+import org.apache.hadoop.hive.ql.QueryState;
+import org.apache.hadoop.hive.ql.ddl.DDLSemanticAnalyzerFactory;
+import org.apache.hadoop.hive.ql.ddl.DDLWork;
+import org.apache.hadoop.hive.ql.ddl.table.AbstractAlterTableAnalyzer;
+import org.apache.hadoop.hive.ql.ddl.table.AlterTableType;
+import org.apache.hadoop.hive.ql.exec.TaskFactory;
+import org.apache.hadoop.hive.ql.hooks.ReadEntity;
+import org.apache.hadoop.hive.ql.metadata.Table;
+import org.apache.hadoop.hive.ql.parse.ASTNode;
+import org.apache.hadoop.hive.ql.parse.AlterTableCreateBranchSpec;
+import org.apache.hadoop.hive.ql.parse.HiveParser;
+import org.apache.hadoop.hive.ql.parse.SemanticException;
+
+@DDLSemanticAnalyzerFactory.DDLType(types =
HiveParser.TOK_ALTERTABLE_CREATE_BRANCH)
+public class AlterTableCreateBranchAnalyzer extends AbstractAlterTableAnalyzer
{
+
+ public AlterTableCreateBranchAnalyzer(QueryState queryState) throws
SemanticException {
+super(queryState);
+ }
+
+ @Override
+ protected void analyzeCommand(TableName tableName, Map
partitionSpec, ASTNode command)
+ throws SemanticException {
+Table table = getTable(tableName);
+validateAlterTableType(table, AlterTableType.CREATEBRANCH, false);
+if
(!HiveMetaHook.ICEBERG.equalsIgnoreCase(table.getParameters().get(HiveMetaHook.TABLE_TYPE)))
{
Review Comment:
would be great to have a short helper method like` isIceberg()` instead of
this long construct (used in multiple places).
Found similar thing in `HiveTableOperations#validateTableIsIceberg(Table
table, String fullName)`
Issue Time Tracking
---
Worklog Id: (was: 861048)
Time Spent: 8h (was: 7h 50m)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 8h
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=861047&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-861047
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 08/May/23 16:11
Start Date: 08/May/23 16:11
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1187626822
##
ql/src/java/org/apache/hadoop/hive/ql/parse/AlterTableCreateBranchSpec.java:
##
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.parse;
+
+import com.google.common.base.MoreObjects;
+
+public class AlterTableCreateBranchSpec {
Review Comment:
I think so, ExecuteOperationType was extended with those 3 operations:
ROLLBACK,
EXPIRE_SNAPSHOT,
SET_CURRENT_SNAPSHOT
note, if you think that create/drop branch doesn't semantically fit this
design or complicates it, feel free to create a a new abstraction
Issue Time Tracking
---
Worklog Id: (was: 861047)
Time Spent: 7h 50m (was: 7h 40m)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 7h 50m
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=861045&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-861045
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 08/May/23 16:05
Start Date: 08/May/23 16:05
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1187621453
##
parser/src/java/org/apache/hadoop/hive/ql/parse/AlterClauseParser.g:
##
@@ -477,6 +478,34 @@ alterStatementSuffixExecute
-> ^(TOK_ALTERTABLE_EXECUTE KW_SET_CURRENT_SNAPSHOT $snapshotParam)
;
+alterStatementSuffixCreateBranch
+@init { gParent.pushMsg("alter table create branch", state); }
+@after { gParent.popMsg(state); }
+: KW_CREATE KW_BRANCH branchName=identifier snapshotIdOfBranch?
branchRetain? retentionOfSnapshots?
+-> ^(TOK_ALTERTABLE_CREATE_BRANCH $branchName snapshotIdOfBranch?
branchRetain? retentionOfSnapshots?)
+;
+
+snapshotIdOfBranch
+@init { gParent.pushMsg("alter table create branch as of version", state); }
Review Comment:
👍
Issue Time Tracking
---
Worklog Id: (was: 861045)
Time Spent: 7h 40m (was: 7.5h)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 7h 40m
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=861044&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-861044
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 08/May/23 16:05
Start Date: 08/May/23 16:05
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1187620978
##
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java:
##
@@ -676,6 +678,35 @@ public void
executeOperation(org.apache.hadoop.hive.ql.metadata.Table hmsTable,
}
}
+ @Override
+ public void createBranchOperation(org.apache.hadoop.hive.ql.metadata.Table
hmsTable,
+ AlterTableCreateBranchSpec createBranchSpec) {
+TableDesc tableDesc = Utilities.getTableDesc(hmsTable);
+Table icebergTable = IcebergTableUtil.getTable(conf,
tableDesc.getProperties());
+
+String branchName = createBranchSpec.getBranchName();
+Optional.ofNullable(icebergTable.currentSnapshot()).orElseThrow(() -> new
UnsupportedOperationException(
+String.format("Cannot create branch %s on iceberg table %s.%s which
has no snapshot",
+branchName, hmsTable.getDbName(), hmsTable.getTableName(;
+Long snapshotId = Optional.ofNullable(createBranchSpec.getSnapshotId())
+.orElse(icebergTable.currentSnapshot().snapshotId());
+LOG.info("Creating branch {} on iceberg table {}.{}", branchName,
hmsTable.getDbName(),
+hmsTable.getTableName());
+ManageSnapshots manageSnapshots = icebergTable.manageSnapshots();
+manageSnapshots.createBranch(branchName, snapshotId);
Review Comment:
👍
Issue Time Tracking
---
Worklog Id: (was: 861044)
Time Spent: 7.5h (was: 7h 20m)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 7.5h
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[ https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=860593&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-860593 ] ASF GitHub Bot logged work on HIVE-27234: - Author: ASF GitHub Bot Created on: 04/May/23 17:03 Start Date: 04/May/23 17:03 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4216: URL: https://github.com/apache/hive/pull/4216#issuecomment-1535099360 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4216) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [1 Bug](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [2 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4216&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4216&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 860593) Time Spent: 7h 20m (was: 7h 10m) > Iceberg: CREATE BRANCH SQL implementation > -- > > Key: HIVE-27234 > URL: https://issues.apache.org/jira/browse/HIVE-27234 > Project: Hive > Issue Type: Sub-task > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > Time Spent: 7h 20m > Remaining Estimate: 0h > > Maybe we can follow spark sql about branch ddl implementation > [https://github.com/apache/iceberg/pull/6617] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[ https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=860538&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-860538 ] ASF GitHub Bot logged work on HIVE-27234: - Author: ASF GitHub Bot Created on: 04/May/23 10:37 Start Date: 04/May/23 10:37 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4216: URL: https://github.com/apache/hive/pull/4216#issuecomment-1534519816 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4216) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [1 Bug](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [3 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4216&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4216&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 860538) Time Spent: 7h 10m (was: 7h) > Iceberg: CREATE BRANCH SQL implementation > -- > > Key: HIVE-27234 > URL: https://issues.apache.org/jira/browse/HIVE-27234 > Project: Hive > Issue Type: Sub-task > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > Time Spent: 7h 10m > Remaining Estimate: 0h > > Maybe we can follow spark sql about branch ddl implementation > [https://github.com/apache/iceberg/pull/6617] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[ https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=860466&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-860466 ] ASF GitHub Bot logged work on HIVE-27234: - Author: ASF GitHub Bot Created on: 04/May/23 05:56 Start Date: 04/May/23 05:56 Worklog Time Spent: 10m Work Description: zhangbutao closed pull request #4216: HIVE-27234: Iceberg: CREATE BRANCH SQL implementation URL: https://github.com/apache/hive/pull/4216 Issue Time Tracking --- Worklog Id: (was: 860466) Time Spent: 6h 50m (was: 6h 40m) > Iceberg: CREATE BRANCH SQL implementation > -- > > Key: HIVE-27234 > URL: https://issues.apache.org/jira/browse/HIVE-27234 > Project: Hive > Issue Type: Sub-task > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > Time Spent: 6h 50m > Remaining Estimate: 0h > > Maybe we can follow spark sql about branch ddl implementation > [https://github.com/apache/iceberg/pull/6617] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=860467&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-860467
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 04/May/23 05:56
Start Date: 04/May/23 05:56
Worklog Time Spent: 10m
Work Description: zhangbutao opened a new pull request, #4216:
URL: https://github.com/apache/hive/pull/4216
### What changes were proposed in this pull request?
This PR refers to spark-sql about iceberg branch ddl implementation
https://github.com/apache/iceberg/pull/6617
If someone has different opinions about the sql syntax of branch, we can
discuss here.
### Why are the changes needed?
Personally, branch is more useful than snapshot in iceberg, and it is more
friendly to users. We can use branch do lots of meaningfull work.
### Does this PR introduce _any_ user-facing change?
Added a new sql syntax and hive users can create iceberg branch using the
sql.
```
ALTER TABLE tableName
{CREATE BRANCH branchName [AS OF VERSION {snapshotId}]
[RETAIN interval {DAYS | HOURS | MINUTES}]
[WITH SNAPSHOT RETENTION {[num_snapshots SNAPSHOTS] [interval {DAYS | HOURS
| MINUTES}]}]}]
```
### How was this patch tested?
UT
Issue Time Tracking
---
Worklog Id: (was: 860467)
Time Spent: 7h (was: 6h 50m)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 7h
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=860463&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-860463
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 04/May/23 04:50
Start Date: 04/May/23 04:50
Worklog Time Spent: 10m
Work Description: zhangbutao commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1184537248
##
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java:
##
@@ -676,6 +678,35 @@ public void
executeOperation(org.apache.hadoop.hive.ql.metadata.Table hmsTable,
}
}
+ @Override
+ public void createBranchOperation(org.apache.hadoop.hive.ql.metadata.Table
hmsTable,
+ AlterTableCreateBranchSpec createBranchSpec) {
+TableDesc tableDesc = Utilities.getTableDesc(hmsTable);
+Table icebergTable = IcebergTableUtil.getTable(conf,
tableDesc.getProperties());
+
+String branchName = createBranchSpec.getBranchName();
+Optional.ofNullable(icebergTable.currentSnapshot()).orElseThrow(() -> new
UnsupportedOperationException(
+String.format("Cannot create branch %s on iceberg table %s.%s which
has no snapshot",
+branchName, hmsTable.getDbName(), hmsTable.getTableName(;
+Long snapshotId = Optional.ofNullable(createBranchSpec.getSnapshotId())
+.orElse(icebergTable.currentSnapshot().snapshotId());
+LOG.info("Creating branch {} on iceberg table {}.{}", branchName,
hmsTable.getDbName(),
Review Comment:
Added `snapshotId `. I think INFO is enough as this DDL is rarely executed.
Issue Time Tracking
---
Worklog Id: (was: 860463)
Time Spent: 6h 40m (was: 6.5h)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 6h 40m
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=860462&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-860462
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 04/May/23 04:47
Start Date: 04/May/23 04:47
Worklog Time Spent: 10m
Work Description: zhangbutao commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1184536300
##
ql/src/java/org/apache/hadoop/hive/ql/ddl/table/AlterTableType.java:
##
@@ -40,6 +40,7 @@ public enum AlterTableType {
ALTERPARTITION("alter partition"), // Note: this is never used in
AlterTableDesc.
SETPARTITIONSPEC("set partition spec"),
EXECUTE("execute"),
+ CREATEBRANCH("create branch"),
Review Comment:
done
Issue Time Tracking
---
Worklog Id: (was: 860462)
Time Spent: 6.5h (was: 6h 20m)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 6.5h
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=860461&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-860461
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 04/May/23 04:46
Start Date: 04/May/23 04:46
Worklog Time Spent: 10m
Work Description: zhangbutao commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1184536183
##
parser/src/java/org/apache/hadoop/hive/ql/parse/AlterClauseParser.g:
##
@@ -477,6 +478,34 @@ alterStatementSuffixExecute
-> ^(TOK_ALTERTABLE_EXECUTE KW_SET_CURRENT_SNAPSHOT $snapshotParam)
;
+alterStatementSuffixCreateBranch
+@init { gParent.pushMsg("alter table create branch", state); }
+@after { gParent.popMsg(state); }
+: KW_CREATE KW_BRANCH branchName=identifier snapshotIdOfBranch?
branchRetain? retentionOfSnapshots?
+-> ^(TOK_ALTERTABLE_CREATE_BRANCH $branchName snapshotIdOfBranch?
branchRetain? retentionOfSnapshots?)
+;
+
+snapshotIdOfBranch
+@init { gParent.pushMsg("alter table create branch as of version", state); }
Review Comment:
done
##
parser/src/java/org/apache/hadoop/hive/ql/parse/AlterClauseParser.g:
##
@@ -477,6 +478,34 @@ alterStatementSuffixExecute
-> ^(TOK_ALTERTABLE_EXECUTE KW_SET_CURRENT_SNAPSHOT $snapshotParam)
;
+alterStatementSuffixCreateBranch
+@init { gParent.pushMsg("alter table create branch", state); }
+@after { gParent.popMsg(state); }
+: KW_CREATE KW_BRANCH branchName=identifier snapshotIdOfBranch?
branchRetain? retentionOfSnapshots?
+-> ^(TOK_ALTERTABLE_CREATE_BRANCH $branchName snapshotIdOfBranch?
branchRetain? retentionOfSnapshots?)
+;
+
+snapshotIdOfBranch
+@init { gParent.pushMsg("alter table create branch as of version", state); }
+@after { gParent.popMsg(state); }
+: KW_AS KW_OF KW_VERSION snapshotId=Number
+-> ^(TOK_AS_OF_VERSION_BRANCH $snapshotId)
Review Comment:
done
Issue Time Tracking
---
Worklog Id: (was: 860461)
Time Spent: 6h 20m (was: 6h 10m)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 6h 20m
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=860287&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-860287
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 03/May/23 10:51
Start Date: 03/May/23 10:51
Worklog Time Spent: 10m
Work Description: zhangbutao commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1183529581
##
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java:
##
@@ -676,6 +678,35 @@ public void
executeOperation(org.apache.hadoop.hive.ql.metadata.Table hmsTable,
}
}
+ @Override
+ public void createBranchOperation(org.apache.hadoop.hive.ql.metadata.Table
hmsTable,
+ AlterTableCreateBranchSpec createBranchSpec) {
+TableDesc tableDesc = Utilities.getTableDesc(hmsTable);
+Table icebergTable = IcebergTableUtil.getTable(conf,
tableDesc.getProperties());
+
+String branchName = createBranchSpec.getBranchName();
+Optional.ofNullable(icebergTable.currentSnapshot()).orElseThrow(() -> new
UnsupportedOperationException(
+String.format("Cannot create branch %s on iceberg table %s.%s which
has no snapshot",
+branchName, hmsTable.getDbName(), hmsTable.getTableName(;
+Long snapshotId = Optional.ofNullable(createBranchSpec.getSnapshotId())
+.orElse(icebergTable.currentSnapshot().snapshotId());
+LOG.info("Creating branch {} on iceberg table {}.{}", branchName,
hmsTable.getDbName(),
+hmsTable.getTableName());
+ManageSnapshots manageSnapshots = icebergTable.manageSnapshots();
+manageSnapshots.createBranch(branchName, snapshotId);
Review Comment:
If user omits ` snapshotId ` when creating branch, then current snapshotId
will be used.
if user passes a non-existent `snapshotId` when creating branch, exception
`unknown snapshot` will be thrown from iceberg lib.
Issue Time Tracking
---
Worklog Id: (was: 860287)
Time Spent: 6h 10m (was: 6h)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 6h 10m
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=860268&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-860268
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 03/May/23 10:21
Start Date: 03/May/23 10:21
Worklog Time Spent: 10m
Work Description: zhangbutao commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1183501618
##
ql/src/java/org/apache/hadoop/hive/ql/parse/AlterTableCreateBranchSpec.java:
##
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.parse;
+
+import com.google.common.base.MoreObjects;
+
+public class AlterTableCreateBranchSpec {
Review Comment:
That's what I thought before. Also I want to reuse method:
`HiveIcebergStorageHandler::executeOperation` :
https://github.com/apache/hive/blob/e413b445e95e7c8beb4e43f0f6b08a98934e1c2c/iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java#L645
But it seems to be that these codes are for syntax `alter table execute`.
Do you think we can reuse them for `alter table create branch
` or `alter table drop branch`?
Issue Time Tracking
---
Worklog Id: (was: 860268)
Time Spent: 6h (was: 5h 50m)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 6h
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=860264&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-860264
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 03/May/23 10:11
Start Date: 03/May/23 10:11
Worklog Time Spent: 10m
Work Description: zhangbutao commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1183492278
##
ql/src/java/org/apache/hadoop/hive/ql/ddl/table/branch/create/AlterTableCreateBranchAnalyzer.java:
##
@@ -0,0 +1,93 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.ddl.table.branch.create;
+
+import java.util.Locale;
+import java.util.Map;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.hadoop.hive.common.TableName;
+import org.apache.hadoop.hive.metastore.HiveMetaHook;
+import org.apache.hadoop.hive.ql.QueryState;
+import org.apache.hadoop.hive.ql.ddl.DDLSemanticAnalyzerFactory;
+import org.apache.hadoop.hive.ql.ddl.DDLWork;
+import org.apache.hadoop.hive.ql.ddl.table.AbstractAlterTableAnalyzer;
+import org.apache.hadoop.hive.ql.ddl.table.AlterTableType;
+import org.apache.hadoop.hive.ql.exec.TaskFactory;
+import org.apache.hadoop.hive.ql.hooks.ReadEntity;
+import org.apache.hadoop.hive.ql.metadata.Table;
+import org.apache.hadoop.hive.ql.parse.ASTNode;
+import org.apache.hadoop.hive.ql.parse.AlterTableCreateBranchSpec;
+import org.apache.hadoop.hive.ql.parse.HiveParser;
+import org.apache.hadoop.hive.ql.parse.SemanticException;
+
+@DDLSemanticAnalyzerFactory.DDLType(types =
HiveParser.TOK_ALTERTABLE_CREATE_BRANCH)
+public class AlterTableCreateBranchAnalyzer extends AbstractAlterTableAnalyzer
{
+
+ public AlterTableCreateBranchAnalyzer(QueryState queryState) throws
SemanticException {
+super(queryState);
+ }
+
+ @Override
+ protected void analyzeCommand(TableName tableName, Map
partitionSpec, ASTNode command)
+ throws SemanticException {
+Table table = getTable(tableName);
+validateAlterTableType(table, AlterTableType.CREATEBRANCH, false);
+if
(!HiveMetaHook.ICEBERG.equalsIgnoreCase(table.getParameters().get(HiveMetaHook.TABLE_TYPE)))
{
Review Comment:
Sorry, can you elaborate on where this logic shoud move into? This is just a
simple iceberg check statement.
Issue Time Tracking
---
Worklog Id: (was: 860264)
Time Spent: 5h 50m (was: 5h 40m)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 5h 50m
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=860260&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-860260
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 03/May/23 10:07
Start Date: 03/May/23 10:07
Worklog Time Spent: 10m
Work Description: zhangbutao commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1183488456
##
parser/src/java/org/apache/hadoop/hive/ql/parse/AlterClauseParser.g:
##
@@ -477,6 +478,34 @@ alterStatementSuffixExecute
-> ^(TOK_ALTERTABLE_EXECUTE KW_SET_CURRENT_SNAPSHOT $snapshotParam)
;
+alterStatementSuffixCreateBranch
+@init { gParent.pushMsg("alter table create branch", state); }
+@after { gParent.popMsg(state); }
+: KW_CREATE KW_BRANCH branchName=identifier snapshotIdOfBranch?
branchRetain? retentionOfSnapshots?
+-> ^(TOK_ALTERTABLE_CREATE_BRANCH $branchName snapshotIdOfBranch?
branchRetain? retentionOfSnapshots?)
+;
+
+snapshotIdOfBranch
+@init { gParent.pushMsg("alter table create branch as of version", state); }
+@after { gParent.popMsg(state); }
+: KW_AS KW_OF KW_VERSION snapshotId=Number
+-> ^(TOK_AS_OF_VERSION_BRANCH $snapshotId)
Review Comment:
sure, as mentioned above, we can reuse `TOK_AS_OF_VERSION` if we use `FOR
SYSTEM_VERSION AS OF` instead of `AS OF VERSION`.
Issue Time Tracking
---
Worklog Id: (was: 860260)
Time Spent: 5.5h (was: 5h 20m)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 5.5h
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=860261&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-860261
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 03/May/23 10:07
Start Date: 03/May/23 10:07
Worklog Time Spent: 10m
Work Description: zhangbutao commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1183488456
##
parser/src/java/org/apache/hadoop/hive/ql/parse/AlterClauseParser.g:
##
@@ -477,6 +478,34 @@ alterStatementSuffixExecute
-> ^(TOK_ALTERTABLE_EXECUTE KW_SET_CURRENT_SNAPSHOT $snapshotParam)
;
+alterStatementSuffixCreateBranch
+@init { gParent.pushMsg("alter table create branch", state); }
+@after { gParent.popMsg(state); }
+: KW_CREATE KW_BRANCH branchName=identifier snapshotIdOfBranch?
branchRetain? retentionOfSnapshots?
+-> ^(TOK_ALTERTABLE_CREATE_BRANCH $branchName snapshotIdOfBranch?
branchRetain? retentionOfSnapshots?)
+;
+
+snapshotIdOfBranch
+@init { gParent.pushMsg("alter table create branch as of version", state); }
+@after { gParent.popMsg(state); }
+: KW_AS KW_OF KW_VERSION snapshotId=Number
+-> ^(TOK_AS_OF_VERSION_BRANCH $snapshotId)
Review Comment:
As mentioned above, we can reuse `TOK_AS_OF_VERSION` if we use `FOR
SYSTEM_VERSION AS OF` instead of `AS OF VERSION`.
Issue Time Tracking
---
Worklog Id: (was: 860261)
Time Spent: 5h 40m (was: 5.5h)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 5h 40m
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=860258&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-860258
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 03/May/23 10:04
Start Date: 03/May/23 10:04
Worklog Time Spent: 10m
Work Description: zhangbutao commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1183485755
##
parser/src/java/org/apache/hadoop/hive/ql/parse/AlterClauseParser.g:
##
@@ -477,6 +478,34 @@ alterStatementSuffixExecute
-> ^(TOK_ALTERTABLE_EXECUTE KW_SET_CURRENT_SNAPSHOT $snapshotParam)
;
+alterStatementSuffixCreateBranch
+@init { gParent.pushMsg("alter table create branch", state); }
+@after { gParent.popMsg(state); }
+: KW_CREATE KW_BRANCH branchName=identifier snapshotIdOfBranch?
branchRetain? retentionOfSnapshots?
+-> ^(TOK_ALTERTABLE_CREATE_BRANCH $branchName snapshotIdOfBranch?
branchRetain? retentionOfSnapshots?)
+;
+
+snapshotIdOfBranch
+@init { gParent.pushMsg("alter table create branch as of version", state); }
Review Comment:
My original thought was to be consistent with spark-iceberg syntax. But it
seems more reasonable to be sync with current hive` timet_travel` syntax.
I will be change the syntax and add the syntax `FOR SYSTEM_TIME AS OF`.
The new syntax will be as follows:
```
ALTER TABLE tableName
{CREATE BRANCH branchName [FOR SYSTEM_VERSION AS OF {snapshotId} | FOR
SYSTEM_TIME AS OF {timestamp}]
[RETAIN interval {DAYS | HOURS | MINUTES}]
[WITH SNAPSHOT RETENTION {[num_snapshots SNAPSHOTS] [interval {DAYS | HOURS
| MINUTES}]}]}]
```
Issue Time Tracking
---
Worklog Id: (was: 860258)
Time Spent: 5h 20m (was: 5h 10m)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 5h 20m
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=859954&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-859954
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 02/May/23 02:49
Start Date: 02/May/23 02:49
Worklog Time Spent: 10m
Work Description: ayushtkn commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1182036232
##
parser/src/java/org/apache/hadoop/hive/ql/parse/AlterClauseParser.g:
##
@@ -477,6 +478,34 @@ alterStatementSuffixExecute
-> ^(TOK_ALTERTABLE_EXECUTE KW_SET_CURRENT_SNAPSHOT $snapshotParam)
;
+alterStatementSuffixCreateBranch
+@init { gParent.pushMsg("alter table create branch", state); }
+@after { gParent.popMsg(state); }
+: KW_CREATE KW_BRANCH branchName=identifier snapshotIdOfBranch?
branchRetain? retentionOfSnapshots?
+-> ^(TOK_ALTERTABLE_CREATE_BRANCH $branchName snapshotIdOfBranch?
branchRetain? retentionOfSnapshots?)
+;
+
+snapshotIdOfBranch
+@init { gParent.pushMsg("alter table create branch as of version", state); }
Review Comment:
Makes sense to be in sync. @zhangbutao can you change it
Issue Time Tracking
---
Worklog Id: (was: 859954)
Time Spent: 5h 10m (was: 5h)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 5h 10m
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=859879&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-859879
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 01/May/23 14:55
Start Date: 01/May/23 14:55
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1181520628
##
parser/src/java/org/apache/hadoop/hive/ql/parse/AlterClauseParser.g:
##
@@ -477,6 +478,34 @@ alterStatementSuffixExecute
-> ^(TOK_ALTERTABLE_EXECUTE KW_SET_CURRENT_SNAPSHOT $snapshotParam)
;
+alterStatementSuffixCreateBranch
+@init { gParent.pushMsg("alter table create branch", state); }
+@after { gParent.popMsg(state); }
+: KW_CREATE KW_BRANCH branchName=identifier snapshotIdOfBranch?
branchRetain? retentionOfSnapshots?
+-> ^(TOK_ALTERTABLE_CREATE_BRANCH $branchName snapshotIdOfBranch?
branchRetain? retentionOfSnapshots?)
+;
+
+snapshotIdOfBranch
+@init { gParent.pushMsg("alter table create branch as of version", state); }
Review Comment:
should we support `as of timestamp` as well? Note, that syntax is different
from the 'select' statement:
FOR SYSTEM_VERSION AS OF
FOR SYSTEM_TIME AS OF
i think we should be consistent here, WDYT? cc @ayushtkn
Issue Time Tracking
---
Worklog Id: (was: 859879)
Time Spent: 5h (was: 4h 50m)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 5h
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=859862&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-859862
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 01/May/23 12:10
Start Date: 01/May/23 12:10
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1181518994
##
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java:
##
@@ -676,6 +678,35 @@ public void
executeOperation(org.apache.hadoop.hive.ql.metadata.Table hmsTable,
}
}
+ @Override
+ public void createBranchOperation(org.apache.hadoop.hive.ql.metadata.Table
hmsTable,
+ AlterTableCreateBranchSpec createBranchSpec) {
+TableDesc tableDesc = Utilities.getTableDesc(hmsTable);
+Table icebergTable = IcebergTableUtil.getTable(conf,
tableDesc.getProperties());
+
+String branchName = createBranchSpec.getBranchName();
+Optional.ofNullable(icebergTable.currentSnapshot()).orElseThrow(() -> new
UnsupportedOperationException(
+String.format("Cannot create branch %s on iceberg table %s.%s which
has no snapshot",
+branchName, hmsTable.getDbName(), hmsTable.getTableName(;
+Long snapshotId = Optional.ofNullable(createBranchSpec.getSnapshotId())
+.orElse(icebergTable.currentSnapshot().snapshotId());
+LOG.info("Creating branch {} on iceberg table {}.{}", branchName,
hmsTable.getDbName(),
Review Comment:
Could we add `snapshotId` to the log output? Btw, should we log under DEBUG?
##
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java:
##
@@ -676,6 +678,35 @@ public void
executeOperation(org.apache.hadoop.hive.ql.metadata.Table hmsTable,
}
}
+ @Override
+ public void createBranchOperation(org.apache.hadoop.hive.ql.metadata.Table
hmsTable,
+ AlterTableCreateBranchSpec createBranchSpec) {
+TableDesc tableDesc = Utilities.getTableDesc(hmsTable);
+Table icebergTable = IcebergTableUtil.getTable(conf,
tableDesc.getProperties());
+
+String branchName = createBranchSpec.getBranchName();
+Optional.ofNullable(icebergTable.currentSnapshot()).orElseThrow(() -> new
UnsupportedOperationException(
+String.format("Cannot create branch %s on iceberg table %s.%s which
has no snapshot",
+branchName, hmsTable.getDbName(), hmsTable.getTableName(;
+Long snapshotId = Optional.ofNullable(createBranchSpec.getSnapshotId())
+.orElse(icebergTable.currentSnapshot().snapshotId());
+LOG.info("Creating branch {} on iceberg table {}.{}", branchName,
hmsTable.getDbName(),
+hmsTable.getTableName());
+ManageSnapshots manageSnapshots = icebergTable.manageSnapshots();
+manageSnapshots.createBranch(branchName, snapshotId);
Review Comment:
what happens if supplied `snapshotId` doesn't exist?
##
parser/src/java/org/apache/hadoop/hive/ql/parse/AlterClauseParser.g:
##
@@ -477,6 +478,34 @@ alterStatementSuffixExecute
-> ^(TOK_ALTERTABLE_EXECUTE KW_SET_CURRENT_SNAPSHOT $snapshotParam)
;
+alterStatementSuffixCreateBranch
+@init { gParent.pushMsg("alter table create branch", state); }
+@after { gParent.popMsg(state); }
+: KW_CREATE KW_BRANCH branchName=identifier snapshotIdOfBranch?
branchRetain? retentionOfSnapshots?
+-> ^(TOK_ALTERTABLE_CREATE_BRANCH $branchName snapshotIdOfBranch?
branchRetain? retentionOfSnapshots?)
+;
+
+snapshotIdOfBranch
+@init { gParent.pushMsg("alter table create branch as of version", state); }
+@after { gParent.popMsg(state); }
+: KW_AS KW_OF KW_VERSION snapshotId=Number
+-> ^(TOK_AS_OF_VERSION_BRANCH $snapshotId)
Review Comment:
can't we reuse `TOK_AS_OF_VERSION`?
##
parser/src/java/org/apache/hadoop/hive/ql/parse/AlterClauseParser.g:
##
@@ -477,6 +478,34 @@ alterStatementSuffixExecute
-> ^(TOK_ALTERTABLE_EXECUTE KW_SET_CURRENT_SNAPSHOT $snapshotParam)
;
+alterStatementSuffixCreateBranch
+@init { gParent.pushMsg("alter table create branch", state); }
+@after { gParent.popMsg(state); }
+: KW_CREATE KW_BRANCH branchName=identifier snapshotIdOfBranch?
branchRetain? retentionOfSnapshots?
+-> ^(TOK_ALTERTABLE_CREATE_BRANCH $branchName snapshotIdOfBranch?
branchRetain? retentionOfSnapshots?)
+;
+
+snapshotIdOfBranch
+@init { gParent.pushMsg("alter table create branch as of version", state); }
Review Comment:
should we support `as of timestamp` as well? Note, that syntax is different
from the 'select' statement:
FOR SYSTEM_VERSION AS OF
FOR SYSTEM_TIME AS OF
i think we should be consistent here. cc @ayushtkn
##
ql/src/java/org/apache/hadoop/hive/ql/ddl/table/AlterTableType.java:
###
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[ https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=859815&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-859815 ] ASF GitHub Bot logged work on HIVE-27234: - Author: ASF GitHub Bot Created on: 30/Apr/23 02:44 Start Date: 30/Apr/23 02:44 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4216: URL: https://github.com/apache/hive/pull/4216#issuecomment-1528924677 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4216) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [2 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [3 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4216&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4216&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 859815) Time Spent: 4h 40m (was: 4.5h) > Iceberg: CREATE BRANCH SQL implementation > -- > > Key: HIVE-27234 > URL: https://issues.apache.org/jira/browse/HIVE-27234 > Project: Hive > Issue Type: Sub-task > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > Time Spent: 4h 40m > Remaining Estimate: 0h > > Maybe we can follow spark sql about branch ddl implementation > [https://github.com/apache/iceberg/pull/6617] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[ https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=859801&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-859801 ] ASF GitHub Bot logged work on HIVE-27234: - Author: ASF GitHub Bot Created on: 29/Apr/23 18:23 Start Date: 29/Apr/23 18:23 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4216: URL: https://github.com/apache/hive/pull/4216#issuecomment-1528844875 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4216) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [2 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [3 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4216&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4216&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 859801) Time Spent: 4.5h (was: 4h 20m) > Iceberg: CREATE BRANCH SQL implementation > -- > > Key: HIVE-27234 > URL: https://issues.apache.org/jira/browse/HIVE-27234 > Project: Hive > Issue Type: Sub-task > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > Time Spent: 4.5h > Remaining Estimate: 0h > > Maybe we can follow spark sql about branch ddl implementation > [https://github.com/apache/iceberg/pull/6617] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=859800&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-859800
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 29/Apr/23 17:29
Start Date: 29/Apr/23 17:29
Worklog Time Spent: 10m
Work Description: zhangbutao commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1181105965
##
iceberg/iceberg-handler/src/test/queries/negative/alter_table_create_branch_negative.q:
##
@@ -0,0 +1,3 @@
+create table ice_tbl (id int, name string) Stored by Iceberg;
+
+alter table ice_tbl create branch test_branch_1;
Review Comment:
Qtest: creating branch on table without current snapshot will fail
##
iceberg/iceberg-handler/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergBranchOperation.java:
##
@@ -0,0 +1,166 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iceberg.mr.hive;
+
+import java.io.IOException;
+import java.util.concurrent.TimeUnit;
+import org.apache.iceberg.SnapshotRef;
+import org.apache.iceberg.Table;
+import org.junit.Assert;
+import org.junit.Test;
+
+public class TestHiveIcebergBranchOperation extends
HiveIcebergStorageHandlerWithEngineBase {
+
+ @Test
+ public void testCreateBranchWithDefaultConfig() throws InterruptedException,
IOException {
+Table table =
+testTables.createTableWithVersions(shell, "customers",
HiveIcebergStorageHandlerTestUtils.CUSTOMER_SCHEMA,
+fileFormat, HiveIcebergStorageHandlerTestUtils.CUSTOMER_RECORDS,
2);
+
+String branchName = "test_branch_1";
+shell.executeStatement(String.format("ALTER TABLE customers CREATE BRANCH
%s", branchName));
+table.refresh();
+SnapshotRef ref = table.refs().get(branchName);
+Assert.assertEquals(table.currentSnapshot().snapshotId(),
ref.snapshotId());
+Assert.assertNull(ref.minSnapshotsToKeep());
+Assert.assertNull(ref.maxSnapshotAgeMs());
+Assert.assertNull(ref.maxRefAgeMs());
+
+// creating a branch which is already exists will fail
+try {
+ shell.executeStatement(String.format("ALTER TABLE customers CREATE
BRANCH %s", branchName));
Review Comment:
Test: creating a branch which is already exists will fail
##
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java:
##
@@ -676,6 +678,35 @@ public void
executeOperation(org.apache.hadoop.hive.ql.metadata.Table hmsTable,
}
}
+ @Override
+ public void createBranchOperation(org.apache.hadoop.hive.ql.metadata.Table
hmsTable,
+ AlterTableCreateBranchSpec createBranchSpec) {
+TableDesc tableDesc = Utilities.getTableDesc(hmsTable);
+Table icebergTable = IcebergTableUtil.getTable(conf,
tableDesc.getProperties());
+
+String branchName = createBranchSpec.getBranchName();
+Optional.ofNullable(icebergTable.currentSnapshot()).orElseThrow(() -> new
UnsupportedOperationException(
Review Comment:
Here give a check if table have current snapshot.
##
iceberg/iceberg-handler/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergBranchOperation.java:
##
@@ -0,0 +1,166 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iceberg.mr.hive;
+
+import java
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[ https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=859799&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-859799 ] ASF GitHub Bot logged work on HIVE-27234: - Author: ASF GitHub Bot Created on: 29/Apr/23 17:24 Start Date: 29/Apr/23 17:24 Worklog Time Spent: 10m Work Description: zhangbutao commented on PR #4216: URL: https://github.com/apache/hive/pull/4216#issuecomment-1528833729 > Thanx @zhangbutao, overall changes looks good, dropped some comments. > > Additionally your code ain't formatted according to hive formatting rules, line length should be less than 120 and other, can refer here: https://github.com/apache/hive/blob/master/dev-support/eclipse-styles.xml > > Additionally add a q test. Where you do an explain of the create branch statement before creating the branch. > > Add some failure test cases: Branch already exist and you try to create a branch Attempt to create a branch on a non iceberg table > > In the positive test case check for the data as well. I think i have addressed these comments. Please take a look again if you have time. Thanks @ayushtkn Issue Time Tracking --- Worklog Id: (was: 859799) Time Spent: 4h 10m (was: 4h) > Iceberg: CREATE BRANCH SQL implementation > -- > > Key: HIVE-27234 > URL: https://issues.apache.org/jira/browse/HIVE-27234 > Project: Hive > Issue Type: Sub-task > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > Time Spent: 4h 10m > Remaining Estimate: 0h > > Maybe we can follow spark sql about branch ddl implementation > [https://github.com/apache/iceberg/pull/6617] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=859797&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-859797
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 29/Apr/23 17:20
Start Date: 29/Apr/23 17:20
Worklog Time Spent: 10m
Work Description: zhangbutao commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1181105133
##
ql/src/java/org/apache/hadoop/hive/ql/ddl/table/branch/create/AlterTableCreateBranchAnalyzer.java:
##
@@ -0,0 +1,89 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.ddl.table.branch.create;
+
+import java.util.Locale;
+import java.util.Map;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.hadoop.hive.common.TableName;
+import org.apache.hadoop.hive.ql.QueryState;
+import org.apache.hadoop.hive.ql.ddl.DDLSemanticAnalyzerFactory;
+import org.apache.hadoop.hive.ql.ddl.DDLWork;
+import org.apache.hadoop.hive.ql.ddl.table.AbstractAlterTableAnalyzer;
+import org.apache.hadoop.hive.ql.ddl.table.AlterTableType;
+import org.apache.hadoop.hive.ql.exec.TaskFactory;
+import org.apache.hadoop.hive.ql.hooks.ReadEntity;
+import org.apache.hadoop.hive.ql.metadata.Table;
+import org.apache.hadoop.hive.ql.parse.ASTNode;
+import org.apache.hadoop.hive.ql.parse.AlterTableCreateBranchSpec;
+import org.apache.hadoop.hive.ql.parse.HiveParser;
+import org.apache.hadoop.hive.ql.parse.SemanticException;
+
+@DDLSemanticAnalyzerFactory.DDLType(types =
HiveParser.TOK_ALTERTABLE_CREATE_BRANCH)
+public class AlterTableCreateBranchAnalyzer extends AbstractAlterTableAnalyzer
{
+
+ public AlterTableCreateBranchAnalyzer(QueryState queryState) throws
SemanticException {
+super(queryState);
+ }
+
+ @Override
+ protected void analyzeCommand(TableName tableName, Map
partitionSpec, ASTNode command)
+ throws SemanticException {
+Table table = getTable(tableName);
+validateAlterTableType(table, AlterTableType.CREATEBRANCH, false);
+if (!"ICEBERG".equalsIgnoreCase(table.getParameters().get("table_type"))) {
+ throw new SemanticException("Cannot perform ALTER CREATE BRANCH
statement on non-iceberg table.");
+}
+inputs.add(new ReadEntity(table));
+
+String branchName = command.getChild(0).getText();
+Long snapshotId = null;
+Long maxRefAgeMs = null;
+Integer minSnapshotsToKeep = null;
+Long maxSnapshotAgeMs = null;
+for (int i = 1; i < command.getChildCount(); i++) {
+ ASTNode childNode = (ASTNode) command.getChild(i);
+ switch (childNode.getToken().getType()) {
+ case HiveParser.TOK_AS_OF_VERSION_BRANCH:
+snapshotId = Long.valueOf(childNode.getChild(0).getText());
+break;
+ case HiveParser.TOK_RETAIN:
+String maxRefAge = childNode.getChild(0).getText();
+String timeUnitOfBranchRetain = childNode.getChild(1).getText();
+maxRefAgeMs =
TimeUnit.valueOf(timeUnitOfBranchRetain.toUpperCase(Locale.ENGLISH)).toMillis(Long.valueOf(maxRefAge));
+break;
+ case HiveParser.TOK_WITH_SNAPSHOT_RETENTION:
+minSnapshotsToKeep = Integer.valueOf(childNode.getChild(0).getText());
+if (childNode.getChildren().size() > 1) {
+ String maxSnapshotAge = childNode.getChild(1).getText();
+ String timeUnitOfSnapshotsRetention =
childNode.getChild(2).getText();
+ maxSnapshotAgeMs =
TimeUnit.valueOf(timeUnitOfSnapshotsRetention.toUpperCase(Locale.ENGLISH)).toMillis(Long.valueOf(maxSnapshotAge));
+}
+break;
+ default:
+throw new SemanticException("Unrecognized token in ALTER CREATE BRANCH
statement");
+ }
+}
+
+AlterTableCreateBranchSpec spec = new
AlterTableCreateBranchSpec(branchName, snapshotId, maxRefAgeMs,
minSnapshotsToKeep, maxSnapshotAgeMs);
Review Comment:
fixed. Thx.
Issue Time Tracking
---
Worklog Id: (was: 859797)
Time Spent: 4h (was: 3h 50m)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
>
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=859795&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-859795
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 29/Apr/23 17:19
Start Date: 29/Apr/23 17:19
Worklog Time Spent: 10m
Work Description: zhangbutao commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1181105049
##
ql/src/java/org/apache/hadoop/hive/ql/ddl/table/branch/create/AlterTableCreateBranchAnalyzer.java:
##
@@ -0,0 +1,89 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.ddl.table.branch.create;
+
+import java.util.Locale;
+import java.util.Map;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.hadoop.hive.common.TableName;
+import org.apache.hadoop.hive.ql.QueryState;
+import org.apache.hadoop.hive.ql.ddl.DDLSemanticAnalyzerFactory;
+import org.apache.hadoop.hive.ql.ddl.DDLWork;
+import org.apache.hadoop.hive.ql.ddl.table.AbstractAlterTableAnalyzer;
+import org.apache.hadoop.hive.ql.ddl.table.AlterTableType;
+import org.apache.hadoop.hive.ql.exec.TaskFactory;
+import org.apache.hadoop.hive.ql.hooks.ReadEntity;
+import org.apache.hadoop.hive.ql.metadata.Table;
+import org.apache.hadoop.hive.ql.parse.ASTNode;
+import org.apache.hadoop.hive.ql.parse.AlterTableCreateBranchSpec;
+import org.apache.hadoop.hive.ql.parse.HiveParser;
+import org.apache.hadoop.hive.ql.parse.SemanticException;
+
+@DDLSemanticAnalyzerFactory.DDLType(types =
HiveParser.TOK_ALTERTABLE_CREATE_BRANCH)
+public class AlterTableCreateBranchAnalyzer extends AbstractAlterTableAnalyzer
{
+
+ public AlterTableCreateBranchAnalyzer(QueryState queryState) throws
SemanticException {
+super(queryState);
+ }
+
+ @Override
+ protected void analyzeCommand(TableName tableName, Map
partitionSpec, ASTNode command)
+ throws SemanticException {
+Table table = getTable(tableName);
+validateAlterTableType(table, AlterTableType.CREATEBRANCH, false);
+if (!"ICEBERG".equalsIgnoreCase(table.getParameters().get("table_type"))) {
Review Comment:
fixed. Thx
Issue Time Tracking
---
Worklog Id: (was: 859795)
Time Spent: 3h 40m (was: 3.5h)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=859794&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-859794
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 29/Apr/23 17:19
Start Date: 29/Apr/23 17:19
Worklog Time Spent: 10m
Work Description: zhangbutao commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1181104998
##
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java:
##
@@ -676,6 +678,32 @@ public void
executeOperation(org.apache.hadoop.hive.ql.metadata.Table hmsTable,
}
}
+ @Override
+ public void createBranchOperation(org.apache.hadoop.hive.ql.metadata.Table
hmsTable,
+ AlterTableCreateBranchSpec createBranchSpec) {
+TableDesc tableDesc = Utilities.getTableDesc(hmsTable);
+Table icebergTable = IcebergTableUtil.getTable(conf,
tableDesc.getProperties());
+
+String branchName = createBranchSpec.getBranchName();
+ManageSnapshots manageSnapshots = icebergTable.manageSnapshots();
+Long snapShotId = Optional.ofNullable(createBranchSpec.getSnapshotId())
+.orElse(icebergTable.currentSnapshot().snapshotId());
+LOG.info("Creating branch {} on iceberg table {}.{}", branchName,
hmsTable.getDbName(),
+hmsTable.getTableName());
+manageSnapshots.createBranch(branchName, snapShotId);
Review Comment:
fixed.
Issue Time Tracking
---
Worklog Id: (was: 859794)
Time Spent: 3.5h (was: 3h 20m)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=859796&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-859796
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 29/Apr/23 17:19
Start Date: 29/Apr/23 17:19
Worklog Time Spent: 10m
Work Description: zhangbutao commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1181105078
##
ql/src/java/org/apache/hadoop/hive/ql/ddl/table/branch/create/AlterTableCreateBranchAnalyzer.java:
##
@@ -0,0 +1,89 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.ddl.table.branch.create;
+
+import java.util.Locale;
+import java.util.Map;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.hadoop.hive.common.TableName;
+import org.apache.hadoop.hive.ql.QueryState;
+import org.apache.hadoop.hive.ql.ddl.DDLSemanticAnalyzerFactory;
+import org.apache.hadoop.hive.ql.ddl.DDLWork;
+import org.apache.hadoop.hive.ql.ddl.table.AbstractAlterTableAnalyzer;
+import org.apache.hadoop.hive.ql.ddl.table.AlterTableType;
+import org.apache.hadoop.hive.ql.exec.TaskFactory;
+import org.apache.hadoop.hive.ql.hooks.ReadEntity;
+import org.apache.hadoop.hive.ql.metadata.Table;
+import org.apache.hadoop.hive.ql.parse.ASTNode;
+import org.apache.hadoop.hive.ql.parse.AlterTableCreateBranchSpec;
+import org.apache.hadoop.hive.ql.parse.HiveParser;
+import org.apache.hadoop.hive.ql.parse.SemanticException;
+
+@DDLSemanticAnalyzerFactory.DDLType(types =
HiveParser.TOK_ALTERTABLE_CREATE_BRANCH)
+public class AlterTableCreateBranchAnalyzer extends AbstractAlterTableAnalyzer
{
+
+ public AlterTableCreateBranchAnalyzer(QueryState queryState) throws
SemanticException {
+super(queryState);
+ }
+
+ @Override
+ protected void analyzeCommand(TableName tableName, Map
partitionSpec, ASTNode command)
+ throws SemanticException {
+Table table = getTable(tableName);
+validateAlterTableType(table, AlterTableType.CREATEBRANCH, false);
+if (!"ICEBERG".equalsIgnoreCase(table.getParameters().get("table_type"))) {
+ throw new SemanticException("Cannot perform ALTER CREATE BRANCH
statement on non-iceberg table.");
+}
+inputs.add(new ReadEntity(table));
+
+String branchName = command.getChild(0).getText();
+Long snapshotId = null;
+Long maxRefAgeMs = null;
+Integer minSnapshotsToKeep = null;
+Long maxSnapshotAgeMs = null;
+for (int i = 1; i < command.getChildCount(); i++) {
+ ASTNode childNode = (ASTNode) command.getChild(i);
+ switch (childNode.getToken().getType()) {
+ case HiveParser.TOK_AS_OF_VERSION_BRANCH:
+snapshotId = Long.valueOf(childNode.getChild(0).getText());
+break;
+ case HiveParser.TOK_RETAIN:
+String maxRefAge = childNode.getChild(0).getText();
+String timeUnitOfBranchRetain = childNode.getChild(1).getText();
+maxRefAgeMs =
TimeUnit.valueOf(timeUnitOfBranchRetain.toUpperCase(Locale.ENGLISH)).toMillis(Long.valueOf(maxRefAge));
Review Comment:
fixed. Thx.
Issue Time Tracking
---
Worklog Id: (was: 859796)
Time Spent: 3h 50m (was: 3h 40m)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=859793&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-859793
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 29/Apr/23 17:19
Start Date: 29/Apr/23 17:19
Worklog Time Spent: 10m
Work Description: zhangbutao commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1181104982
##
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java:
##
@@ -676,6 +678,32 @@ public void
executeOperation(org.apache.hadoop.hive.ql.metadata.Table hmsTable,
}
}
+ @Override
+ public void createBranchOperation(org.apache.hadoop.hive.ql.metadata.Table
hmsTable,
+ AlterTableCreateBranchSpec createBranchSpec) {
+TableDesc tableDesc = Utilities.getTableDesc(hmsTable);
+Table icebergTable = IcebergTableUtil.getTable(conf,
tableDesc.getProperties());
+
+String branchName = createBranchSpec.getBranchName();
+ManageSnapshots manageSnapshots = icebergTable.manageSnapshots();
+Long snapShotId = Optional.ofNullable(createBranchSpec.getSnapshotId())
+.orElse(icebergTable.currentSnapshot().snapshotId());
Review Comment:
Nice catch! fixed. Thanks very much!
Issue Time Tracking
---
Worklog Id: (was: 859793)
Time Spent: 3h 20m (was: 3h 10m)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=859792&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-859792
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 29/Apr/23 17:15
Start Date: 29/Apr/23 17:15
Worklog Time Spent: 10m
Work Description: zhangbutao commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1181104673
##
iceberg/iceberg-handler/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergBranchOperation.java:
##
@@ -0,0 +1,141 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iceberg.mr.hive;
+
+import java.io.IOException;
+import java.util.concurrent.TimeUnit;
+import org.apache.iceberg.SnapshotRef;
+import org.apache.iceberg.Table;
+import org.junit.Assert;
+import org.junit.Test;
+
+public class TestHiveIcebergBranchOperation extends
HiveIcebergStorageHandlerWithEngineBase {
+
+ @Test
Review Comment:
I think multiple small tests are better for reading. But if you think one
test is ok, I'm fine with that. ;)
Issue Time Tracking
---
Worklog Id: (was: 859792)
Time Spent: 3h 10m (was: 3h)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[ https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=859690&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-859690 ] ASF GitHub Bot logged work on HIVE-27234: - Author: ASF GitHub Bot Created on: 28/Apr/23 15:42 Start Date: 28/Apr/23 15:42 Worklog Time Spent: 10m Work Description: zhangbutao commented on PR #4216: URL: https://github.com/apache/hive/pull/4216#issuecomment-1527752401 @ayushtkn Thanks for review! I will probably fix these comments after a few days of vacation. Issue Time Tracking --- Worklog Id: (was: 859690) Time Spent: 3h (was: 2h 50m) > Iceberg: CREATE BRANCH SQL implementation > -- > > Key: HIVE-27234 > URL: https://issues.apache.org/jira/browse/HIVE-27234 > Project: Hive > Issue Type: Sub-task > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > Time Spent: 3h > Remaining Estimate: 0h > > Maybe we can follow spark sql about branch ddl implementation > [https://github.com/apache/iceberg/pull/6617] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[ https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=859581&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-859581 ] ASF GitHub Bot logged work on HIVE-27234: - Author: ASF GitHub Bot Created on: 28/Apr/23 08:37 Start Date: 28/Apr/23 08:37 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4216: URL: https://github.com/apache/hive/pull/4216#issuecomment-1527193917 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4216) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [1 Bug](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [3 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4216&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4216&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 859581) Time Spent: 2h 50m (was: 2h 40m) > Iceberg: CREATE BRANCH SQL implementation > -- > > Key: HIVE-27234 > URL: https://issues.apache.org/jira/browse/HIVE-27234 > Project: Hive > Issue Type: Sub-task > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > Time Spent: 2h 50m > Remaining Estimate: 0h > > Maybe we can follow spark sql about branch ddl implementation > [https://github.com/apache/iceberg/pull/6617] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=859548&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-859548
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 28/Apr/23 04:46
Start Date: 28/Apr/23 04:46
Worklog Time Spent: 10m
Work Description: ayushtkn commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1179920182
##
ql/src/java/org/apache/hadoop/hive/ql/ddl/table/branch/create/AlterTableCreateBranchAnalyzer.java:
##
@@ -0,0 +1,89 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.ddl.table.branch.create;
+
+import java.util.Locale;
+import java.util.Map;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.hadoop.hive.common.TableName;
+import org.apache.hadoop.hive.ql.QueryState;
+import org.apache.hadoop.hive.ql.ddl.DDLSemanticAnalyzerFactory;
+import org.apache.hadoop.hive.ql.ddl.DDLWork;
+import org.apache.hadoop.hive.ql.ddl.table.AbstractAlterTableAnalyzer;
+import org.apache.hadoop.hive.ql.ddl.table.AlterTableType;
+import org.apache.hadoop.hive.ql.exec.TaskFactory;
+import org.apache.hadoop.hive.ql.hooks.ReadEntity;
+import org.apache.hadoop.hive.ql.metadata.Table;
+import org.apache.hadoop.hive.ql.parse.ASTNode;
+import org.apache.hadoop.hive.ql.parse.AlterTableCreateBranchSpec;
+import org.apache.hadoop.hive.ql.parse.HiveParser;
+import org.apache.hadoop.hive.ql.parse.SemanticException;
+
+@DDLSemanticAnalyzerFactory.DDLType(types =
HiveParser.TOK_ALTERTABLE_CREATE_BRANCH)
+public class AlterTableCreateBranchAnalyzer extends AbstractAlterTableAnalyzer
{
+
+ public AlterTableCreateBranchAnalyzer(QueryState queryState) throws
SemanticException {
+super(queryState);
+ }
+
+ @Override
+ protected void analyzeCommand(TableName tableName, Map
partitionSpec, ASTNode command)
+ throws SemanticException {
+Table table = getTable(tableName);
+validateAlterTableType(table, AlterTableType.CREATEBRANCH, false);
+if (!"ICEBERG".equalsIgnoreCase(table.getParameters().get("table_type"))) {
+ throw new SemanticException("Cannot perform ALTER CREATE BRANCH
statement on non-iceberg table.");
+}
+inputs.add(new ReadEntity(table));
+
+String branchName = command.getChild(0).getText();
+Long snapshotId = null;
+Long maxRefAgeMs = null;
+Integer minSnapshotsToKeep = null;
+Long maxSnapshotAgeMs = null;
+for (int i = 1; i < command.getChildCount(); i++) {
+ ASTNode childNode = (ASTNode) command.getChild(i);
+ switch (childNode.getToken().getType()) {
+ case HiveParser.TOK_AS_OF_VERSION_BRANCH:
+snapshotId = Long.valueOf(childNode.getChild(0).getText());
+break;
+ case HiveParser.TOK_RETAIN:
+String maxRefAge = childNode.getChild(0).getText();
+String timeUnitOfBranchRetain = childNode.getChild(1).getText();
+maxRefAgeMs =
TimeUnit.valueOf(timeUnitOfBranchRetain.toUpperCase(Locale.ENGLISH)).toMillis(Long.valueOf(maxRefAge));
+break;
+ case HiveParser.TOK_WITH_SNAPSHOT_RETENTION:
+minSnapshotsToKeep = Integer.valueOf(childNode.getChild(0).getText());
+if (childNode.getChildren().size() > 1) {
+ String maxSnapshotAge = childNode.getChild(1).getText();
+ String timeUnitOfSnapshotsRetention =
childNode.getChild(2).getText();
+ maxSnapshotAgeMs =
TimeUnit.valueOf(timeUnitOfSnapshotsRetention.toUpperCase(Locale.ENGLISH)).toMillis(Long.valueOf(maxSnapshotAge));
Review Comment:
use. ``Long.parseLong(maxSnapshotAge)``
Issue Time Tracking
---
Worklog Id: (was: 859548)
Time Spent: 2h 40m (was: 2.5h)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=859547&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-859547
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 28/Apr/23 04:46
Start Date: 28/Apr/23 04:46
Worklog Time Spent: 10m
Work Description: ayushtkn commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1179917689
##
iceberg/iceberg-handler/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergBranchOperation.java:
##
@@ -0,0 +1,141 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iceberg.mr.hive;
+
+import java.io.IOException;
+import java.util.concurrent.TimeUnit;
+import org.apache.iceberg.SnapshotRef;
+import org.apache.iceberg.Table;
+import org.junit.Assert;
+import org.junit.Test;
+
+public class TestHiveIcebergBranchOperation extends
HiveIcebergStorageHandlerWithEngineBase {
+
+ @Test
Review Comment:
Do you need so many tests? Just create a table once and can create multiple
branches of that with different parameters ?
##
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java:
##
@@ -676,6 +678,32 @@ public void
executeOperation(org.apache.hadoop.hive.ql.metadata.Table hmsTable,
}
}
+ @Override
+ public void createBranchOperation(org.apache.hadoop.hive.ql.metadata.Table
hmsTable,
+ AlterTableCreateBranchSpec createBranchSpec) {
+TableDesc tableDesc = Utilities.getTableDesc(hmsTable);
+Table icebergTable = IcebergTableUtil.getTable(conf,
tableDesc.getProperties());
+
+String branchName = createBranchSpec.getBranchName();
+ManageSnapshots manageSnapshots = icebergTable.manageSnapshots();
+Long snapShotId = Optional.ofNullable(createBranchSpec.getSnapshotId())
+.orElse(icebergTable.currentSnapshot().snapshotId());
Review Comment:
What happens to this in case of any empty table? Just a create iceberg table
and nothing. As far as I remember in that case currentSnapshot is null? Just
give a check once.
##
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java:
##
@@ -676,6 +678,32 @@ public void
executeOperation(org.apache.hadoop.hive.ql.metadata.Table hmsTable,
}
}
+ @Override
+ public void createBranchOperation(org.apache.hadoop.hive.ql.metadata.Table
hmsTable,
+ AlterTableCreateBranchSpec createBranchSpec) {
+TableDesc tableDesc = Utilities.getTableDesc(hmsTable);
+Table icebergTable = IcebergTableUtil.getTable(conf,
tableDesc.getProperties());
+
+String branchName = createBranchSpec.getBranchName();
+ManageSnapshots manageSnapshots = icebergTable.manageSnapshots();
+Long snapShotId = Optional.ofNullable(createBranchSpec.getSnapshotId())
+.orElse(icebergTable.currentSnapshot().snapshotId());
+LOG.info("Creating branch {} on iceberg table {}.{}", branchName,
hmsTable.getDbName(),
+hmsTable.getTableName());
+manageSnapshots.createBranch(branchName, snapShotId);
Review Comment:
nit:
snapShotId -> snapshotId
##
ql/src/java/org/apache/hadoop/hive/ql/ddl/table/branch/create/AlterTableCreateBranchAnalyzer.java:
##
@@ -0,0 +1,89 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations und
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[ https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=858111&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-858111 ] ASF GitHub Bot logged work on HIVE-27234: - Author: ASF GitHub Bot Created on: 20/Apr/23 07:28 Start Date: 20/Apr/23 07:28 Worklog Time Spent: 10m Work Description: TuroczyX commented on PR #4216: URL: https://github.com/apache/hive/pull/4216#issuecomment-1515852966 yes, let's upgrade the iceberg version Issue Time Tracking --- Worklog Id: (was: 858111) Time Spent: 2h 20m (was: 2h 10m) > Iceberg: CREATE BRANCH SQL implementation > -- > > Key: HIVE-27234 > URL: https://issues.apache.org/jira/browse/HIVE-27234 > Project: Hive > Issue Type: Sub-task > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > Maybe we can follow spark sql about branch ddl implementation > [https://github.com/apache/iceberg/pull/6617] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[ https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=858087&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-858087 ] ASF GitHub Bot logged work on HIVE-27234: - Author: ASF GitHub Bot Created on: 20/Apr/23 04:37 Start Date: 20/Apr/23 04:37 Worklog Time Spent: 10m Work Description: zhangbutao commented on PR #4216: URL: https://github.com/apache/hive/pull/4216#issuecomment-1515700762 https://github.com/apache/hive/pull/4252 Upgrade iceberg to 1.2.1 in order to better integrate branch&tag features. Issue Time Tracking --- Worklog Id: (was: 858087) Time Spent: 2h 10m (was: 2h) > Iceberg: CREATE BRANCH SQL implementation > -- > > Key: HIVE-27234 > URL: https://issues.apache.org/jira/browse/HIVE-27234 > Project: Hive > Issue Type: Sub-task > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > Time Spent: 2h 10m > Remaining Estimate: 0h > > Maybe we can follow spark sql about branch ddl implementation > [https://github.com/apache/iceberg/pull/6617] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=858071&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-858071
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 20/Apr/23 02:20
Start Date: 20/Apr/23 02:20
Worklog Time Spent: 10m
Work Description: zhangbutao commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1172014050
##
ql/src/java/org/apache/hadoop/hive/ql/ddl/table/branch/create/AlterTableCreateBranchAnalyzer.java:
##
@@ -0,0 +1,89 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.ddl.table.branch.create;
+
+import java.util.Locale;
+import java.util.Map;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.hadoop.hive.common.TableName;
+import org.apache.hadoop.hive.ql.QueryState;
+import org.apache.hadoop.hive.ql.ddl.DDLSemanticAnalyzerFactory;
+import org.apache.hadoop.hive.ql.ddl.DDLWork;
+import org.apache.hadoop.hive.ql.ddl.table.AbstractAlterTableAnalyzer;
+import org.apache.hadoop.hive.ql.ddl.table.AlterTableType;
+import org.apache.hadoop.hive.ql.exec.TaskFactory;
+import org.apache.hadoop.hive.ql.hooks.ReadEntity;
+import org.apache.hadoop.hive.ql.metadata.Table;
+import org.apache.hadoop.hive.ql.parse.ASTNode;
+import org.apache.hadoop.hive.ql.parse.AlterTableCreateBranchSpec;
+import org.apache.hadoop.hive.ql.parse.HiveParser;
+import org.apache.hadoop.hive.ql.parse.SemanticException;
+
+@DDLSemanticAnalyzerFactory.DDLType(types =
HiveParser.TOK_ALTERTABLE_CREATE_BRANCH)
+public class AlterTableCreateBranchAnalyzer extends AbstractAlterTableAnalyzer
{
+
+ public AlterTableCreateBranchAnalyzer(QueryState queryState) throws
SemanticException {
+super(queryState);
+ }
+
+ @Override
+ protected void analyzeCommand(TableName tableName, Map
partitionSpec, ASTNode command)
Review Comment:
Thanks for comment @TuroczyX . IIUC, I think no need check null here as the
parser moduel will do this. We have defined all required and optional
parameters in [`AlterClauseParser.g`
](https://github.com/apache/hive/pull/4216/files#diff-3b0a324eebcad6e32a1bf8bf42499df0739207dbe96f32a3d5e64a44eb002d0fR473),
and the then Antlr parser will check if the parameter is correct.
Issue Time Tracking
---
Worklog Id: (was: 858071)
Time Spent: 2h (was: 1h 50m)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[ https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=858068&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-858068 ] ASF GitHub Bot logged work on HIVE-27234: - Author: ASF GitHub Bot Created on: 20/Apr/23 01:59 Start Date: 20/Apr/23 01:59 Worklog Time Spent: 10m Work Description: zhangbutao commented on PR #4216: URL: https://github.com/apache/hive/pull/4216#issuecomment-1515606342 @TuroczyX @ayushtkn yes, branch and tag are important feats in iceberg 1.2.0. I have created a umbrella ticket https://issues.apache.org/jira/browse/HIVE-27233 to track the new feats about branch&tag. > The main thing to chase is not creating a branch. But to insert into those branches. I absolutely agree. I am exploring how to achieve this in hive. Next maybe we shoud consider upgrade iceberg to latest version 1.2.1. In addition, we can refer to this PR to implement the sql syntax for` insert into branch` Issue Time Tracking --- Worklog Id: (was: 858068) Time Spent: 1h 50m (was: 1h 40m) > Iceberg: CREATE BRANCH SQL implementation > -- > > Key: HIVE-27234 > URL: https://issues.apache.org/jira/browse/HIVE-27234 > Project: Hive > Issue Type: Sub-task > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > Maybe we can follow spark sql about branch ddl implementation > [https://github.com/apache/iceberg/pull/6617] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=858058&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-858058
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 19/Apr/23 22:35
Start Date: 19/Apr/23 22:35
Worklog Time Spent: 10m
Work Description: TuroczyX commented on code in PR #4216:
URL: https://github.com/apache/hive/pull/4216#discussion_r1171914271
##
ql/src/java/org/apache/hadoop/hive/ql/ddl/table/branch/create/AlterTableCreateBranchAnalyzer.java:
##
@@ -0,0 +1,89 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.ddl.table.branch.create;
+
+import java.util.Locale;
+import java.util.Map;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.hadoop.hive.common.TableName;
+import org.apache.hadoop.hive.ql.QueryState;
+import org.apache.hadoop.hive.ql.ddl.DDLSemanticAnalyzerFactory;
+import org.apache.hadoop.hive.ql.ddl.DDLWork;
+import org.apache.hadoop.hive.ql.ddl.table.AbstractAlterTableAnalyzer;
+import org.apache.hadoop.hive.ql.ddl.table.AlterTableType;
+import org.apache.hadoop.hive.ql.exec.TaskFactory;
+import org.apache.hadoop.hive.ql.hooks.ReadEntity;
+import org.apache.hadoop.hive.ql.metadata.Table;
+import org.apache.hadoop.hive.ql.parse.ASTNode;
+import org.apache.hadoop.hive.ql.parse.AlterTableCreateBranchSpec;
+import org.apache.hadoop.hive.ql.parse.HiveParser;
+import org.apache.hadoop.hive.ql.parse.SemanticException;
+
+@DDLSemanticAnalyzerFactory.DDLType(types =
HiveParser.TOK_ALTERTABLE_CREATE_BRANCH)
+public class AlterTableCreateBranchAnalyzer extends AbstractAlterTableAnalyzer
{
+
+ public AlterTableCreateBranchAnalyzer(QueryState queryState) throws
SemanticException {
+super(queryState);
+ }
+
+ @Override
+ protected void analyzeCommand(TableName tableName, Map
partitionSpec, ASTNode command)
Review Comment:
I know other does not use it, in the code, but in defensive code, when you
pass a reference it should be checked if it not a null in the beginning of the
method. Just to avoid any unpredicted null reference exception. It is pretty
optional, because other commands does not do it at all.
Btw nice code.
Issue Time Tracking
---
Worklog Id: (was: 858058)
Time Spent: 1h 40m (was: 1.5h)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[ https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=858056&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-858056 ] ASF GitHub Bot logged work on HIVE-27234: - Author: ASF GitHub Bot Created on: 19/Apr/23 22:29 Start Date: 19/Apr/23 22:29 Worklog Time Spent: 10m Work Description: TuroczyX commented on PR #4216: URL: https://github.com/apache/hive/pull/4216#issuecomment-1515464168  Issue Time Tracking --- Worklog Id: (was: 858056) Time Spent: 1.5h (was: 1h 20m) > Iceberg: CREATE BRANCH SQL implementation > -- > > Key: HIVE-27234 > URL: https://issues.apache.org/jira/browse/HIVE-27234 > Project: Hive > Issue Type: Sub-task > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > > Maybe we can follow spark sql about branch ddl implementation > [https://github.com/apache/iceberg/pull/6617] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[ https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=858055&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-858055 ] ASF GitHub Bot logged work on HIVE-27234: - Author: ASF GitHub Bot Created on: 19/Apr/23 22:27 Start Date: 19/Apr/23 22:27 Worklog Time Spent: 10m Work Description: TuroczyX commented on PR #4216: URL: https://github.com/apache/hive/pull/4216#issuecomment-1515462400 Tags 😍 Issue Time Tracking --- Worklog Id: (was: 858055) Time Spent: 1h 20m (was: 1h 10m) > Iceberg: CREATE BRANCH SQL implementation > -- > > Key: HIVE-27234 > URL: https://issues.apache.org/jira/browse/HIVE-27234 > Project: Hive > Issue Type: Sub-task > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > > Maybe we can follow spark sql about branch ddl implementation > [https://github.com/apache/iceberg/pull/6617] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[ https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=858054&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-858054 ] ASF GitHub Bot logged work on HIVE-27234: - Author: ASF GitHub Bot Created on: 19/Apr/23 22:18 Start Date: 19/Apr/23 22:18 Worklog Time Spent: 10m Work Description: ayushtkn commented on PR #4216: URL: https://github.com/apache/hive/pull/4216#issuecomment-1515454235 Didn't check the code, But it was something I was also reading coincidently 2-3 hours before only :) The main thing to chase is not creating a branch. But to insert into those branches. @zhangbutao We should chase that once we have the create in. Will review this in a day or two along with Denys. :) Just FYI. I think Iceberg has options to create Tags as well now, and an option to do cherry-pick to branches as well. They are gonna release it for spark in 1.12.0 Good to have stuff!!! Issue Time Tracking --- Worklog Id: (was: 858054) Time Spent: 1h 10m (was: 1h) > Iceberg: CREATE BRANCH SQL implementation > -- > > Key: HIVE-27234 > URL: https://issues.apache.org/jira/browse/HIVE-27234 > Project: Hive > Issue Type: Sub-task > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > Time Spent: 1h 10m > Remaining Estimate: 0h > > Maybe we can follow spark sql about branch ddl implementation > [https://github.com/apache/iceberg/pull/6617] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[ https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=858051&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-858051 ] ASF GitHub Bot logged work on HIVE-27234: - Author: ASF GitHub Bot Created on: 19/Apr/23 22:11 Start Date: 19/Apr/23 22:11 Worklog Time Spent: 10m Work Description: TuroczyX commented on PR #4216: URL: https://github.com/apache/hive/pull/4216#issuecomment-1515449032 uh! This is super cool idea! I want it! Actually with this it is possible to play on dataset without any risk. I love it! I want it! @deniskuzZ @ayushtkn @simhadri-g Could you please make this review a priority? Documents about it: https://docs.google.com/document/d/1tbATFPrKF3vNlzkgZQdaW8CAJmbjvryfrlg6C2Ci_aA/edit#heading=h.v8gsu2fe19q2 Issue Time Tracking --- Worklog Id: (was: 858051) Time Spent: 1h (was: 50m) > Iceberg: CREATE BRANCH SQL implementation > -- > > Key: HIVE-27234 > URL: https://issues.apache.org/jira/browse/HIVE-27234 > Project: Hive > Issue Type: Sub-task > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > Maybe we can follow spark sql about branch ddl implementation > [https://github.com/apache/iceberg/pull/6617] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[ https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=856354&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-856354 ] ASF GitHub Bot logged work on HIVE-27234: - Author: ASF GitHub Bot Created on: 12/Apr/23 08:00 Start Date: 12/Apr/23 08:00 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4216: URL: https://github.com/apache/hive/pull/4216#issuecomment-1504835458 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4216) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [1 Bug](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [2 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4216&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4216&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 856354) Time Spent: 50m (was: 40m) > Iceberg: CREATE BRANCH SQL implementation > -- > > Key: HIVE-27234 > URL: https://issues.apache.org/jira/browse/HIVE-27234 > Project: Hive > Issue Type: Sub-task > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > Maybe we can follow spark sql about branch ddl implementation > [https://github.com/apache/iceberg/pull/6617] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[ https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=856327&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-856327 ] ASF GitHub Bot logged work on HIVE-27234: - Author: ASF GitHub Bot Created on: 12/Apr/23 06:06 Start Date: 12/Apr/23 06:06 Worklog Time Spent: 10m Work Description: zhangbutao closed pull request #4216: HIVE-27234: Iceberg: CREATE BRANCH SQL implementation URL: https://github.com/apache/hive/pull/4216 Issue Time Tracking --- Worklog Id: (was: 856327) Time Spent: 0.5h (was: 20m) > Iceberg: CREATE BRANCH SQL implementation > -- > > Key: HIVE-27234 > URL: https://issues.apache.org/jira/browse/HIVE-27234 > Project: Hive > Issue Type: Sub-task > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Maybe we can follow spark sql about branch ddl implementation > [https://github.com/apache/iceberg/pull/6617] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[
https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=856328&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-856328
]
ASF GitHub Bot logged work on HIVE-27234:
-
Author: ASF GitHub Bot
Created on: 12/Apr/23 06:06
Start Date: 12/Apr/23 06:06
Worklog Time Spent: 10m
Work Description: zhangbutao opened a new pull request, #4216:
URL: https://github.com/apache/hive/pull/4216
### What changes were proposed in this pull request?
This PR refers to spark-sql about iceberg branch ddl implementation
https://github.com/apache/iceberg/pull/6617
If someone has different opinions about the sql syntax of branch, we can
discuss here.
### Why are the changes needed?
Personally, branch is more useful than snapshot in iceberg, and it is more
friendly to users. We can use branch do lots of meaningfull work.
### Does this PR introduce _any_ user-facing change?
Added a new sql syntax and hive users can create iceberg branch using the
sql.
```
ALTER TABLE tableName
{CREATE BRANCH branchName [AS OF VERSION {snapshotId}]
[RETAIN interval {DAYS | HOURS | MINUTES}]
[WITH SNAPSHOT RETENTION {[num_snapshots SNAPSHOTS] [interval {DAYS | HOURS
| MINUTES}]}]}]
```
### How was this patch tested?
UT
Issue Time Tracking
---
Worklog Id: (was: 856328)
Time Spent: 40m (was: 0.5h)
> Iceberg: CREATE BRANCH SQL implementation
> --
>
> Key: HIVE-27234
> URL: https://issues.apache.org/jira/browse/HIVE-27234
> Project: Hive
> Issue Type: Sub-task
> Components: Iceberg integration
>Reporter: zhangbutao
>Assignee: zhangbutao
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Maybe we can follow spark sql about branch ddl implementation
> [https://github.com/apache/iceberg/pull/6617]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[ https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=856122&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-856122 ] ASF GitHub Bot logged work on HIVE-27234: - Author: ASF GitHub Bot Created on: 11/Apr/23 13:18 Start Date: 11/Apr/23 13:18 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4216: URL: https://github.com/apache/hive/pull/4216#issuecomment-1503338426 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4216) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [2 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4216&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [8 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4216&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4216&metric=coverage&view=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4216&metric=duplicated_lines_density&view=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 856122) Time Spent: 20m (was: 10m) > Iceberg: CREATE BRANCH SQL implementation > -- > > Key: HIVE-27234 > URL: https://issues.apache.org/jira/browse/HIVE-27234 > Project: Hive > Issue Type: Sub-task > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Maybe we can follow spark sql about branch ddl implementation > [https://github.com/apache/iceberg/pull/6617] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27234) Iceberg: CREATE BRANCH SQL implementation
[ https://issues.apache.org/jira/browse/HIVE-27234?focusedWorklogId=856042&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-856042 ] ASF GitHub Bot logged work on HIVE-27234: - Author: ASF GitHub Bot Created on: 11/Apr/23 09:02 Start Date: 11/Apr/23 09:02 Worklog Time Spent: 10m Work Description: zhangbutao opened a new pull request, #4216: URL: https://github.com/apache/hive/pull/4216 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? Issue Time Tracking --- Worklog Id: (was: 856042) Remaining Estimate: 0h Time Spent: 10m > Iceberg: CREATE BRANCH SQL implementation > -- > > Key: HIVE-27234 > URL: https://issues.apache.org/jira/browse/HIVE-27234 > Project: Hive > Issue Type: Sub-task > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Maybe we can follow spark sql about branch ddl implementation > [https://github.com/apache/iceberg/pull/6617] -- This message was sent by Atlassian Jira (v8.20.10#820010)
