[
https://issues.apache.org/jira/browse/HIVE-25057?focusedWorklogId=589757&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-589757
]
ASF GitHub Bot logged work on HIVE-25057:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 27/Apr/21 13:58
Start Date: 27/Apr/21 13:58
Worklog Time Spent: 10m
Work Description: marton-bod commented on a change in pull request #2219:
URL: https://github.com/apache/hive/pull/2219#discussion_r621235567
##########
File path:
iceberg/iceberg-handler/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerWithEngine.java
##########
@@ -1158,32 +1172,39 @@ private void validateBasicStats(Table icebergTable,
String dbName, String tableN
}
private void validateMigration(String tableName) throws TException,
InterruptedException {
- List<Object[]> originalResult = shell.executeStatement("SELECT * FROM " +
tableName);
+ List<Object[]> originalResult = shell.executeStatement("SELECT * FROM " +
tableName + " ORDER BY a");
shell.executeStatement("ALTER TABLE " + tableName + " SET TBLPROPERTIES " +
"('storage_handler'='org.apache.iceberg.mr.hive.HiveIcebergStorageHandler')");
- List<Object[]> alterResult = shell.executeStatement("SELECT * FROM " +
tableName);
+ List<Object[]> alterResult = shell.executeStatement("SELECT * FROM " +
tableName + " ORDER BY a");
Assert.assertEquals(originalResult.size(), alterResult.size());
for (int i = 0; i < originalResult.size(); i++) {
- Arrays.equals(originalResult.get(i), alterResult.get(i));
+ Assert.assertTrue(Arrays.equals(originalResult.get(i),
alterResult.get(i)));
}
validateSd(tableName, "iceberg");
}
private void validateMigrationRollback(String tableName) throws TException,
InterruptedException {
- List<Object[]> originalResult = shell.executeStatement("SELECT * FROM " +
tableName);
+ List<Object[]> originalResult = shell.executeStatement("SELECT * FROM " +
tableName + " ORDER BY a");
try (MockedStatic<HiveTableUtil> mockedTableUtil =
Mockito.mockStatic(HiveTableUtil.class)) {
mockedTableUtil.when(() ->
HiveTableUtil.importFiles(ArgumentMatchers.anyString(),
ArgumentMatchers.anyString(),
ArgumentMatchers.any(PartitionSpecProxy.class),
ArgumentMatchers.anyList(),
ArgumentMatchers.any(Properties.class),
ArgumentMatchers.any(Configuration.class)))
.thenThrow(new MetaException());
- shell.executeStatement("ALTER TABLE " + tableName + " SET TBLPROPERTIES
" +
-
"('storage_handler'='org.apache.iceberg.mr.hive.HiveIcebergStorageHandler')");
- List<Object[]> alterResult = shell.executeStatement("SELECT * FROM " +
tableName);
- Assert.assertEquals(originalResult.size(), alterResult.size());
- for (int i = 0; i < originalResult.size(); i++) {
- Arrays.equals(originalResult.get(i), alterResult.get(i));
+ try {
+ shell.executeStatement("ALTER TABLE " + tableName + " SET
TBLPROPERTIES " +
+
"('storage_handler'='org.apache.iceberg.mr.hive.HiveIcebergStorageHandler')");
+ } catch (IllegalArgumentException e) {
+ Assert.assertTrue(e.getMessage().contains("Error occurred during hive
table migration to iceberg."));
+ shell.executeStatement("MSCK REPAIR TABLE " + tableName);
+ List<Object[]> alterResult = shell.executeStatement("SELECT * FROM " +
tableName + " ORDER BY a");
+ Assert.assertEquals(originalResult.size(), alterResult.size());
+ for (int i = 0; i < originalResult.size(); i++) {
+ Assert.assertTrue(Arrays.equals(originalResult.get(i),
alterResult.get(i)));
+ }
+ validateSd(tableName, fileFormat.name());
Review comment:
nit: I think this sd validation check might belong logically before the
msck repair command. In case the sd wasn't reverted correctly, we'd get a
failure during the above select which could be harder to figure out why
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 589757)
Time Spent: 6.5h (was: 6h 20m)
> Implement rollback for hive to iceberg migration
> ------------------------------------------------
>
> Key: HIVE-25057
> URL: https://issues.apache.org/jira/browse/HIVE-25057
> Project: Hive
> Issue Type: Improvement
> Reporter: László Pintér
> Assignee: László Pintér
> Priority: Major
> Labels: pull-request-available
> Time Spent: 6.5h
> Remaining Estimate: 0h
>
> This is a follow-up Jira of HIVE-25008.
> In case of an error during hive to iceberg migration, the original hive table
> must be restored.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
