wombatu-kun commented on code in PR #19426:
URL: https://github.com/apache/hudi/pull/19426#discussion_r3688293186
##########
hudi-sync/hudi-hive-sync/src/test/java/org/apache/hudi/hive/TestHiveSyncTool.java:
##########
@@ -271,7 +272,40 @@ public Option<String>
getMetastoreLocation(IMetaStoreClient client, String datab
throw new HoodieHiveSyncException("Failed to get the metastore location
from the table " + tableName, e);
}
}
-
+
+ @ParameterizedTest
+ @MethodSource("syncMode")
+ void testForceRecreateTable(String syncMode) throws Exception {
+ hiveSyncProps.setProperty(HIVE_SYNC_MODE.key(), syncMode);
+ String instantTime = "100";
+ HiveTestUtil.createCOWTable(instantTime, 1, true);
+ reInitHiveSyncClient();
+ reSyncHiveTable();
+ assertTrue(hiveClient.tableExists(HiveTestUtil.TABLE_NAME));
+
+ // simulate manual drift on the metastore table that a fully-synced
incremental sync would never touch
+ ddlExecutor.runSQL("ALTER TABLE `" + HiveTestUtil.TABLE_NAME + "` SET
TBLPROPERTIES ('drift_marker'='true')");
+ IMetaStoreClient client = IMetaStoreClientUtil.getMSC(getHiveConf());
+ assertTrue(client.getTable(HiveTestUtil.DB_NAME,
HiveTestUtil.TABLE_NAME).getParameters().containsKey("drift_marker"));
+
+ // nothing changed on the Hoodie timeline, so a normal sync is a no-op and
leaves the drift behind
+ reInitHiveSyncClient();
+ reSyncHiveTable();
+ client.reconnect();
+ assertTrue(client.getTable(HiveTestUtil.DB_NAME,
HiveTestUtil.TABLE_NAME).getParameters().containsKey("drift_marker"),
+ "a no-op incremental sync must not touch the table");
+
+ // forcing recreation drops and rebuilds the table from scratch, even
though nothing else changed
+ hiveSyncProps.setProperty(META_SYNC_FORCE_RECREATE_TABLE.key(), "true");
+ reInitHiveSyncClient();
+ reSyncHiveTable();
+ client.reconnect();
+ assertFalse(client.getTable(HiveTestUtil.DB_NAME,
HiveTestUtil.TABLE_NAME).getParameters().containsKey("drift_marker"),
+ "force-recreate must drop and recreate the table even when the
incremental sync would otherwise be a no-op");
+ assertEquals(instantTime,
hiveClient.getLastCommitTimeSynced(HiveTestUtil.TABLE_NAME).get());
Review Comment:
Nothing in TestHiveSyncTool asserts on hudi_writer_version, so the new
updateHoodieWriterVersion call in recreateAndSyncHiveTable would keep passing
if it were removed. Add an assertion on the HOODIE_WRITER_VERSION table
parameter after the force recreate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
