lawofcycles commented on code in PR #10276:
URL: https://github.com/apache/iceberg/pull/10276#discussion_r1666063693
##########
aws/src/integration/java/org/apache/iceberg/aws/glue/TestGlueCatalogTable.java:
##########
@@ -221,6 +221,69 @@ public void testUpdateTable() {
assertThat(response.table().description()).isEqualTo(updatedComment);
}
+ @Test
+ public void testDropColumn() {
+ String namespace = createNamespace();
+ String tableName = createTable(namespace);
+ Table table = glueCatalog.loadTable(TableIdentifier.of(namespace,
tableName));
+ table
+ .updateSchema()
+ .addColumn("c2", Types.StringType.get(), "updated from Iceberg API")
+ .addColumn("c3", Types.StringType.get())
+ .commit();
+
+ updateTableColumns(
+ namespace,
+ tableName,
+ column -> {
+ if (column.name().equals("c3")) {
+ return column.toBuilder().comment("updated from Glue API").build();
+ } else {
+ return column;
+ }
+ });
+
+ table.updateSchema().deleteColumn("c2").deleteColumn("c3").commit();
Review Comment:
@rahil-c
> Or is this commit only changing the iceberg metadata in S3
Yes.
On current Iceberg's Glue integration, Glue Data Catalog holds all the
column info that are registered in Iceberg Table even if these are deleted.
IcebergToGlueConverter always try to sync all the historical column to Glue
Data Catalog every time schema is updated.
For example, following scenario is expected behaviour.
1. Create Table with column x, y, z with Iceberg DDL using Glue Data Catalog
as Iceberg Catalog
2. drop column y with Iceberg DDL -> column y remain on Glue Data Catalog
3. drop column y, z from Glue Data Catalog API
4. add column y2 with Iceberg DDL -> column y, z appeared again on Glue
Data Catalog in addition to column y2
I think those Information will be used to refer the historical columns on
AWS Console when user query tables from such as Athena.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
