snazy commented on code in PR #8909:
URL: https://github.com/apache/iceberg/pull/8909#discussion_r1402432505
##########
nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java:
##########
@@ -540,4 +634,35 @@ public void close() {
api.close();
}
}
+
+ public void commitView(
+ ViewMetadata base,
+ ViewMetadata metadata,
+ String newMetadataLocation,
+ String contentId,
+ ContentKey key)
+ throws NessieConflictException, NessieNotFoundException {
+
+ long versionId = metadata.currentVersion().versionId();
+ ImmutableIcebergView.Builder newViewBuilder =
ImmutableIcebergView.builder();
+ IcebergView newView =
+ newViewBuilder
+ .id(contentId)
+ .versionId(versionId)
+ .schemaId(metadata.currentSchemaId())
+ .metadataLocation(newMetadataLocation)
+ // Only view metadata location need to be tracked from Nessie.
+ // Other information can be extracted by parsing the view metadata
file.
+ .sqlText("default sql text")
Review Comment:
Better just a `-` instead of a misleading text, same for dialect
##########
core/src/test/java/org/apache/iceberg/view/ViewCatalogTests.java:
##########
@@ -400,8 +400,15 @@ public void
replaceTableViaTransactionThatAlreadyExistsAsView() {
.buildTable(viewIdentifier, SCHEMA)
.replaceTransaction()
.commitTransaction())
- .isInstanceOf(NoSuchTableException.class)
- .hasMessageStartingWith("Table does not exist: ns.view");
+ .satisfiesAnyOf(
Review Comment:
It looks awkward that this test has different outcomes, only because
`TestInMemoryViewCatalog` / `InMemoryCatalog` behaves differently. /cc @nastra
##########
nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java:
##########
@@ -170,11 +184,15 @@ private TableIdentifier
toIdentifier(EntriesResponse.Entry entry) {
return TableIdentifier.of(elements.toArray(new String[elements.size()]));
}
- public IcebergTable table(TableIdentifier tableIdentifier) {
+ public IcebergContent fetchContent(TableIdentifier tableIdentifier) {
try {
ContentKey key = NessieUtil.toKey(tableIdentifier);
- Content table = withReference(api.getContent().key(key)).get().get(key);
- return table != null ? table.unwrap(IcebergTable.class).orElse(null) :
null;
+ Content content =
withReference(api.getContent().key(key)).get().get(key);
+ if (content == null) {
Review Comment:
Nit: why is the ternary replaced w/ more boilerplate code?
##########
nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java:
##########
@@ -378,27 +403,71 @@ public void renameTable(TableIdentifier from,
TableIdentifier to) {
// behavior. So better be safe than sorry.
}
+ private static void validateToContentForRename(
+ TableIdentifier from, TableIdentifier to, IcebergContent
existingToContent) {
+ if (existingToContent != null) {
+ if (existingToContent.getType() == Content.Type.ICEBERG_VIEW) {
+ throw new AlreadyExistsException("Cannot rename %s to %s. View already
exists", from, to);
+ } else if (existingToContent.getType() == Content.Type.ICEBERG_TABLE) {
+ throw new AlreadyExistsException("Cannot rename %s to %s. Table
already exists", from, to);
+ } else {
+ throw new AlreadyExistsException(
+ "Cannot rename %s to %s. Another content with same name already
exists", from, to);
+ }
+ }
+ }
+
+ private static void validateFromContentForRename(
+ TableIdentifier from, Content.Type type, IcebergContent
existingFromContent) {
+ if (existingFromContent == null) {
+ if (type == Content.Type.ICEBERG_VIEW) {
+ throw new NoSuchViewException("View does not exist: %s", from);
+ } else if (type == Content.Type.ICEBERG_TABLE) {
+ throw new NoSuchTableException("Table does not exist: %s", from);
+ } else {
+ throw new UnsupportedOperationException("Cannot perform rename for
content type: " + type);
+ }
+ } else if (existingFromContent.getType() != type) {
+ throw new UnsupportedOperationException(
+ String.format("content type of from identifier %s should be of %s",
from, type));
+ }
+ }
+
public boolean dropTable(TableIdentifier identifier, boolean purge) {
+ return dropContent(identifier, purge, Content.Type.ICEBERG_TABLE);
+ }
+
+ public boolean dropView(TableIdentifier identifier, boolean purge) {
+ return dropContent(identifier, purge, Content.Type.ICEBERG_VIEW);
+ }
+
+ private boolean dropContent(TableIdentifier identifier, boolean purge,
Content.Type type) {
getRef().checkMutable();
- IcebergTable existingTable = table(identifier);
- if (existingTable == null) {
+ IcebergContent existingContent = fetchContent(identifier);
+
+ if (existingContent == null || existingContent.getType() != type) {
Review Comment:
`false` is bubbled up to the user as a "table/view does not exist" error -
other places in the code base throw an error saying that the content exists,
but is not of the expected type.
This looks inconsistent to me.
/cc @nastra
##########
nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java:
##########
@@ -136,15 +142,23 @@ private UpdateableReference loadReference(String
requestedRef, String hash) {
}
public List<TableIdentifier> listTables(Namespace namespace) {
+ return listContents(namespace, Content.Type.ICEBERG_TABLE);
+ }
+
+ public List<TableIdentifier> listViews(Namespace namespace) {
+ return listContents(namespace, Content.Type.ICEBERG_VIEW);
+ }
+
+ private List<TableIdentifier> listContents(Namespace namespace, Content.Type
type) {
try {
return withReference(api.getEntries()).get().getEntries().stream()
.filter(namespacePredicate(namespace))
- .filter(e -> Content.Type.ICEBERG_TABLE == e.getType())
+ .filter(e -> type == e.getType())
.map(this::toIdentifier)
.collect(Collectors.toList());
} catch (NessieNotFoundException ex) {
throw new NoSuchNamespaceException(
- ex, "Unable to list tables due to missing ref '%s'",
getRef().getName());
+ ex, "Unable to list %s due to missing ref '%s'", type,
getRef().getName());
Review Comment:
Should be a human readable type (not ICEBERG_VIEW/TABLE)
##########
core/src/test/java/org/apache/iceberg/view/ViewCatalogTests.java:
##########
@@ -465,8 +472,15 @@ public void replaceViewThatAlreadyExistsAsTable() {
.withDefaultNamespace(tableIdentifier.namespace())
.withQuery("spark", "select * from ns.tbl")
.replace())
- .isInstanceOf(NoSuchViewException.class)
- .hasMessageStartingWith("View does not exist: ns.table");
+ .satisfiesAnyOf(
Review Comment:
It looks awkward that this test has different outcomes, only because
`TestInMemoryViewCatalog` / `InMemoryCatalog` behaves differently. /cc @nastra
##########
nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java:
##########
@@ -530,6 +615,15 @@ private String buildCommitMsg(TableMetadata base,
TableMetadata metadata, String
return String.format("Iceberg commit against %s", tableName);
Review Comment:
This should now be `Iceberg commit against table %s` - similar for the
other commit messages in this function
##########
nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java:
##########
@@ -378,27 +403,71 @@ public void renameTable(TableIdentifier from,
TableIdentifier to) {
// behavior. So better be safe than sorry.
}
+ private static void validateToContentForRename(
+ TableIdentifier from, TableIdentifier to, IcebergContent
existingToContent) {
+ if (existingToContent != null) {
+ if (existingToContent.getType() == Content.Type.ICEBERG_VIEW) {
+ throw new AlreadyExistsException("Cannot rename %s to %s. View already
exists", from, to);
+ } else if (existingToContent.getType() == Content.Type.ICEBERG_TABLE) {
+ throw new AlreadyExistsException("Cannot rename %s to %s. Table
already exists", from, to);
+ } else {
+ throw new AlreadyExistsException(
+ "Cannot rename %s to %s. Another content with same name already
exists", from, to);
+ }
+ }
+ }
+
+ private static void validateFromContentForRename(
+ TableIdentifier from, Content.Type type, IcebergContent
existingFromContent) {
+ if (existingFromContent == null) {
+ if (type == Content.Type.ICEBERG_VIEW) {
+ throw new NoSuchViewException("View does not exist: %s", from);
+ } else if (type == Content.Type.ICEBERG_TABLE) {
+ throw new NoSuchTableException("Table does not exist: %s", from);
+ } else {
+ throw new UnsupportedOperationException("Cannot perform rename for
content type: " + type);
+ }
+ } else if (existingFromContent.getType() != type) {
+ throw new UnsupportedOperationException(
Review Comment:
Looks like this `UnsupportedOperationException` is propagated to users when
trying to rename a view using `ALTER TABLE ... RENAME` - "unsupported" sounds
weird in that case.
##########
nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java:
##########
@@ -540,4 +634,35 @@ public void close() {
api.close();
}
}
+
+ public void commitView(
Review Comment:
Nit: this function should be next to `commtContent`
##########
nessie/src/main/java/org/apache/iceberg/nessie/NessieUtil.java:
##########
@@ -165,4 +180,95 @@ public static TableMetadata
updateTableMetadataWithNessieSpecificProperties(
return builder.discardChanges().build();
}
+
+ public static ViewMetadata loadViewMetadata(
+ ViewMetadata metadata, String metadataLocation, Reference reference) {
+ Map<String, String> newProperties = Maps.newHashMap(metadata.properties());
+ newProperties.put(NessieTableOperations.NESSIE_COMMIT_ID_PROPERTY,
reference.getHash());
+
+ return ViewMetadata.buildFrom(
+
ViewMetadata.buildFrom(metadata).setProperties(newProperties).build())
+ .setMetadataLocation(metadataLocation)
+ .build();
+ }
+
+ static void handleExceptionsForCommits(Exception exception, String refName,
Content.Type type) {
+ if (exception instanceof NessieConflictException) {
+ if (exception instanceof NessieReferenceConflictException) {
+ // Throws a specialized exception, if possible
+ NessieUtil.maybeThrowSpecializedException(
+ (NessieReferenceConflictException) exception, type);
+ }
+
+ throw new CommitFailedException(
+ exception,
+ "Cannot commit: Reference hash is out of date. Update the reference
'%s' and try again",
+ refName);
+ }
+
+ if (exception instanceof NessieNotFoundException) {
+ throw new RuntimeException(
+ String.format("Cannot commit: Reference '%s' no longer exists",
refName), exception);
+ }
+
+ if (exception instanceof HttpClientException) {
+ // Intentionally catch all nessie-client-exceptions here and not just
the "timeout" variant
+ // to catch all kinds of network errors (e.g. connection reset). Network
code implementation
+ // details and all kinds of network devices can induce unexpected
behavior. So better be
+ // safe than sorry.
+ throw new CommitStateUnknownException(exception);
+ }
+ }
+
+ private static void maybeThrowSpecializedException(
+ NessieReferenceConflictException ex, Content.Type type) {
+ // Check if the server returned 'ReferenceConflicts' information
+ ReferenceConflicts referenceConflicts = ex.getErrorDetails();
+ if (referenceConflicts == null) {
+ return;
+ }
+
+ // Can only narrow down to a single exception, if there is only one
conflict.
+ List<Conflict> conflicts = referenceConflicts.conflicts();
+ if (conflicts.size() != 1) {
+ return;
+ }
+
+ String contentType = contentTypeString(type);
+
+ Conflict conflict = conflicts.get(0);
+ Conflict.ConflictType conflictType = conflict.conflictType();
+ if (conflictType != null) {
+ switch (conflictType) {
+ case NAMESPACE_ABSENT:
+ throw new NoSuchNamespaceException(ex, "Namespace does not exist:
%s", conflict.key());
+ case NAMESPACE_NOT_EMPTY:
+ throw new NamespaceNotEmptyException(ex, "Namespace not empty: %s",
conflict.key());
+ case KEY_DOES_NOT_EXIST:
+ if (type == Content.Type.ICEBERG_VIEW) {
+ throw new NoSuchViewException(ex, "%s does not exist: %s",
contentType, conflict.key());
+ } else {
+ throw new NoSuchTableException(
+ ex, "%s does not exist: %s", contentType, conflict.key());
+ }
+ case KEY_EXISTS:
+ throw new AlreadyExistsException(
+ ex, "%s already exists: %s", contentType, conflict.key());
+ default:
+ // Explicit fall-through
+ break;
+ }
+ }
+ }
+
+ static String contentTypeString(Content.Type type) {
+ if (type.equals(Content.Type.ICEBERG_VIEW)) {
+ return "View";
+ } else if (type.equals(Content.Type.ICEBERG_TABLE)) {
+ return "Table";
+ } else if (type.equals(Content.Type.NAMESPACE)) {
+ return "Namespace";
+ }
+ throw new UnsupportedOperationException("Unknown Iceberg content type " +
type.name());
Review Comment:
It's not an _Iceberg_ content type, no?
Should read `Unsupported Nessie content type` and be an
IllegalArgumentException instead
##########
nessie/src/main/java/org/apache/iceberg/nessie/NessieUtil.java:
##########
@@ -165,4 +180,95 @@ public static TableMetadata
updateTableMetadataWithNessieSpecificProperties(
return builder.discardChanges().build();
}
+
+ public static ViewMetadata loadViewMetadata(
Review Comment:
This function should be in `NessieViewOperations` - it's only used there
##########
nessie/src/test/java/org/apache/iceberg/nessie/TestNessieTable.java:
##########
@@ -309,7 +305,8 @@ public void testRenameWithTableReferenceInvalidCase()
throws NessieNotFoundExcep
Assertions.assertThatThrownBy(() -> catalog.renameTable(fromIdentifier,
toIdentifier))
.isInstanceOf(IllegalArgumentException.class)
- .hasMessage("from: Something and to: iceberg-table-test reference name
must be same");
+ .hasMessage(
+ "Cannot rename ICEBERG_TABLE 'tbl' on reference 'Something' to
'rename_table_name' on reference 'iceberg-table-test': source and target
references must be the same.");
Review Comment:
Shouldn't this read `Cannot rename table ...`? Same for the message below.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
