nastra commented on code in PR #7986:
URL: https://github.com/apache/iceberg/pull/7986#discussion_r1252189267
##########
spark/v3.4/spark/src/main/java/org/apache/iceberg/spark/CommitMetadata.java:
##########
@@ -35,13 +40,18 @@ private CommitMetadata() {}
* running the code wrapped as a caller, and any snapshot committed within
the callable object
* will be attached with the metadata defined in properties
*
- * @param properties extra commit metadata to attach to the snapshot
committed within callable
+ * @param properties extra commit metadata to attach to the snapshot
committed within callable.
+ * The prefix will be removed for properties starting with {@link
+ * SnapshotSummary#EXTRA_METADATA_PREFIX}
* @param callable the code to be executed
* @param exClass the expected type of exception which would be thrown from
callable
*/
public static <R, E extends Exception> R withCommitProperties(
Map<String, String> properties, Callable<R> callable, Class<E> exClass)
throws E {
- COMMIT_PROPERTIES.set(properties);
+ Map<String, String> props = Maps.newHashMap();
+ properties.forEach((k, v) -> props.put(REMOVE_PREFIX.apply(k), v));
Review Comment:
fixed
##########
docs/spark-configuration.md:
##########
@@ -189,13 +189,13 @@ df.write
| write-format | Table write.format.default | File format to use for
this write operation; parquet, avro, or orc |
| target-file-size-bytes | As per table property | Overrides this table's
write.target-file-size-bytes |
| check-nullability | true | Sets the nullable
check on fields |
-| snapshot-property._custom-key_ | null | Adds an entry with
custom-key and corresponding value in the snapshot summary |
+| snapshot-property._custom-key_ | null | Adds an entry with
custom-key and corresponding value in the snapshot summary (the
`snapshot-property.` prefix is only required for DSv2 and not for properties
passed through `CommitMetadata`) |
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
