[
https://issues.apache.org/jira/browse/GOBBLIN-2157?focusedWorklogId=934804&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-934804
]
ASF GitHub Bot logged work on GOBBLIN-2157:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 13/Sep/24 23:23
Start Date: 13/Sep/24 23:23
Worklog Time Spent: 10m
Work Description: phet commented on code in PR #4056:
URL: https://github.com/apache/gobblin/pull/4056#discussion_r1759607239
##########
gobblin-data-management/src/main/java/org/apache/gobblin/data/management/copy/iceberg/IcebergTable.java:
##########
@@ -213,7 +215,10 @@ public DatasetDescriptor getDatasetDescriptor(FileSystem
fs) {
protected void registerIcebergTable(TableMetadata srcMetadata, TableMetadata
dstMetadata) {
if (dstMetadata != null) {
// use current destination metadata as 'base metadata' and source as
'updated metadata' while committing
- this.tableOps.commit(dstMetadata,
srcMetadata.replaceProperties(dstMetadata.properties()));
+ Map<String, String> combinedMetadataProperties = Maps.newHashMap();
+ combinedMetadataProperties.putAll(dstMetadata.properties());
+ combinedMetadataProperties.putAll(srcMetadata.properties());
Review Comment:
please add a comment that captures the reasoning behind overshadowing dest
with src
Issue Time Tracking
-------------------
Worklog Id: (was: 934804)
Time Spent: 20m (was: 10m)
> Copy Iceberg table properties during iceberg table replication
> --------------------------------------------------------------
>
> Key: GOBBLIN-2157
> URL: https://issues.apache.org/jira/browse/GOBBLIN-2157
> Project: Apache Gobblin
> Issue Type: Improvement
> Components: gobblin-core
> Reporter: William Lo
> Assignee: Abhishek Tiwari
> Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Currently, Iceberg table copies do not replicate any properties from the
> source table. This can be unintuitive as the expectation for most pipelines
> is that the full table data and properties are replicated on both the source
> and destination.
> We want to copy all the properties from the source table while maintaining
> any pre-existing properties that were created from the destination table.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
