[
https://issues.apache.org/jira/browse/GOBBLIN-2049?focusedWorklogId=915421&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-915421
]
ASF GitHub Bot logged work on GOBBLIN-2049:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 18/Apr/24 23:21
Start Date: 18/Apr/24 23:21
Worklog Time Spent: 10m
Work Description: phet commented on code in PR #3929:
URL: https://github.com/apache/gobblin/pull/3929#discussion_r1571501261
##########
gobblin-data-management/src/main/java/org/apache/gobblin/data/management/copy/writer/FileAwareInputStreamDataWriter.java:
##########
@@ -90,6 +90,8 @@ public class FileAwareInputStreamDataWriter extends
InstrumentedDataWriter<FileA
public static final boolean DEFAULT_GOBBLIN_COPY_CHECK_FILESIZE = false;
public static final String GOBBLIN_COPY_TASK_OVERWRITE_ON_COMMIT =
"gobblin.copy.task.overwrite.on.commit";
public static final boolean DEFAULT_GOBBLIN_COPY_TASK_OVERWRITE_ON_COMMIT =
false;
+ public static final String GOBBLIN_COPY_REQUIRE_PERMISSION_SET_FOR_SUCCESS =
"gobblin.copy.requirePermissionSetForSuccess";
+ public static final boolean DEFAULT_COPY_REQUIRE_PERMISSION_SET_FOR_SUCCESS
= false;
Review Comment:
when wouldn't one want failure? I'd be more aggressive in choosing the
default. while it does require a config change, one can always easily revert
to the legacy semantics.
so while not b/w-compat, quite easily opted-into, as desired
Issue Time Tracking
-------------------
Worklog Id: (was: 915421)
Time Spent: 40m (was: 0.5h)
> Configure Gobblin Distcp Writer to fail if setPermission fails
> --------------------------------------------------------------
>
> Key: GOBBLIN-2049
> URL: https://issues.apache.org/jira/browse/GOBBLIN-2049
> Project: Apache Gobblin
> Issue Type: New Feature
> Components: gobblin-service
> Reporter: Urmi Mustafi
> Assignee: Abhishek Tiwari
> Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Gobblin {{safeSetPathPermission}} does not throw an exception when setting
> permissions fail on a path. We want to change this behavior especially for
> use cases like manifest distcp where hundreds of thousands of files are
> involved in a distcp job and permission settings are important to replicate
> correctly as they cannot be updated or verified by hand.
> This PR adds a new configuration to fail writing or publishing tasks when the
> job is configured to report success when permissions are not replicated
> properly. The default behavior remains the same as before to allow the job to
> succeed without this for backwards compatibility.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
