[
https://issues.apache.org/jira/browse/GOBBLIN-2128?focusedWorklogId=929619&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-929619
]
ASF GitHub Bot logged work on GOBBLIN-2128:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 09/Aug/24 21:36
Start Date: 09/Aug/24 21:36
Worklog Time Spent: 10m
Work Description: Will-Lo opened a new pull request, #4020:
URL: https://github.com/apache/gobblin/pull/4020
… with equal file depths
Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I
have checked off all the steps below!
### JIRA
- [ ] My PR addresses the following [Gobblin
JIRA](https://issues.apache.org/jira/browse/GOBBLIN/) issues and references
them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"
- https://issues.apache.org/jira/browse/GOBBLIN-2128
### Description
- [ ] Here are some details about my PR, including screenshots (if
applicable):
`SetPermissionCommitStep` has a bug where it does not handle paths with the
same file depth appropriately.
If directories of the same nested path have different permissions, they can
lead to unknown behavior as the key equality is incorrect here and doesn't
maintain both separate paths/permissions.
The key comparator:
```
Long.compare(o1.chars().filter(ch -> ch == '/').count(),
o2.chars().filter(ch -> ch == '/').count())
```
should be replaced with:
```
Comparator.comparingInt((String o) -> o.split("/").length).thenComparing(o
-> o)
```
Additionally, build the paths and permissions from the existing map instead
of re-reading from FS to reduce FS calls and ensure consistency.
Add a setting to disable this permission step with the configuration
```
gobblin.copy.manifestBased.enableSetPermissionPostPublish
```
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
Unit tests.
### Commits
- [ ] My commits all reference JIRA issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
2. Subject is limited to 50 characters
3. Subject does not end with a period
4. Subject uses the imperative mood ("add", not "adding")
5. Body wraps at 72 characters
6. Body explains "what" and "why", not "how"
Issue Time Tracking
-------------------
Worklog Id: (was: 929619)
Remaining Estimate: 0h
Time Spent: 10m
> Set permission in Manifest distcp does not set all paths correctly
> ------------------------------------------------------------------
>
> Key: GOBBLIN-2128
> URL: https://issues.apache.org/jira/browse/GOBBLIN-2128
> Project: Apache Gobblin
> Issue Type: Bug
> Reporter: William Lo
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> There's a bug in the treemap for `SetPermissionCommitStep` where paths of the
> equal file depth can be misassigned permissions due to an error in the
> Treemap initialization:
> {code:java}
> Long.compare(o1.chars().filter(ch -> ch == '/').count(), o2.chars().filter(ch
> -> ch == '/').count()) {code}
> Treats any key with the same depth as equal, this could lead to incorrect
> behavior when performing map operations in this map.
> We want to rearrange this so that the comparator also compares the source
> string.
> Additionally, build off the map from the same file permissions as the
> ancestor tree in copyable file so that it can stay consistent permission
> reading and reduce FS calls.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
