[ https://issues.apache.org/jira/browse/HDDS-4155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17190989#comment-17190989 ]
Marton Elek commented on HDDS-4155: ----------------------------------- Thanks the answer [~arp]. Sorry, If I used the wrong words in my question ("support") let's talk about behavior instead of how to name it (bug or not). Let's say I have an S3 bucket: {code} aws s3api list-objects --bucket ozonetest --prefix=a/b | jq '.Contents[] | .Key' "a/b/../c" "a/b/../e" "a/b/./c" "a/b/./f" "a/b//c" "a/b//y" "a/b/c" "a/b/c/file1" "a/b/h" "a/b/h/" "a/b/i/" "a/b/x/README.md" {code} Using s3a we can see that some (invalid) path are hidden, others (for example the directory and file with the same name): {code} drwxrwxrwx - elek elek 0 2020-09-05 08:23 s3a://ozonetest/a/b/a -rw-rw-rw- 1 elek elek 3841 2020-08-28 11:18 s3a://ozonetest/a/b/c drwxrwxrwx - elek elek 0 2020-09-05 08:23 s3a://ozonetest/a/b/c -rw-rw-rw- 1 elek elek 3841 2020-09-02 12:34 s3a://ozonetest/a/b/h drwxrwxrwx - elek elek 0 2020-09-05 08:23 s3a://ozonetest/a/b/h drwxrwxrwx - elek elek 0 2020-09-05 08:23 s3a://ozonetest/a/b/i drwxrwxrwx - elek elek 0 2020-09-05 08:23 s3a://ozonetest/a/b/x {code} I don't think it's an "unfixable" bug, but it's a decision about the mapping: to display both file and directories instead of hiding one (or throwing an exception). I understand that HDFS couldn't support it as it's not an object store. But Ozone is an object store, and I asked (without the intention to suggest anything different) **why** do we choose to follow the behavior of HDFS instead of S3A which seems to be close to S3FS. Second question: what is the behavior of some other object store connectors (ADLS, google ....)? In general (as I wrote in the other thread) I think we should consider some level of compatibility between S3A (and other object store connectors) and ofs/o3fs to support seamless move between on-prem and cloud in case of hybrid cloud. > Directory and filename can end up with same name in a path > ---------------------------------------------------------- > > Key: HDDS-4155 > URL: https://issues.apache.org/jira/browse/HDDS-4155 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Reporter: Bharat Viswanadham > Assignee: Bharat Viswanadham > Priority: Major > Labels: pull-request-available > > Scenario: > Create Key via S3, and Create Directory through Fs. > # open key -> /a/b/c > # CreateDirectory -> /a/b/c > # CommitKey -> /a/b/c > So, now in Ozone we will have directory and file with name "c" > When created through Fs interface. > # create file -> /a/b/c > # CreateDirectory -> /a/b/c > # CommitKey -> /a/b/c > So, now in Ozone we will have directory and file with name "c" > > # InitiateMPU /a/b/c > # Create Part1 /a/b/c > # Commit Part1 /a/b/c > # Create Directory /a/b/c > # Complete MPU /a/b/c > So, now in Ozone, we will have directory and file with name "c". In MPU this > is one example scenario. > > Few proposals/ideas to solve this: > # Check during commit whether a directory already exists with same name. But > disadvantage is after user uploads the entire data during last stage we fail. > (File system with create in progress acts similarly. Scenario: 1. vi t1 2. > mkdir t1 3. Save t1: (Fail:"t1" is a directory) > # During create directory check are there any open key creation with same > name and fail. > > Any of the above approaches are not final, this Jira is opened to discuss > this issue and come up with solution. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org