[
https://issues.apache.org/jira/browse/HDDS-2939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17025842#comment-17025842
]
Yiqun Lin commented on HDDS-2939:
---------------------------------
Hi [~sdeka], I am reading for this design doc, some comments from me:
For the Filesystem Namespace Operations, the ls(list files/folders) operation
will also a common operation. But under current implementation, for example,
list a directory, we have to traverse the whole directory/file table to lookup
the child file/sub-folders.This is an ineffective way. Do we have any other
improvement for this? Can we additionally store the child ID for each record in
directory table? That can help us quickly find the child file or child folder.
{quote}
Associating a lock with each parent prefix being accessed by an operation in
the OM, is sufficient to control concurrent operations on the same prefix. When
the OM starts to process create “/a/b/c/1.txt”, a prefix lock is taken for
“/a/b/c”...
{quote}
For the concurrency control, we create the lock for each parent prefix level.
There will be large number of lock instances to be maintained in OM memory once
there are millions of directory folders. Current way is so fine-grained lock
way, have we considered about the partition namespace way? Divided the whole
namespace into logic sub-namespaces by the prefix key. Then each sub-namespace
will have its lock. This is a compromise approach than just having a global
exclusive lock or having uncontrollable number of locks that depended on parent
prefix's number.
Is there a future plan to have a way(API or command Tool) to convert object key
to Ozone FS namespace? Because object store is now the major use case for the
users. Maybe users want to use a filesystem way to access the data without
moving their data.
> Ozone FS namespace
> ------------------
>
> Key: HDDS-2939
> URL: https://issues.apache.org/jira/browse/HDDS-2939
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Components: Ozone Manager
> Reporter: Supratim Deka
> Assignee: Supratim Deka
> Priority: Major
> Attachments: Ozone FS Namespace Proposal v1.0.docx
>
>
> Create the structures and metadata layout required to support efficient FS
> namespace operations in Ozone - operations involving folders/directories
> required to support the Hadoop compatible Filesystem interface.
> The details are described in the attached document. The work is divided up
> into sub-tasks as per the task list in the document.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
