[
https://issues.apache.org/jira/browse/HDDS-17697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chi-Hsuan Huang updated HDDS-17697:
-----------------------------------
Description:
Requests that write a file in an FSO bucket \(or a LEGACY bucket with
{{ozone.om.enable.filesystem.paths}}\) call
{{OMFileRequest.verifyDirectoryKeysInPath}} or {{verifyFilesInPath}}, and then
each request decides on its own which {{OMDirectoryResult}} values are errors.
The decision is copied into several places, and the copies have drifted:
||Request||Target is a directory||A parent is a file||Target is a file||
|{{OMFileCreateRequest}} \(also used by
WithFSO\)|NOT\_A\_FILE|NOT\_A\_FILE|FILE\_ALREADY\_EXISTS unless overwrite|
|{{OMKeyCreateRequest}} \(LEGACY \+ fs
paths\)|NOT\_A\_FILE|NOT\_A\_FILE|allowed|
|{{OMKeyCreateRequestWithFSO}}|NOT\_A\_FILE|NOT\_A\_FILE|allowed|
|{{S3InitiateMultipartUploadRequestWithFSO}}|NOT\_A\_FILE|NOT\_A\_FILE
\(HDDS\-17644\)|allowed|
|{{S3InitiateMultipartUploadRequest}} \(LEGACY \+ fs paths\)|not checked|not
checked|allowed|
|{{S3MultipartUploadCompleteRequestWithFSO}}|NOT\_A\_FILE|NOT\_A\_FILE
\(HDDS\-17696\)|allowed|
|{{S3MultipartUploadCompleteRequest}} \(LEGACY \+ fs paths\)|NOT\_A\_FILE|not
checked|allowed|
HDDS\-17644 and HDDS\-17696 were both caused by a copy that missed the "a
parent is a file" case. The exception messages also differ between copies for
the same result.
LEGACY {{S3InitiateMultipartUploadRequest}} calls {{verifyFilesInPath}} but
only uses the result for ACLs. LEGACY {{S3MultipartUploadCompleteRequest}} does
not use {{verifyFilesInPath}}; it only looks up {{key/}} in the key table.
_Proposal_
{{OMFileCreateRequest.checkDirectoryResult(keyName, isOverWrite, result)}}
already covers all three columns. Move it to a static method in
{{OMFileRequest}} and make the FSO and LEGACY create requests above call it,
with overwrite allowed for key create and MPU.
* Keep this a pure refactor: no change in result codes or in the order of
checks in each request. For example, {{OMKeyCreateRequestWithFSO}} runs the
atomic rewrite and ETag checks before the path check, so each call site
replaces its own if/else in place.
* {{OMDirectoryCreateRequest}} has different rules \(a file in the path is
{{FILE_ALREADY_EXISTS}}\) and stays as is.
* Exception messages may be unified. Result codes must not change, since
clients map them to exceptions.
* Key commit \({{OMKeyCommitRequest}}, and {{OMFileRequest.getParentID}} for
FSO\) has its own checks with a different shape and is out of scope.
* The remaining gaps in LEGACY buckets, {{S3InitiateMultipartUploadRequest}}
not checking the path and {{S3MultipartUploadCompleteRequest}} not checking for
a file in the parent path, are behavior changes and should be a separate fix.
They change what the OM apply path writes, so they may need a version check
once ZDU is supported.
was:
Requests that write a file in an FSO bucket \(or a LEGACY bucket with
{{ozone.om.enable.filesystem.paths}}\) call
{{OMFileRequest.verifyDirectoryKeysInPath}} or {{verifyFilesInPath}}, and then
each request decides on its own which {{OMDirectoryResult}} values are errors.
The decision is copied into several places, and the copies have drifted:
||Request||Target is a directory||A parent is a file||Target is a file||
|{{OMFileCreateRequest}} \(also used by
WithFSO\)|NOT\_A\_FILE|NOT\_A\_FILE|FILE\_ALREADY\_EXISTS unless overwrite|
|{{OMKeyCreateRequest}} \(LEGACY \+ fs
paths\)|NOT\_A\_FILE|NOT\_A\_FILE|allowed|
|{{OMKeyCreateRequestWithFSO}}|NOT\_A\_FILE|NOT\_A\_FILE|allowed|
|{{S3InitiateMultipartUploadRequestWithFSO}}|NOT\_A\_FILE|not checked \(fixed
by HDDS\-17644\)|allowed|
|{{S3InitiateMultipartUploadRequest}} \(LEGACY \+ fs paths\)|not checked|not
checked|allowed|
|{{S3MultipartUploadCompleteRequestWithFSO}}|NOT\_A\_FILE|not checked
\(HDDS\-17696\)|allowed|
HDDS\-17644 and HDDS\-17696 are both caused by a copy that missed the "a parent
is a file" case. The exception messages also differ between copies for the same
result.
_Proposal_
Add one shared check for "this path can be written as a file", for example a
static method in {{OMFileRequest}} that takes the key name, the
{{OMDirectoryResult}} and whether overwriting an existing file is allowed, and
make the requests above call it.
* Keep this a pure refactor: no change in result codes or in the order of
checks in each request. {{OMDirectoryCreateRequest}} has different rules \(a
file in the path is {{FILE\_ALREADY\_EXISTS}}\) and stays as is.
* Exception messages may be unified. Result codes must not change, since
clients map them to exceptions.
* Do this after HDDS\-17644 and HDDS\-17696 are merged, so the shared check
starts from the fixed behavior.
* The remaining gap, {{S3InitiateMultipartUploadRequest}} for LEGACY buckets
not checking the path at all, is a behavior change and should be a separate
fix. It changes what the OM apply path writes, so it may need a version check
once ZDU is supported.
> Share one path check across OM requests that write a file
> ---------------------------------------------------------
>
> Key: HDDS-17697
> URL: https://issues.apache.org/jira/browse/HDDS-17697
> Project: Apache Ozone
> Issue Type: Improvement
> Components: OM
> Reporter: Chi-Hsuan Huang
> Assignee: Chi-Hsuan Huang
> Priority: Minor
>
> Requests that write a file in an FSO bucket \(or a LEGACY bucket with
> {{ozone.om.enable.filesystem.paths}}\) call
> {{OMFileRequest.verifyDirectoryKeysInPath}} or {{verifyFilesInPath}}, and
> then each request decides on its own which {{OMDirectoryResult}} values are
> errors. The decision is copied into several places, and the copies have
> drifted:
> ||Request||Target is a directory||A parent is a file||Target is a file||
> |{{OMFileCreateRequest}} \(also used by
> WithFSO\)|NOT\_A\_FILE|NOT\_A\_FILE|FILE\_ALREADY\_EXISTS unless overwrite|
> |{{OMKeyCreateRequest}} \(LEGACY \+ fs
> paths\)|NOT\_A\_FILE|NOT\_A\_FILE|allowed|
> |{{OMKeyCreateRequestWithFSO}}|NOT\_A\_FILE|NOT\_A\_FILE|allowed|
> |{{S3InitiateMultipartUploadRequestWithFSO}}|NOT\_A\_FILE|NOT\_A\_FILE
> \(HDDS\-17644\)|allowed|
> |{{S3InitiateMultipartUploadRequest}} \(LEGACY \+ fs paths\)|not checked|not
> checked|allowed|
> |{{S3MultipartUploadCompleteRequestWithFSO}}|NOT\_A\_FILE|NOT\_A\_FILE
> \(HDDS\-17696\)|allowed|
> |{{S3MultipartUploadCompleteRequest}} \(LEGACY \+ fs paths\)|NOT\_A\_FILE|not
> checked|allowed|
> HDDS\-17644 and HDDS\-17696 were both caused by a copy that missed the "a
> parent is a file" case. The exception messages also differ between copies for
> the same result.
> LEGACY {{S3InitiateMultipartUploadRequest}} calls {{verifyFilesInPath}} but
> only uses the result for ACLs. LEGACY {{S3MultipartUploadCompleteRequest}}
> does not use {{verifyFilesInPath}}; it only looks up {{key/}} in the key
> table.
> _Proposal_
> {{OMFileCreateRequest.checkDirectoryResult(keyName, isOverWrite, result)}}
> already covers all three columns. Move it to a static method in
> {{OMFileRequest}} and make the FSO and LEGACY create requests above call it,
> with overwrite allowed for key create and MPU.
> * Keep this a pure refactor: no change in result codes or in the order of
> checks in each request. For example, {{OMKeyCreateRequestWithFSO}} runs the
> atomic rewrite and ETag checks before the path check, so each call site
> replaces its own if/else in place.
> * {{OMDirectoryCreateRequest}} has different rules \(a file in the path is
> {{FILE_ALREADY_EXISTS}}\) and stays as is.
> * Exception messages may be unified. Result codes must not change, since
> clients map them to exceptions.
> * Key commit \({{OMKeyCommitRequest}}, and {{OMFileRequest.getParentID}} for
> FSO\) has its own checks with a different shape and is out of scope.
> * The remaining gaps in LEGACY buckets, {{S3InitiateMultipartUploadRequest}}
> not checking the path and {{S3MultipartUploadCompleteRequest}} not checking
> for a file in the parent path, are behavior changes and should be a separate
> fix. They change what the OM apply path writes, so they may need a version
> check once ZDU is supported.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]