github-actions[bot] commented on code in PR #69528:
URL: https://github.com/apache/doris/pull/69528#discussion_r4365938520
##########
be/src/io/fs/local_file_system.cpp:
##########
@@ -455,28 +455,89 @@ const std::shared_ptr<LocalFileSystem>&
global_local_filesystem() {
Status LocalFileSystem::canonicalize_local_file(const std::string& dir,
const std::string& file_path,
std::string* full_path) {
- const std::string absolute_path = dir + "/" + file_path;
+ // An empty root would turn a relative request into an unrestricted
absolute path.
+ if (dir.empty()) {
+ return Status::InvalidArgument("user_files_secure_path must not be
empty");
+ }
+ // Reject embedded NULs before passing paths to filesystem APIs that use C
strings.
+ if (file_path.empty() || file_path.find('\0') != std::string::npos) {
+ return Status::InvalidArgument("local file path must not be empty or
contain NUL");
+ }
+ Path local_path;
+ // Convert file: URIs to filesystem paths and reject remote schemes before
any reader opens them.
+ RETURN_IF_ERROR(convert_to_abs_path(file_path, local_path));
+ if (file_path.rfind("file:", 0) == 0) {
+ // file:a%20b names "a b", but a bare path containing "%20" must stay
literal.
+ // Decode once, preserve '+' and invalid escapes, matching Lance's
file provider.
+ auto encoded = local_path.string();
+ // Query and fragment are URI components, not part of the filesystem
path.
+ encoded = encoded.substr(0, encoded.find_first_of("?#"));
+ const auto hex_digit = [](char c) -> int {
+ if (c >= '0' && c <= '9') return c - '0';
+ if (c >= 'a' && c <= 'f') return c - 'a' + 10;
+ if (c >= 'A' && c <= 'F') return c - 'A' + 10;
+ return -1;
+ };
+ std::string decoded;
+ decoded.reserve(encoded.size());
+ for (size_t i = 0; i < encoded.size(); ++i) {
+ if (encoded[i] == '%' && i + 2 < encoded.size()) {
+ const int high = hex_digit(encoded[i + 1]);
+ const int low = hex_digit(encoded[i + 2]);
+ if (high >= 0 && low >= 0) {
+ decoded += static_cast<char>((high << 4) | low);
+ i += 2;
+ continue;
+ }
+ }
+ decoded += encoded[i];
+ }
+ // %00 must not truncate the path when canonicalization reaches C
filesystem APIs.
+ if (decoded.find('\0') != std::string::npos) {
+ return Status::InvalidArgument("local file path must not contain
NUL");
+ }
+ local_path = std::move(decoded);
+ }
+ std::string secure_path;
+ RETURN_IF_ERROR(canonicalize(dir, &secure_path));
std::string canonical_path;
- RETURN_IF_ERROR(canonicalize(absolute_path, &canonical_path));
- if (!contain_path(dir, canonical_path)) {
- return Status::InvalidArgument("file path is not allowed: {}",
canonical_path);
+ const auto status = canonicalize(Path(secure_path) / local_path,
&canonical_path);
Review Comment:
[P2] Normalize file-URI dot segments before resolving symlinks
With `enable_local_tvf=true`, an ADMIN/OPERATOR,
`user_files_secure_path=/secure`, and `/secure/link -> /secure/nested/child`,
`file:///secure/link/../sample.lance` previously opened `/secure/sample.lance`:
[Lance parses the URI
first](https://github.com/lance-format/lance/blob/68c12dfd7efe02f90ad2d7f3a239a7eb64884e57/rust/lance-io/src/object_store.rs#L494-L502),
and its [URL parser removes `link/..`
lexically](https://github.com/servo/rust-url/blob/v2.5.8/url/src/parser.rs#L1264-L1275).
This new filesystem canonicalization instead follows `link` before processing
`..`, producing `/secure/nested/sample.lance`. If both datasets exist, schema
discovery and scanning silently read the other dataset; otherwise a valid URI
fails. Both targets are inside the secure root. This is distinct from the fixed
percent-decoding thread: it needs no escaped characters and arises at the new
canonicalization step. Normalize file-URI path segments before filesystem
resolution and containment, preserve
bare-path semantics, and add a schema/scan case with different datasets on
the two paths.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
