zy-kkk commented on PR #68707: URL: https://github.com/apache/doris/pull/68707#issuecomment-5989625426
Thanks, confirmed: a managed branch named `dev` was rejected by the guard, including `@branch(...)` table scans. Fixed in 0eac8211034. I did not align the guard with Lance's character set exactly, because neither route stays aligned: - Native validation: the Lance 12 Java SDK exposes no branch-name validator. `check_valid_branch` only runs inside Rust (`checkout_by_ref` and the dataset builder), and the managed path opens the joined URI without it. - Versioned Unicode data: the tables Lance uses come from the rustc that built the shipped lance-jni and lance-c, not from the minimum Rust version, so FE and BE can differ, and a table copied into Java would drift again on the next toolchain bump. So the guard no longer decides whether a name is valid. It only keeps the joined URI inside the table's `tree/` directory: - Rejected: empty, a leading or trailing `/`, empty segments, `..`, a trailing `.lock`, `\`, and any ASCII character other than letters, digits, `.`, `-`, `_` and `/`. In ASCII this is exactly Lance's set, and no segment can climb out of `tree/`, percent-encoded or not. - Outside ASCII only whitespace and control characters are rejected. ASCII classification does not depend on the Unicode version, and no whitespace or control character has ever been alphabetic or numeric, so every name `check_valid_branch` accepts passes, whichever Unicode version the JDK or rustc uses. - A name that passes but that Lance would reject cannot have been created by Lance, so the namespace reports it as a branch it does not record. Coverage, as you suggested, on an existing managed branch with a post-Unicode-13 letter: - `unicode_branch.lance` has a branch `dev` (U+1C89, Unicode 16), served by the REST fixture as the managed table `unicode_branch_managed`. It is built with pylance 12.0.0, since Lance 7 stores a non-ASCII branch directory percent-encoded, which Lance 12 does not find. - `test_lance_rest_search_snapshot` reads that branch through an `@branch` table scan, `vector_search(... "branch"="dev")` and a two-phase read. - `LanceCatalogClientTest` checks that `dev` and other non-ASCII letters and digits are accepted, and that the unsafe names are still rejected. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
