KazukiKandaKK opened a new pull request, #19609: URL: https://github.com/apache/pinot/pull/19609
Fixes #9016 (first TODO item only): CreateSegment always walked -dataDir recursively. `CreateSegmentCommand` (the `pinot-admin.sh CreateSegment` CLI) always searches `-dataDir` recursively for input files, with no way to opt out. This addresses the first TODO item in #9016: - [ ] provide recursive flag in commandline utils The other TODO items in that issue (glob matcher prefix, plugin config unification, exception pipe-back) are unrelated to this change and are left for separate follow-up PRs. Note: the newer job-spec based ingestion path (`SegmentGenerationJobSpec#isSearchRecursively`, used by `LaunchDataIngestionJob`) already has an equivalent flag. This PR only adds the same capability to the older, still-supported `CreateSegment` command, which had none. ## Changes - Added a `-searchRecursively` boolean CLI option to `CreateSegmentCommand`, default `true` (so existing scripts/behavior are unchanged if the flag is omitted). - `arity = "1"` is set explicitly: a plain boolean flag with a `true` default cannot be turned off by simply passing `-searchRecursively` (picocli's default flag semantics only ever set it to `true`), so the option requires an explicit `true`/`false` value. - `getDataFilesHelper` now only recurses into subdirectories when the flag is `true`; top-level scanning behavior is unchanged either way. - Added `CreateSegmentCommandTest` with 6 cases: 3 exercising `getDataFiles(...)` directly via the fluent setter (default / explicit true / false), and 3 exercising the same scenarios through actual CLI argument parsing (`picocli.CommandLine#parseArgs`), so the `-searchRecursively false` path is verified the same way a real invocation would parse it. ## Testing - `mvn -pl pinot-tools -am install -DskipTests` — build succeeds. - `mvn -pl pinot-tools test -Dtest=CreateSegmentCommandTest` — all 6 new tests pass. - `mvn -pl pinot-tools test` (full module) — all existing tests still pass, no regressions. - `mvn -pl pinot-tools -am spotless:apply` / `checkstyle:check` / `license:check` — all clean. - Verified locally that omitting `arity = "1"` causes `-searchRecursively false` to fail with `picocli.CommandLine$UnmatchedArgumentException: Unmatched argument at index 5: 'false'`, confirming the explicit arity is required for this option to actually work as a false-able boolean flag. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
