dongjoon-hyun opened a new pull request, #59612: URL: https://github.com/apache/spark/pull/59612
### What changes were proposed in this pull request? This PR aims to upgrade `scopt` to 4.2.0 for the `examples` module. - https://github.com/scopt/scopt/releases/tag/v4.2.0 (2026-10-05) - https://github.com/scopt/scopt/releases/tag/v4.1.0 (2022-07-02) - https://github.com/scopt/scopt/releases/tag/v4.0.1 (2021-03-06) - https://github.com/scopt/scopt/releases/tag/v4.0.0 (2020-11-29) ### Why are the changes needed? Apache Spark has been using `scopt` 3.7.1, which was released on 2018-12-13, since SPARK-29289. `scopt` 4.x keeps source compatibility with the 3.x style object-oriented DSL (`scopt.OptionParser`), which is the only API used by Spark examples, and brings several bug fixes. - Fix parsing options after encountering `--` (scopt/scopt#317) - Fix nested commands parsing (scopt/scopt#339) - Fix parsing multiple arguments with `minOccurs(...)` (scopt/scopt#314) - Support passing `-` (a single dash) as an argument (scopt/scopt#316) - Add support for space-separated list (scopt/scopt#367) In addition, `scopt_2.13:4.2.0` is built with Scala 2.13.18, which is the same Scala version used by Apache Spark, and has no additional runtime dependency. ### Does this PR introduce _any_ user-facing change? No behavior change is expected. Only the `scopt` jar in the `examples/jars` directory of the binary distribution changes. The error messages and usage outputs of the examples are identical. ### How was this patch tested? Pass the CIs. In addition, I manually verified the following. 1. Build the distribution and run the examples. ``` $ build/sbt package $ bin/run-example mllib.Correlations --input data/mllib/sample_linear_regression_data.txt $ bin/run-example mllib.DenseKMeans -k 2 --numIterations 5 --initMode Random data/mllib/kmeans_data.txt $ bin/run-example ml.DecisionTreeExample --algo classification --maxDepth 3 --fracTest 0.2 --dataFormat libsvm data/mllib/sample_libsvm_data.txt ``` 2. Compare the error handling between `scopt` 3.7.1 and 4.2.0 (unknown option, missing required option/argument, invalid type, `checkConfig` failure). Both versions produce the identical outputs and exit codes. ``` $ bin/run-example mllib.DenseKMeans -k 2 --bogus 1 in.txt Error: Unknown option --bogus Error: Unknown argument 'in.txt' DenseKMeans: an example k-means app for dense data. Usage: DenseKMeans [options] <input> -k, --k <value> number of clusters, required --numIterations <value> number of iterations, default: 10 --initMode <value> initialization mode (Random,Parallel), default: Parallel <input> input paths to examples ``` ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Opus 5.5 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
