nchammas opened a new pull request #27800: [SPARK-31041][BUILD] Make arguments to make-distribution.sh position-independent URL: https://github.com/apache/spark/pull/27800 ### What changes were proposed in this pull request? This PR makes `dev/make-distribution.sh` a bit easier to use by allowing it to handle options in a position-independent way. ### Why are the changes needed? It's surprising for command-line options to be position-dependent. The errors that get thrown when you pass the correct option (like `--pip`) but in the wrong order are confusing. ### Does this PR introduce any user-facing change? No. ### How was this patch tested? I added some debugging code to `make-distribution.sh`: ```diff diff --git a/dev/make-distribution.sh b/dev/make-distribution.sh index 835a32cf29..d482a340ce 100755 --- a/dev/make-distribution.sh +++ b/dev/make-distribution.sh @@ -90,6 +90,15 @@ while (( "$#" )); do shift done +echo "--tgz $MAKE_TGZ" +echo "--pip $MAKE_PIP" +echo "--r $MAKE_R" +echo "--mvn $MVN" +echo "--name $NAME" +echo "mvn-opts: ${MAVEN_CLI_OPTS[@]}" + +exit 1 + if [ -z "$JAVA_HOME" ]; then # Fall back on JAVA_HOME from rpm, if found if [ $(command -v rpm) ]; then ``` And I ran a few different invocations of `make-distribution.sh` to check that the options were parsed correctly: ```sh $ ./dev/make-distribution.sh \ --name custom-spark \ --pip --r --tgz \ -Psparkr \ -Phadoop-2.7 \ -Phive -Phive-thriftserver \ -Pmesos -Pyarn -Pkubernetes --tgz true --pip true --r true --mvn .../spark/build/mvn --name custom-spark mvn-opts: -Psparkr -Phadoop-2.7 -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes ``` ```sh $ ./dev/make-distribution.sh \ -Psparkr --name custom-spark \ -Phadoop-2.7 -Phive --pip --r \ -Phive-thriftserver -Pmesos --tgz -Pyarn -Pkubernetes --tgz true --pip true --r true --mvn .../spark/build/mvn --name custom-spark mvn-opts: -Psparkr -Phadoop-2.7 -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes ``` ```sh $ ./dev/make-distribution.sh \ -Phadoop-2.7 -Phive -Phadoop-cloud \ --pip --tgz false --pip true --r false --mvn .../spark/build/mvn --name none mvn-opts: -Phadoop-2.7 -Phive -Phadoop-cloud ``` I also ran some invocations of `make-distribution.sh` all the way through to confirm that they succeed.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
