mbutrovich commented on code in PR #2103: URL: https://github.com/apache/datafusion-comet/pull/2103#discussion_r2267186446
########## docs/source/contributor-guide/benchmarking_macos.md: ########## @@ -82,6 +72,17 @@ export SPARK_MASTER=spark://Rustys-MacBook-Pro.local:7077 $SPARK_HOME/sbin/start-worker.sh $SPARK_MASTER ``` +### Start local Apache Spark cluster using `spark-class` +For Apache Spark distributions installed using `brew` tool, it may happen there is no `$SPARK_HOME/sbin` folder on your machine. +In order to start local Apache Spark cluster on `localhost:7077` port, run: +```shell +$SPARK_HOME/bin/spark-class org.apache.spark.deploy.master.Master --host 127.0.0.1 --port 7077 --webui-port 8080 +``` + +Once master has started, in separate console start the worker referring the spark master uri on `localhost:7077` +```shell +$SPARK_HOME/bin/spark-class org.apache.spark.deploy.worker.Worker --cores 8 --memory 16G spark://localhost:7077 Review Comment: Do we want to be prescribing things like `cores` and `memory`? I almost feel like if the user installed from Homebrew and is missing sbin scripts to recommend they retrieve them for their Spark version from the repo. https://github.com/apache/spark/tree/master/sbin -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
