Those two export lines means set SPARK_HOME and PATH *as environment variables* in the session you have in Ubuntu.
Check this website for more info bash - How do I add environment variables? - Ask Ubuntu <https://askubuntu.com/questions/58814/how-do-i-add-environment-variables#:~:text=To%20set%20an%20environment%20variable%20everytime%2C%20use%20the%20export%20command,script%20it%20will%20not%20work.> If you are familiar with windows, then they are equivalent to Windows environment variables. For example, note SPARK_HOME [image: image.png] <https://askubuntu.com/questions/58814/how-do-i-add-environment-variables#:~:text=To%20set%20an%20environment%20variable%20everytime%2C%20use%20the%20export%20command,script%20it%20will%20not%20work.>Next. Now you are trying to start spark in standalone mode. Check the following link Spark Standalone Mode - Spark 3.1.1 Documentation (apache.org) <http://spark.apache.org/docs/latest/spark-standalone.html> to learn more about it And also check the logfile generated by invoking spark-master.sh in your output starting org.apache.spark.deploy.master.Master, logging to* /opt/spark/logs/spark--org.apache.spark.deploy.master.Master-1-23d865d7f117.out* HTH view my Linkedin profile <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> *Disclaimer:* Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction. On Tue, 30 Mar 2021 at 20:48, GUINKO Ferdinand <tonguimferdin...@guinko.net> wrote: > This is what I am having now: > > root@33z261w1a18:/opt/spark# *export SPARK_HOME=/opt/spark* > root@33z261w1a18:/opt/spark# *export > PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/bin:/sbin:$SPARK_HOME/bin:$SPARK_HOME/sbin* > root@33z261w1a18:/opt/spark# *echo $PATH* > > /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/bin:/sbin:/opt/spark/bin:/opt/spark/sbin > root@33z261w1a18:/opt/spark# *start-master.sh* > starting org.apache.spark.deploy.master.Master, logging to > /opt/spark/logs/spark--org.apache.spark.deploy.master.Master-1-23d865d7f117.out > root@33z261w1a18:/opt/spark# > > It seems that SPARK was trying to start but couldn't. > > Please would you explain me what the following lines means and do: > > export SPARK_HOME=/opt/spark > export > PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/bin:/sbin:$SPARK_HOME/bin:$SPARK_HOME/sbin > > then > > echo $PATH > > > Thank you for the assistance. > > -- > «Ce dont le monde a le plus besoin, c'est d'hommes, non pas des hommes > qu'on achète et qui se vendent, mais d'hommes profondément loyaux et > intègres, des hommes qui ne craignent pas d'appeler le péché par son nom, > des hommes dont la conscience soit aussi fidèle à son devoir que la > boussole l'est au pôle, des hommes qui défendraient la justice et la vérité > même si l'univers s'écroulait.» Ellen Gould WHITE, education, P. 55 > > Le Mardi, Mars 30, 2021 15:00 GMT, Mich Talebzadeh < > mich.talebza...@gmail.com> a écrit: > > > Ok simple > > export SPARK_HOME=/opt/spark > export > PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/bin:/sbin:$SPARK_HOME/bin:$SPARK_HOME/sbin > > then > > echo $PATH > > which start-master.sh > > and send the output > > HTH > > > > > > view my Linkedin profile > <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> > > > > *Disclaimer:* Use it at your own risk. Any and all responsibility for any > loss, damage or destruction of data or any other property which may arise > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > > On Tue, 30 Mar 2021 at 13:23, GUINKO Ferdinand < > tonguimferdin...@guinko.net> wrote: > >> Where is SPARK? I also tried this: >> >> root@33z261w1a18:/opt/spark# *whereis spark-shell* >> spark-shell: /opt/spark/bin/spark-shell2.cmd >> /opt/spark/bin/spark-shell.cmd /opt/spark/bin/spark-shell >> >> -- >> «Ce dont le monde a le plus besoin, c'est d'hommes, non pas des hommes >> qu'on achète et qui se vendent, mais d'hommes profondément loyaux et >> intègres, des hommes qui ne craignent pas d'appeler le péché par son nom, >> des hommes dont la conscience soit aussi fidèle à son devoir que la >> boussole l'est au pôle, des hommes qui défendraient la justice et la vérité >> même si l'univers s'écroulait.» Ellen Gould WHITE, education, P. 55 >> >> Le Mardi, Mars 30, 2021 09:00 GMT, Mich Talebzadeh < >> mich.talebza...@gmail.com> a écrit: >> >> >> OK >> >> on your path where is SPARK? >> >> Try to edit your .profile with simple editor of your choice (vi, vim etc >> if you are on Linux) >> >> export >> PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/bin:/sbin >> *:$PATH* >> export SPARK_HOME=/opt/spark >> ## Add it to the PATH!!!! >> export PATH=$SPARK_HOME/bin:$SPARK_HOME/sbin:$PATH >> >> >> Once you have done it do >> >> . ./.profile >> >> send the output of >> >> echo $PATH >> >> HTH >> >> >> >> >> >> >> view my Linkedin profile >> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> >> >> >> >> *Disclaimer:* Use it at your own risk. Any and all responsibility for >> any loss, damage or destruction of data or any other property which may >> arise from relying on this email's technical content is explicitly >> disclaimed. The author will in no case be liable for any monetary damages >> arising from such loss, damage or destruction. >> >> >> >> On Tue, 30 Mar 2021 at 09:23, GUINKO Ferdinand < >> tonguimferdin...@guinko.net> wrote: >> >>> Good day Talebzadeh, >>> >>> here are the outputs: >>> >>> *cat ~/.profile* >>> >>> root@33z261w1a18:/home# cat ~/.profile >>> # ~/.profile: executed by Bourne-compatible login shells. >>> >>> if [ "$BASH" ]; then >>> if [ -f ~/.bashrc ]; then >>> . ~/.bashrc >>> fi >>> fi >>> >>> mesg n 2> /dev/null || true >>> export SPARK_HOME=/opt/spark >>> export >>> PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/bin:/sbin >>> export PYSPARK_PYTHON=/usr/bin/python3 >>> >>> >>> *echo $PATH* >>> >>> /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/bin:/sbin >>> >>> Thank you. >>> >>> -- >>> «Ce dont le monde a le plus besoin, c'est d'hommes, non pas des hommes >>> qu'on achète et qui se vendent, mais d'hommes profondément loyaux et >>> intègres, des hommes qui ne craignent pas d'appeler le péché par son nom, >>> des hommes dont la conscience soit aussi fidèle à son devoir que la >>> boussole l'est au pôle, des hommes qui défendraient la justice et la vérité >>> même si l'univers s'écroulait.» Ellen Gould WHITE, education, P. 55 >>> >>> Le Lundi, Mars 29, 2021 22:24 GMT, Mich Talebzadeh < >>> mich.talebza...@gmail.com> a écrit: >>> >>> >>> OK just do >>> >>> cat ~/.profile >>> >>> and send the content please >>> >>> also do >>> >>> echo $PATH >>> >>> and send the output as well. >>> >>> HTH >>> >>> >>> >>> >>> >>> view my Linkedin profile >>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> >>> >>> >>> >>> *Disclaimer:* Use it at your own risk. Any and all responsibility for >>> any loss, damage or destruction of data or any other property which may >>> arise from relying on this email's technical content is explicitly >>> disclaimed. The author will in no case be liable for any monetary damages >>> arising from such loss, damage or destruction. >>> >>> >>> >>> On Mon, 29 Mar 2021 at 22:06, GUINKO Ferdinand < >>> tonguimferdin...@guinko.net> wrote: >>> >>>> Hi Talebzadeh, >>>> >>>> thank you for your answer. I am sorry, but I don't undertand what I >>>> need to do. It seems that I already put *SPARK_HOME/sbin* in my path >>>> as I have this line among the 3 lines I have sent in my initial email: >>>> >>>> echo "export PATH=$PATH:$SPARK_HOME/bin:$SPARK_HOME/sbin" >> ~/.profile >>>> >>>> Isn't it? Also should I remove the "echo" as it seems you don't have >>>> them in your commands. >>>> >>>> I am following this tutorial: >>>> https://phoenixnap.com/kb/install-spark-on-ubuntu but installing Spark >>>> 3.1.1 with Hadoop 2.7. >>>> >>>> Thank you for any help. >>>> >>>> Ferdinand >>>> -- >>>> «Ce dont le monde a le plus besoin, c'est d'hommes, non pas des hommes >>>> qu'on achète et qui se vendent, mais d'hommes profondément loyaux et >>>> intègres, des hommes qui ne craignent pas d'appeler le péché par son nom, >>>> des hommes dont la conscience soit aussi fidèle à son devoir que la >>>> boussole l'est au pôle, des hommes qui défendraient la justice et la vérité >>>> même si l'univers s'écroulait.» Ellen Gould WHITE, education, P. 55 >>>> >>>> Le Lundi, Mars 29, 2021 18:57 GMT, Mich Talebzadeh < >>>> mich.talebza...@gmail.com> a écrit: >>>> >>>> >>>> In my case >>>> >>>> export SPARK_HOME=/d4T/hduser/spark-3.1.1-bin-hadoop3.2 >>>> export PATH=$SPARK_HOME/bin:*$SPARK_HOME/sbin*:$PATH >>>> >>>> which start-master.sh >>>> /d4T/hduser/spark-3.1.1-bin-hadoop3.2/sbin/start-master.sh >>>> >>>> You need to add *$SPARK_HOME/sbin* to your path as well >>>> >>>> HTH >>>> >>>> >>>> >>>> >>>> view my Linkedin profile >>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> >>>> >>>> >>>> >>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for >>>> any loss, damage or destruction of data or any other property which may >>>> arise from relying on this email's technical content is explicitly >>>> disclaimed. The author will in no case be liable for any monetary damages >>>> arising from such loss, damage or destruction. >>>> >>>> >>>> >>>> On Mon, 29 Mar 2021 at 19:19, GUINKO Ferdinand < >>>> tonguimferdin...@guinko.net> wrote: >>>> >>>>> Hi, >>>>> >>>>> I have installed Docker, Spark 3.1.1 and Hadoop 2.7 on Ubuntu 18.04. >>>>> >>>>> After I have executed the following 3 lines >>>>> >>>>> echo "export SPARK_HOME=/opt/spark" >> ~/.profile >>>>> echo "export PATH=$PATH:$SPARK_HOME/bin:$SPARK_HOME/sbin" >> ~/.profile >>>>> echo "export PYSPARK_PYTHON=/usr/bin/python3" >> ~/.profile >>>>> >>>>> I tried to start the master server by typing >>>>> >>>>> start-master.sh >>>>> >>>>> and got the following error message >>>>> >>>>> bash: start-master.sh: command not found >>>>> >>>>> Thank you for any help. >>>>> Ferdinand >>>>> -- >>>>> «Ce dont le monde a le plus besoin, c'est d'hommes, non pas des hommes >>>>> qu'on achète et qui se vendent, mais d'hommes profondément loyaux et >>>>> intègres, des hommes qui ne craignent pas d'appeler le péché par son nom, >>>>> des hommes dont la conscience soit aussi fidèle à son devoir que la >>>>> boussole l'est au pôle, des hommes qui défendraient la justice et la >>>>> vérité >>>>> même si l'univers s'écroulait.» Ellen Gould WHITE, education, P. 55 >>>> >>>> >>>> >>> >>> >>> >> >> >> > > >
