Hi Diego, Thx for the quick response.
Yes, I'm running it from oa-spot/. I just re-pasted the output of ps -ax | grep ipython at the end of this mail. Here are the screenshots (@Alan: thx for the info). Screenshot1: http://imgur.com/wR4g86M Screenshot2: http://imgur.com/MHA0sQQ The thing is that when I set up CDH on the 4 pseudo-nodes (master, s1,s2,s3), I realized that some of their assigned roles (I used the auto-assign option for the CDH roles) didn't match the requirements for each Spot node, as described in the documentation. Thus, I used only 3 nodes as described below: *pseudo-node name | Spot node name:* master | MASTER1, MASTER2, OA s3 | EDGE s2 |WORKER I know that the above setup wasn't going to provide any real functionality, but then again I was testing the steps before moving to a real cluster. spotuser@master:~/spot-oa$ ps -ax | grep ipython 5953 ? Sl 1:36 /usr/bin/python -m IPython.parallel.controller --profile-dir /home/spotuser/.ipython/profile_ia -- cluster-id --log-to-file --log-level=20 5962 ? Sl 0:00 /usr/bin/python -m IPython.parallel.controller --profile-dir /home/spotuser/.ipython/profile_ia -- cluster-id --log-to-file --log-level=20 5963 ? Sl 0:00 /usr/bin/python -m IPython.parallel.controller --profile-dir /home/spotuser/.ipython/profile_ia -- cluster-id --log-to-file --log-level=20 5966 ? Sl 0:00 /usr/bin/python -m IPython.parallel.controller --profile-dir /home/spotuser/.ipython/profile_ia -- cluster-id --log-to-file --log-level=20 5969 ? Sl 0:00 /usr/bin/python -m IPython.parallel.controller --profile-dir /home/spotuser/.ipython/profile_ia -- cluster-id --log-to-file --log-level=20 5974 ? Sl 0:52 /usr/bin/python -m IPython.parallel.engine --profile-dir /home/spotuser/.ipython/profile_ia --clus ter-id --log-to-file --log-level=20 5976 ? Sl 0:51 /usr/bin/python -m IPython.parallel.engine --profile-dir /home/spotuser/.ipython/profile_ia --clus ter-id --log-to-file --log-level=20 5981 ? Sl 0:49 /usr/bin/python -m IPython.parallel.engine --profile-dir /home/spotuser/.ipython/profile_ia --clus ter-id --log-to-file --log-level=20 5998 ? Sl 0:50 /usr/bin/python -m IPython.parallel.engine --profile-dir /home/spotuser/.ipython/profile_ia --clus ter-id --log-to-file --log-level=20 10914 ? S 0:00 /usr/bin/python /usr/local/bin/ipython notebook --no-mathjax --profile=ia --port=8889 --ip=0.0.0.0 --no-browser --NotebookApp.extra_static_paths=["ui/ipython/"] 12417 ? S 0:00 /usr/bin/python /usr/local/bin/ipython notebook --no-mathjax --profile=ia --port=8889 --ip=0.0.0.0 --no-browser --NotebookApp.extra_static_paths=["ui/ipython/"] 19058 ? Sl 1:34 /usr/bin/python -m IPython.parallel.controller --profile-dir /home/spotuser/.ipython/profile_defau lt --cluster-id --log-to-file --log-level=20 19066 ? Sl 0:00 /usr/bin/python -m IPython.parallel.controller --profile-dir /home/spotuser/.ipython/profile_defau lt --cluster-id --log-to-file --log-level=20 19067 ? Sl 0:00 /usr/bin/python -m IPython.parallel.controller --profile-dir /home/spotuser/.ipython/profile_defau lt --cluster-id --log-to-file --log-level=20 19070 ? Sl 0:00 /usr/bin/python -m IPython.parallel.controller --profile-dir /home/spotuser/.ipython/profile_defau lt --cluster-id --log-to-file --log-level=20 19074 ? Sl 0:00 /usr/bin/python -m IPython.parallel.controller --profile-dir /home/spotuser/.ipython/profile_defau lt --cluster-id --log-to-file --log-level=20 19084 ? Sl 0:52 /usr/bin/python -m IPython.parallel.engine --profile-dir /home/spotuser/.ipython/profile_default - -cluster-id --log-to-file --log-level=20 19092 ? Sl 0:51 /usr/bin/python -m IPython.parallel.engine --profile-dir /home/spotuser/.ipython/profile_default - -cluster-id --log-to-file --log-level=20 19102 ? Sl 0:49 /usr/bin/python -m IPython.parallel.engine --profile-dir /home/spotuser/.ipython/profile_default - -cluster-id --log-to-file --log-level=20 19118 ? Sl 0:49 /usr/bin/python -m IPython.parallel.engine --profile-dir /home/spotuser/.ipython/profile_default - -cluster-id --log-to-file --log-level=20 26655 lxc/tty1 R+ 0:00 grep --color=auto ipython 26719 ? R 156:55 /usr/bin/python /usr/local/bin/ipython notebook --no-mathjax --profile=ia --port=8889 --ip=0.0.0.0 --no-browser --NotebookApp.extra_static_paths=["ui/ipython/"] 2017-04-08 18:03 GMT+03:00 Ross, Alan D <[email protected]>: > No attachments here. You can upload to imgur or somewhere else and then > post the link to the screenshot. > > -----Original Message----- > From: Diego (ChInO) [mailto:[email protected]] > Sent: Saturday, April 8, 2017 6:48 AM > To: [email protected] > Subject: Re: Help on Spot installation (following Slack discussion with > Nate) > > Hi, > > I just want to make sure you are running ipython from the right place. > Could you please confirm you are running it from spot-oa/ dir? > > Btw, I don't see any screenshot :p > > On Fri, Apr 7, 2017, 11:05 PM Dimitris Papadopoulos < > [email protected]> wrote: > > > It appears i hadnt added myself to the dev mailing list. > > > > Hi again, > > > > First things first, let me recap our discussion so far: > > > > -We are trying to install Spot on a server with pseudo-distributed nodes. > > -Each one of the nodes is actually an *lxc,* while the server runs on > > *Ubuntu 14.04*. > > -We started by installing CDH 10, following the cloudera site > > instructions and it seems to be working as intented. > > -We went on to install Spot, following the installation instructions > > as provided in the documentation webpage. Some parts (mainly where > > .conf files are mentioned) seem to be a bit vague for us, so we 've > > probably not configured the nodes right. > > -It seems that we haven't managed to get Spot running (when we enter > > the ".../files/ui/flow/suspicious.html." we get an unresponsive page > > [Screenshot1] and we would like to fix that asap. > > -I have submitted two screenshots, one of the suspicious tab > > [Screenshot1] and one of the ".../tree" view which lets me navigate > > normally through the files [Screenshot2]. At the end of the message is > > the result from the ps -ax | grep ipython > > > > We are probably asking too much effort from your side, but we could > > really benefit from a clean installation tutorial (video-blog-text > > format) that install all dependencies and sets all the configuration > > values for each one of the nodes. We could ofc provide a server for > > this purpose. Dempnstrating a Spot installation in a > > pseudo-distributed configuration could help us set it up on a large > distributed-computing infrastructure. > > > > Thank you so much for your time, > > Dimitris > > > > ------------------------------------------------------------------- > > spotuser@master:~$ ps -ax | grep ipython > > 5953 ? Sl 0:41 /usr/bin/python -m IPython.parallel.controller > > --profile-dir /home/spotuser/.ipython/profile_ia -- cluster-id > > --log-to-file --log-level=20 > > 5962 ? Sl 0:00 /usr/bin/python -m IPython.parallel.controller > > --profile-dir /home/spotuser/.ipython/profile_ia -- cluster-id > > --log-to-file --log-level=20 > > 5963 ? Sl 0:00 /usr/bin/python -m IPython.parallel.controller > > --profile-dir /home/spotuser/.ipython/profile_ia -- cluster-id > > --log-to-file --log-level=20 > > 5966 ? Sl 0:00 /usr/bin/python -m IPython.parallel.controller > > --profile-dir /home/spotuser/.ipython/profile_ia -- cluster-id > > --log-to-file --log-level=20 > > 5969 ? Sl 0:00 /usr/bin/python -m IPython.parallel.controller > > --profile-dir /home/spotuser/.ipython/profile_ia -- cluster-id > > --log-to-file --log-level=20 > > 5974 ? Sl 0:22 /usr/bin/python -m IPython.parallel.engine > > --profile-dir /home/spotuser/.ipython/profile_ia --clus ter-id > > --log-to-file --log-level=20 > > 5976 ? Sl 0:21 /usr/bin/python -m IPython.parallel.engine > > --profile-dir /home/spotuser/.ipython/profile_ia --clus ter-id > > --log-to-file --log-level=20 > > 5981 ? Sl 0:20 /usr/bin/python -m IPython.parallel.engine > > --profile-dir /home/spotuser/.ipython/profile_ia --clus ter-id > > --log-to-file --log-level=20 > > 5998 ? Sl 0:21 /usr/bin/python -m IPython.parallel.engine > > --profile-dir /home/spotuser/.ipython/profile_ia --clus ter-id > > --log-to-file --log-level=20 > > 10914 ? S 0:00 /usr/bin/python /usr/local/bin/ipython > > notebook --no-mathjax --profile=ia --port=8889 --ip=0.0.0.0 > > --no-browser --NotebookApp.extra_static_paths=["ui/ipython/"] > > 12417 ? S 0:00 /usr/bin/python /usr/local/bin/ipython > > notebook --no-mathjax --profile=ia --port=8889 --ip=0.0.0.0 > > --no-browser --NotebookApp.extra_static_paths=["ui/ipython/"] > > 19058 ? Sl 0:38 /usr/bin/python -m IPython.parallel.controller > > --profile-dir /home/spotuser/.ipython/profile_defau > > lt --cluster-id --log-to-file --log-level=20 > > 19066 ? Sl 0:00 /usr/bin/python -m IPython.parallel.controller > > --profile-dir /home/spotuser/.ipython/profile_defau > > lt --cluster-id --log-to-file --log-level=20 > > 19067 ? Sl 0:00 /usr/bin/python -m IPython.parallel.controller > > --profile-dir /home/spotuser/.ipython/profile_defau > > lt --cluster-id --log-to-file --log-level=20 > > 19070 ? Sl 0:00 /usr/bin/python -m IPython.parallel.controller > > --profile-dir /home/spotuser/.ipython/profile_defau > > lt --cluster-id --log-to-file --log-level=20 > > 19074 ? Sl 0:00 /usr/bin/python -m IPython.parallel.controller > > --profile-dir /home/spotuser/.ipython/profile_defau > > lt --cluster-id --log-to-file --log-level=20 > > 19084 ? Sl 0:21 /usr/bin/python -m IPython.parallel.engine > > --profile-dir /home/spotuser/.ipython/profile_default - -cluster-id > > --log-to-file --log-level=20 > > 19092 ? Sl 0:21 /usr/bin/python -m IPython.parallel.engine > > --profile-dir /home/spotuser/.ipython/profile_default - -cluster-id > > --log-to-file --log-level=20 > > 19102 ? Sl 0:20 /usr/bin/python -m IPython.parallel.engine > > --profile-dir /home/spotuser/.ipython/profile_default - -cluster-id > > --log-to-file --log-level=20 > > 19118 ? Sl 0:20 /usr/bin/python -m IPython.parallel.engine > > --profile-dir /home/spotuser/.ipython/profile_default - -cluster-id > > --log-to-file --log-level=20 > > 23679 lxc/tty1 S+ 0:00 grep --color=auto ipython > > 26719 ? S 65:15 /usr/bin/python /usr/local/bin/ipython > > notebook --no-mathjax --profile=ia --port=8889 --ip=0.0.0.0 > > --no-browser --NotebookApp.extra_static_paths=["ui/ipython/"] > > spotuser@master:~$ > > > > > > >
