Thanks. Running smpd -d 0 &
did the trick. Although I needed to sleep 5 afterwards because I was getting to mpiexec to quickly. Raymond > Hi, > > Am 17.02.2017 um 21:55 schrieb Raymond Norris: > > >> Hi, > >> > >>> Am 17.02.2017 um 18:54 schrieb Raymond Norris > >> <[email protected]>: > >>> > >>> Hi, > >>> > >>> I'm using SoGE 8.1.9. > >>> > >>> When we setup our jobs to run with loose integration and SSH, things > work > >> fine. I'm now trying to configure this with tight integration and qrsh. > >> Sans > a > >> dozen or so ports, everything's closed. My understanding is that qrsh > uses > >> rsh underneath and that we need to open port 514, which we've done. > >> However, we get the following: > >> > >> No. If you set: > >> > >> $ qconf -mconf > >> … > >> rsh_command builtin > >> rsh_daemon builtin > >> > >> it uses its own protocol. Even when set to `rsh` resp. `rshd`, it will in > >> both > >> cases use a random port, not 514. > >> > >> https://arc.liv.ac.uk/SGE/htmlman/htmlman5/remote_startup.html > might > >> give some insight. > >> > >> > >>> qrsh -inherit -verbose node-01 smpd -s -phrase ... -port ... > >> > >> This looks like an old startup of MPICH2. You don't use Hydra? > > > > We ship MPICH2 with our application, so I'm starting there. I've built it > > with > Hydra, but it fails. > > smpd startup is outdated: > > https://arc.liv.ac.uk/SGE/howto/mpich2-integration/mpich2- > integration.html > > Which exact version of MPICH are you using in detail? > > To ship something with your application you can also try Open MPI or IBM's > platform MPI in the community edition: > https://www.ibm.com/developerworks/downloads/im/mpi/ > > > > I'm guessing, as Simao Afonso references, each ssh connection requires a > different ephemeral port. > > Correct. > > > >>> Starting server daemon at host "node-01" > >>> error: executing task of job 1032 failed: failed sending task to > execd@node- > >> 01: got send error > >>> error: commlib error: got select error (No route to host) > >>> > >>> To me, that tells me we don't have 514 open ("No route to host"). Also, I > >> think the issue is that we're not running rshd on node-01, which I'm not > sure > >> if they're ok with running - which we'd need to, righ? So my question is, > how > >> do I get qrsh working so that I can track resources, but configure it with > SSH - > >> or can this not work - do I need to rsh? > >> > >> See the above provided link, `ssh` is also covered near the end of the > >> document. > > > > I can test the PAM, but in the end, will I be in the same situation where > ssh/sshd will use an array of ports? As it stands now, keeping most of the > ports closed, these are my options > > > > 1. loose integration with SSH: works > > Startup works as it uses port 22 only. It doesn't need any additional ports > for > communication during operation later on? > > > > 2. tight integration with Hydra: fails -- needs ephemeral ports > > Yep. > > > > 3. tight integration with qrsh: fails -- needs ephemeral ports > > Yep. > > > > 4. tight integration with qrsh, using SSH: not tested yet (needs ephemeral > ports?) > > Yes. > > > > 5. recompile SoGE w/ tight SSH > > Doesn't change the original problem. The idea to recompile SoGE with SSH is > to include an additional group ID for proper accounting and job termination > (even when the process jumps out of the process tree) even for processes > started by the job's private `sshd`. > > > > Am I overlooking any other options? > > I see additional problems: at least Open MPI will use a range of ports on its > own after it was started (by whatever means). I would assume that the other > MPI implementations behave the same. > > As we have no firewalls on the nodes (just disabled `ssh` for ordinary users > for the default port 22), I never faced that MPICH may or may not need > additional ports. > > - -- Reuti > -----BEGIN PGP SIGNATURE----- > Comment: GPGTools - https://gpgtools.org > > iEYEARECAAYFAlinaywACgkQo/GbGkBRnRrthACg3yLABkSxDwNNZUl9ShylsM > F3 > FyUAniQknu4WMQ4XYdp4HqCqoibvR9qN > =aYZj > -----END PGP SIGNATURE----- _______________________________________________ SGE-discuss mailing list [email protected] https://arc.liv.ac.uk/mailman/listinfo/sge-discuss
