To All Just letting know that I have solved this problem. After being given some hints I dumped the network traffic between the two servers and found the requests were coming from the incorrect server name. The submission/login node has two interfaces with two different names, public & private, with the private one being used for all HPC/torque traffic. This private interface is the one that is advertised in /etc/hosts however I had run oscar_wizard to check on some configuration and this had modified /etc/hosts and removed the private address name and added the public address name, therefore torque was seeing the wrong name/ipaddress combination and rejecting requests.
I have manually corrected /etc/hosts on all servers and everything now works correctly. Thanks again --------------------------------------------------------------------- Richard A. Young ICT Services Email: richard.yo...@usq.edu.au Phone: (07) 46315557 Mob: 0437544370 Fax: (07) 46312798 --------------------------------------------------------------------- -----Original Message----- From: Richard Young Sent: Thursday, 18 December 2014 9:48 AM To: oscar-users@lists.sourceforge.net Subject: [Oscar-users] FW: Invalid credential for PBS commands To All I am sorry for submitting this again as it seems my original email didn't make it to the list. I have recently come across a problem with one of my submission nodes on our HPC and was wondering if somebody could help. Our HPC has two submission/login nodes and one head node, both submission/login nodes were setup the same and working until recently. One of the submission nodes now gets the following error whenever a torque commands is run: qstat: Invalid credential MSG=Hosts do not match qmgr obj= svr=default: Invalid credential MSG=Hosts do not match torque was install via rpm is version of 4.2.9. Both submission/login nodes are setup in the queue configuration correctly, as below, they were taken out and put back with the torque server and client restarted. The file /var/lib/torque/serv_priv/nodes has both servers in it, and this hasn't changed. set server submit_hosts += server1 set server submit_hosts += server2 The problems exist for all users and the admin account. Has somebody seen this problem before and can provide some help in fixing it. Thanks --------------------------------------------------------------------- Richard A. Young ICT Services HPC Support Officer University of Southern Queensland Toowoomba, Queensland 4350 Australia Email: richard.yo...@usq.edu.au Phone: (07) 46315557 Mob: 0437544370 Fax: (07) 46312798 --------------------------------------------------------------------- _____________________________________________________________ This email (including any attached files) is confidential and is for the intended recipient(s) only. If you received this email by mistake, please, as a courtesy, tell the sender, then delete this email. The views and opinions are the originator's and do not necessarily reflect those of the University of Southern Queensland. Although all reasonable precautions were taken to ensure that this email contained no viruses at the time it was sent we accept no liability for any losses arising from its receipt. The University of Southern Queensland is a registered provider of education with the Australian Government. (CRICOS Institution Code QLD 00244B / NSW 02225M, TEQSA PRV12081 ) ------------------------------------------------------------------------------ Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server from Actuate! Instantly Supercharge Your Business Reports and Dashboards with Interactivity, Sharing, Native Excel Exports, App Integration & more Get technology previously reserved for billion-dollar corporations, FREE http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk _______________________________________________ Oscar-users mailing list Oscar-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oscar-users _____________________________________________________________ This email (including any attached files) is confidential and is for the intended recipient(s) only. If you received this email by mistake, please, as a courtesy, tell the sender, then delete this email. The views and opinions are the originator's and do not necessarily reflect those of the University of Southern Queensland. Although all reasonable precautions were taken to ensure that this email contained no viruses at the time it was sent we accept no liability for any losses arising from its receipt. The University of Southern Queensland is a registered provider of education with the Australian Government. (CRICOS Institution Code QLD 00244B / NSW 02225M, TEQSA PRV12081 ) ------------------------------------------------------------------------------ Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server from Actuate! Instantly Supercharge Your Business Reports and Dashboards with Interactivity, Sharing, Native Excel Exports, App Integration & more Get technology previously reserved for billion-dollar corporations, FREE http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk _______________________________________________ Oscar-users mailing list Oscar-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oscar-users