Thanks Lissa!
We have 3 service nodes, and plenty of RAM and CPU on them and the
MN. It will be a very rare occasion, if ever, that all of the
compute nodes need to be booted at once.
I tested out your commands, and am getting strange results now.
While it is updating the mypostscript file on the service node that
handles the compute node I am using for testing, it seems that it is
not using the mypostscript file from the service node when running
the otherpkgs postscript with updatenode. This is opposite what I
was seeing last time, so I am not sure what is going on. Is
otherpkgs treated differently than the other postscripts?
To make sure that it's actually reading the mypostscript file from
the management node instead of the service node, I manually edited
the OTHERPKGDIR variable contents to have a "45" at the end, then
ran "updatenode c100n1 -P otherpkgs". It is indeed reading it from
the management node:
[root@master v5.4.8-dl-compute]# updatenode c100n1 -P otherpkgs
c100n1: Tue Jan 21 09:18:39 CST 2014 Running postscript: otherpkgs
c100n1: NFSSERVER=master
c100n1:
OTHERPKGDIR=master/install/post/otherpkgs/centos5.4-kernel_three-seven-one/x86_64/v5.4.8-dl-compute45
I also edited the OTHERPKGDIR within the osimage to have a "2" at
the end to make sure it's getting pushed out properly to the service
nodes, and it is:
[root@master v5.4.8-dl-compute]# lsdef -t osimage v5.4.8-dl-compute
Object name: v5.4.8-dl-compute
<snip>
otherpkgdir=/install/post/otherpkgs/centos5.4-kernel_three-seven-one/x86_64/v5.4.8-dl-compute2
And on the service node:
[root@servicenode01 mypostscripts]# grep OTHERPKGDIR
mypostscript.c100n1
OTHERPKGDIR='/install/post/otherpkgs/centos5.4-kernel_three-seven-one/x86_64/v5.4.8-dl-compute2'
Here's an lsdef of the node in question:
[root@master v5.4.8-dl-compute]#
[root@master v5.4.8-dl-compute]# lsdef c100n1
Object name: c100n1
arch=x86_64
chain=osimage
cons=cyclades
currchain=osimage
currstate=netboot
centos5.4-kernel_three-seven-one-x86_64-v5.4.8-dl-compute
groups=compute,c100,all
initrd=xcat/osimage/v5.4.8-dl-compute/initrd-stateless.gz
kcmdline=imgurl=http://master:80//install/netboot/centos5.4-kernel_three-seven-one/x86_64/v5.4.8-dl-compute/rootimg.sfs
XCAT=master:3001 NODE=c100n1 FC=0 ifname=eth0:00:E0:81:32:71:36
netdev=eth0 console=tty0 console=ttyS1,9600
kernel=xcat/osimage/v5.4.8-dl-compute/kernel
mac=00:E0:81:32:71:36
netboot=xnba
os=centos5.4-kernel_three-seven-one
postscripts=remoteshell
power=rackable
primarynic=eth0
profile="">
provmethod=v5.4.8-dl-compute
serialport=1
serialspeed=9600
servicenode=servicenode01
status=booted
statustime=01-20-2014 17:26:34
termport=1
termserver=c100t1
updatestatus=failed
updatestatustime=01-21-2014 09:18:47
xcatmaster=master
Any ideas why it's utilizing the mypostscript file from the headnode
instead of the servicenode when running otherpkgs?
On 1/21/2014 7:51 AM, Lissa Valletta
wrote:
How many service nodes do you
have? I hope you have enough memory, cpus etc on the
Management Node and Service nodes to handle 6000 nodes or
500-1000 nodes from each service node.
We also have some hints and
tips for Large Cluster tuning.
https://sourceforge.net/apps/mediawiki/xcat/index.php?title=Hints_and_Tips_for_Large_Scale_Clusters
updatenode to the node should
update the mypostscript file when you have
precreatemypostscripts=1. It sounds like something is
happening that that does not get downloaded to the nodes.
Try this set the attribute:
chtab key=precreatemypostscripts site.value=1
Run an simple updatenode to one
of your nodes.
updatenode compute-03 -P mytest
Look in on the MN
/tftpboot/mypostscripts
You should see a file
mypostscript.compute-03 where compute-03 is your nodename
It should change (date) each
time you run updatenode or nodeset. That is the file that is
put on the node and modified during updatenode and should
contain the latest changes. It will be modified and put in
/xcatpost/mypostscript
You do have to have /tftpboot
mounted on the Service nodes from the Management Node for
updatenode, if you are updating a compute node attached to a
service node. updatenode is actually running on the service
node in this case.
Lissa K. Valletta
8-3/B10
Poughkeepsie, NY 12601
(tie 293) 433-3102
 Russell Jones
---01/20/2014 09:36:26 PM---Thanks! It is working fine now
that I figured out what the issue is. We will
From: Russell Jones
<russell-l...@jonesmail.me>
To: xcat-user@lists.sourceforge.net,
Date: 01/20/2014 09:36 PM
Subject: Re: [xcat-user] xCAT ignoring my
otherpkgdir setting
Thanks!
It is working fine now that I figured out what the issue is. We
will gain a lot better performance with
site.precreatemypostscripts enabled, so we wish to keep it on.
This is a cluster with ~6000 nodes.
On 1/20/2014 7:32 PM, Xiao Peng Wang
wrote:
The question turns to whether
to use the site.precreatemypostscripts. This site attribute is
used to improve the performance of postscript mechanism in a
big cluster.
If site.precreatpostescripts is enabled, the mypostscript for
the node will be generated by nodeset command on xCAT MN. When
booting of node, the created mypostscript will be copied to
compute node directly for using. The advantage is it can save
the computing resource of MN during the node booting process,
the shortage is you must run nodeset for each changing which
related to postscripts.
If site.precreatescripts is disabled, nodeset command won't do
anything for mypostscirpt. When booting of node, it will try
to require the xCAT MN to generate the mypostscript at that
time.
So for your issue, you can try to disable the
site.precreatemypostscripts and to see whether it works well.
Thanks
Best Regards
----------------------------------------------------------------------
Wang Xiaopeng (王晓朋)
IBM China System Technology Laboratory
Tel: 86-10-82453455
Email: w...@cn.ibm.com
Address: 28,ZhongGuanCun Software Park,No.8 Dong Bei Wang West
Road, Haidian District Beijing P.R.China 100193
Russell Jones ---2014/01/18
03:40:35---I figured out what's going on - we are using the
site.precreatemypostscripts=1 option. Even though
From: Russell Jones <russell-l...@jonesmail.me>
To: xcat-user@lists.sourceforge.net,
Date: 2014/01/18
03:40
Subject: Re:
[xcat-user] xCAT ignoring my otherpkgdir setting
I figured out what's going on - we are using the
site.precreatemypostscripts=1 option. Even though the man page
on the site table states that the mypostscript file for the
node should be updated when an updatenode is ran as all as a
nodeset, this is not happening. I have to do a nodeset on the
node to have that file updated.
There's also a gap here in that since the node utilizes it's
local /xcatpost/mypostscript for pulling in variables that the
otherpkgs script needs, specifically $OTHERPKGDIR and
$OTHERPKGS1, you need to reboot your node or manually push the
new copy of this file to the nodes for your changes to
OTHERPKGDIR in the osimage to be seen by your node. In my
opinion, enhancements to updatenode need to be made so that:
A) updatenode rewrites the mypostscript file to capture any
changes you may have made to your osimage definitions
and
B) updatenode pushes this new mypostscript file to the node so
that updatenode will be aware of any osimage definition
changes.
On 1/17/2014 12:40 PM, Russell Jones wrote:
Well, genimage seems to pick up my
change just fine - it's just updatenode that doesn't. Here's
the temp yum.conf that is created during otherpkgs time with
genimage:
[otherpkgs1]
name=otherpkgs1
baseurl=file:///install/post/otherpkgs/centos5.4-new-kernel/x86_64/v5.4.8-dl-compute//.
gpgpcheck=0
[otherpkgs2]
name=otherpkgs2
baseurl=file:///install/post/otherpkgs/centos5.4-new-kernel/x86_64/v5.4.8-dl-compute//dell
gpgpcheck=0
We are wanting to separate out the otherpkgs based on image
name. Being able to configure otherpkgdir is a good feature
that seems to work fine during genimage time - I would
highly encourage you guys continue with this feature and
push it to the otherpkgs postscript.
On 1/17/2014 12:25 PM, Lissa Valletta wrote:
.
Take a look at this documentation , the otherpkgdir always
ends with the architecture directory and then the
directories under that contain the rpms. I have never seen
anything different so this may be a limitation in our code.
In other words in your case it should be otherpkgdir=/install/post/otherpkgs/centos5.4-new-kernel/x86_64
https://sourceforge.net/apps/mediawiki/xcat/index.php?title=XCAT_iDataPlex_Cluster_Quick_Start#Installing_Additional_Packages_Using_an_Otherpkgs_Pkglist
Lissa K. Valletta
8-3/B10
Poughkeepsie, NY 12601
(tie 293) 433-3102
Russell Jones ---01/17/2014
12:42:10 PM---Hi all, I am manually setting my otherpkgdir
for an osimage that a node belongs
From: Russell Jones
<russell-l...@jonesmail.me>
To: xCAT Users
Mailing list <xcat-user@lists.sourceforge.net>,
Date: 01/17/2014
12:42 PM
Subject: [xcat-user]
xCAT ignoring my otherpkgdir setting
Hi all,
I am manually setting my otherpkgdir for an osimage that a
node belongs to. However when running "updatenode $node -S",
the setting is being ignored. The problem is bolded below.
Example is below, how do I get this thing to look at the
right path?
[root@master v5.4.8-dl-compute]# chdef -t osimage
v5.4.8-dl-compute
otherpkgdir=/install/post/otherpkgs/centos5.4-new-kernel/x86_64/v5.4.8-dl-compute/
1 object definitions have been created or modified.
[root@master v5.4.8-dl-compute]# lsdef -t osimage
v5.4.8-dl-compute
Object name: v5.4.8-dl-compute
exlist=/install/custom/netboot/centos/v5.4.8-dl-compute.exlist
groups=all
imagetype=linux
kerneldir=/install/netboot/centos5.4-new-kernel/x86_64/v5.4.8-dl-compute
netdrivers=libphy,tg3,bnx2,e1000,dca,8021q,igb,forcedeth,squashfs,aufs,loop
nodebootif=eth0
osarch=x86_64
osdistroname=centos5.4-new-kernel-x86_64
osname=Linux
osvers=centos5.4-new-kernel
otherpkgdir=/install/post/otherpkgs/centos5.4-new-kernel/x86_64/v5.4.8-dl-compute/
otherpkglist=/install/custom/netboot/centos/v5.4.8-dl-compute.otherpkgs.pkglist
pkgdir=/install/centos5.4-new-kernel/x86_64/
pkglist=/install/custom/netboot/centos/v5.4.8-dl-compute.pkglist
postinstall=/install/custom/netboot/centos/v5.4.8-dl-compute.postinstall
postscripts=remoteshell,localrepo5,hardeths,postage.sh
profile="">
provmethod=netboot
rootimgdir=/install/netboot/centos5.4-new-kernel/x86_64/v5.4.8-dl-compute
[root@master v5.4.8-dl-compute]# lsdef c100n4
Object name: c100n4
<snip>
profile="">
provmethod=v5.4.8-dl-compute
[root@master v5.4.8-dl-compute]# updatenode c100n4 -S
Performing software maintenance operations. This could take
a while, if there are packages to install.
c100n4: Fri Jan 17 11:32:04 CST 2014 Running postscript:
ospkgs
c100n4: Postscript: ospkgs exited with code 0
c100n4: Fri Jan 17 11:32:57 CST 2014 Running postscript:
otherpkgs
c100n4: NFSSERVER=master
c100n4:
OTHERPKGDIR=master/install/post/otherpkgs/centos5.4-new-kernel/x86_64
c100n4: Loaded plugins: changelog, fastestmirror, keys,
list-data, tmprepo, verify
c100n4: Cleaning up Everything
c100n4: Loaded plugins: changelog, fastestmirror, keys,
list-data, tmprepo, verify
c100n4: Cleaning up Everything
c100n4: Cleaning up list of fastest mirrors
c100n4: yum -y upgrade
c100n4: Loaded plugins: changelog, fastestmirror, keys,
list-data, tmprepo, verify
c100n4: Loading mirror speeds from cached hostfile
c100n4: http://master/install/post/otherpkgs/centos5.4-new-kernel/x86_64/dell/repodata/repomd.xml: [Errno 14] HTTP Error 404: Not Found
c100n4: Trying other mirror.
c100n4: Error: Cannot retrieve repository metadata
(repomd.xml) for repository: xcat-otherpkgs0. Please verify
its path and try again
<snip>
c100n4: Postscript: otherpkgs exited with code 34
c100n4: Running of Software Maintenance has completed.
------------------------------------------------------------------------------
CenturyLink Cloud: The Leader in Enterprise Cloud
Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud
For
Critical Workloads, Development Environments &
Everything In Between.
Get a Quote or Start a Free Trial Today.
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk_______________________________________________
xCAT-user mailing list
xCAT-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xcat-user
------------------------------------------------------------------------------
CenturyLink Cloud: The Leader in Enterprise Cloud
Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud
For
Critical Workloads, Development Environments &
Everything In Between.
Get a Quote or Start a Free Trial Today.
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
_______________________________________________
xCAT-user mailing list
xCAT-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xcat-user
------------------------------------------------------------------------------
CenturyLink Cloud: The Leader in Enterprise Cloud
Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud
For
Critical Workloads, Development Environments &
Everything In Between.
Get a Quote or Start a Free Trial Today.
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
_______________________________________________
xCAT-user mailing list
xCAT-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xcat-user
------------------------------------------------------------------------------
CenturyLink Cloud: The Leader in Enterprise Cloud Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud For
Critical Workloads, Development Environments &
Everything In Between.
Get a Quote or Start a Free Trial Today.
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk_______________________________________________
xCAT-user mailing list
xCAT-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xcat-user
------------------------------------------------------------------------------
CenturyLink Cloud: The Leader in Enterprise Cloud Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud For
Critical Workloads, Development Environments &
Everything In Between.
Get a Quote or Start a Free Trial Today.
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
_______________________________________________
xCAT-user mailing list
xCAT-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xcat-user
------------------------------------------------------------------------------
CenturyLink Cloud: The Leader in Enterprise Cloud Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud For
Critical Workloads, Development Environments & Everything
In Between.
Get a Quote or Start a Free Trial Today.
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk_______________________________________________
xCAT-user mailing list
xCAT-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xcat-user
------------------------------------------------------------------------------
CenturyLink Cloud: The Leader in Enterprise Cloud Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud For
Critical Workloads, Development Environments & Everything In Between.
Get a Quote or Start a Free Trial Today.
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
_______________________________________________
xCAT-user mailing list
xCAT-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xcat-user
|
