Re: Shared /usr across linux guests - what about /var ?
On Tue, Nov 23, 2004 at 11:44:23AM -0800, Ranga Nathan wrote: > When installing s/w, some stuff goes into /var. Many of them temporary and > some not-so-temporary. > What to people do with /var when sharing /usr /var is intended to be unique to a particular system image, as is /etc. You don't want to try to share /var or /etc -- see Bill Scully's presentation on a elegant way to handle this that he developed for CA's development Linux systems. -- db -- For LINUX-390 subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390
Re: Shared /usr across linux guests - what about /var ?
Hi Tia, >When installing s/w, some stuff goes into /var. Many of them temporary and >some not-so-temporary. >What to people do with /var when sharing /usr You're pointing into an open gap in software management here. Although I found that many system programmers do have creative soloutoins for this one (really folks, I was impressed by the skills of our users) I believe that a sophisticated soloution is still missing. My favorite workaround is the following: Do access the shared stuff r+w on one machine and install the software, then use rpm -i --force on the other systems. This causes to rpm writing the parts on the wrieable non-shared filesystem while not aborting becuase of write errors on the read-only shared parts. We aim at solving this in the future... with kind regards Carsten Otte -- omnis enim res, quae dando non deficit, dum habetur et non datur, nondum habetur, quomodo habenda est -- For LINUX-390 subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390
Re: Shared /usr across linux guests - what about /var ?
__
Ranga Nathan / CSG
Systems Programmer - Specialist; Technical Services;
BAX Global Inc. Irvine-California
Tel: 714-442-7591 Fax: 714-442-2840
"Fargusson.Alan" <[EMAIL PROTECTED]>
Sent by: Linux on 390 Port <[EMAIL PROTECTED]>
11/23/2004 11:52 AM
Please respond to Linux on 390 Port
To: [EMAIL PROTECTED]
cc:
Subject: Re: Shared /usr across linux guests - what about
/var ?
This is due to developers not understanding what /var is for. I try to
get the vendors to move these files to /usr. I am usually not successful.
I agree. I would like to see it too. At the least, the software should
create files / directories if not already on /var. Then the sharing will
be worthwhile.
Realistically you have to copy things from /var on the system you
installed on over to the other systems /var.
Right, that is what I was going to do. Though that is not quite elegant,
some partial sharing would be helpful.
-Original Message-
From: Linux on 390 Port [mailto:[EMAIL PROTECTED] Behalf Of
Ranga Nathan
Sent: Tuesday, November 23, 2004 11:44 AM
To: [EMAIL PROTECTED]
Subject: Shared /usr across linux guests - what about /var ?
When installing s/w, some stuff goes into /var. Many of them temporary and
some not-so-temporary.
What to people do with /var when sharing /usr
TIA
__
Ranga Nathan / CSG
Systems Programmer - Specialist; Technical Services;
BAX Global Inc. Irvine-California
Tel: 714-442-7591 Fax: 714-442-2840
--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or
visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or
visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
Re: Shared /usr across linux guests - what about /var ?
This is due to developers not understanding what /var is for. I try to get the vendors to move these files to /usr. I am usually not successful. Realistically you have to copy things from /var on the system you installed on over to the other systems /var. -Original Message- From: Linux on 390 Port [mailto:[EMAIL PROTECTED] Behalf Of Ranga Nathan Sent: Tuesday, November 23, 2004 11:44 AM To: [EMAIL PROTECTED] Subject: Shared /usr across linux guests - what about /var ? When installing s/w, some stuff goes into /var. Many of them temporary and some not-so-temporary. What to people do with /var when sharing /usr TIA __ Ranga Nathan / CSG Systems Programmer - Specialist; Technical Services; BAX Global Inc. Irvine-California Tel: 714-442-7591 Fax: 714-442-2840 -- For LINUX-390 subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For LINUX-390 subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390
Re: Shared /usr across linux guests - what about /var ?
On Nov 23, 2004, at 1:44 PM, Ranga Nathan wrote: When installing s/w, some stuff goes into /var. Many of them temporary and some not-so-temporary. What to people do with /var when sharing /usr /var is machine-specific and writeable. This can cause problems in shared environments (/var/apt/cache, I'm lookin' at you). Adam -- For LINUX-390 subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390
Shared /usr across linux guests - what about /var ?
When installing s/w, some stuff goes into /var. Many of them temporary and some not-so-temporary. What to people do with /var when sharing /usr TIA __ Ranga Nathan / CSG Systems Programmer - Specialist; Technical Services; BAX Global Inc. Irvine-California Tel: 714-442-7591 Fax: 714-442-2840 -- For LINUX-390 subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390
Re: Shared /usr
On Fri, Sep 10, 2004 at 12:16:33PM -0700, Wolfe, Gordon W wrote: > This is one possible architecture. Whether it's recommended or not depends on why > you want to do it. > > The advantages are > 1) saving disk space. Depending on how expensive dasd is in your organization, this > can be considerable. > 2) Allowing minidisk cacheing to take place, reducing the number of physical I/O's > and speeding up response. > 3) keeping your users from installing programs or making modifications on their own > and then calling you at three in the morning when their server goes down. then you > find out after two hours of work that the problem is some modification they made. > 4) Creating a "standard" version of Linux that is easily deployable. > > The disadvantages are: > 1) Service is much more difficult. You have to install updates on a test server, > then compare before and after with tripwire to see what files were updated on /usr > and which were not. You have to route the non-/usr files around then swap /usr > disks and reboot. You end up having almost as many /usr disks with different > versions on them than you would have if everybody just had their own disk. I've got > 38 servers and 6 different shared /usr disks, not to mention 4 or 5 servers with > non-shared /usr. This ist true for a real disk. If you use DCSS in VM than VM takes of running the right Version for your Linux client. Then you do not need to shut down all client at one point in time, but one after the other. DCSS also saves a lot of I/O. If you share a disk, each linux client still generates I/O. With DCSS only the first linux client generates the I/O, the second just reference the same memory location. Have a look at http://oss.software.ibm.com/linux390/docu/lx26dcss00.pdf This explains DCSS in connection with linux detail. Regards Ihno > 2) you have altercations with users who want to write to the /usr disk. Usually you > can get around it by loop-mounting a subdirectory in /home over a /usr subdirectory. > Installing WebSphere with a read-only /usr is virtually impossible, as are other > program products. > > I'd say if all of your linux servers are essentially identical, shared /usr makes a > lot of sense. If they are all configured differently, question it. > > We've been using shared /usr for about three years. We are considering going to > individual read-write /usr areas with SLES9, just for the ease in maintenance. Disk > is cheap here. We bill our customers only $6.14 per gigabyte per month for 3390 > dasd storage. A full-pack 3390-3 for /usr is about 80% full and is about 2.2GB. > > Check out my presentation at SHARE on this topic at > http://linuxvm.org/present/SHARE101/S9343GWa.pdf > > So one elephant says to another, "You'll never believe what happened last night. I > was trying on Groucho Marx's pajamas--and he shot me!" > Gordon Wolfe, Ph.D. (425)865-5940 > VM Technical Services, The Boeing Company > > > ------ > > From: Doug Griswold > > Reply To: Linux on 390 Port > > Sent: Friday, September 10, 2004 11:24 AM > > To: [EMAIL PROTECTED] > > Subject: Shared /usr > > > > I have a question about sharing /usr with multiple vm guests. Is this a > > recommended acrchitecture? Are there any benefits to doing this other > > than saving space. It seems to me this could be problematic when > > applying fixes from yast. I welcome any input on this subject. > > > > > > > > Thanks, > > Doug > > > > -- > > For LINUX-390 subscribe / signoff / archive access instructions, > > send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit > > http://www.marist.edu/htbin/wlvindex?LINUX-390 > > > > > > -- > For LINUX-390 subscribe / signoff / archive access instructions, > send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit > http://www.marist.edu/htbin/wlvindex?LINUX-390 -- Best regards/Mit freundlichen Grüßen Ihno Krumreich "Never trust a computer you can lift." -- Ihno Krumreich[EMAIL PROTECTED] SuSE Linux AG Projectmanager S390 & zSeries Maxfeldstr. 5 +49-911-74053-439 D-90409 Nürnberg http://www.suse.de -- For LINUX-390 subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390
Re: Shared /usr
The maintenance is what would get me. We will have a customised set of packages for each installation because each linux install will have a different task. For our setup the maintenance overhead alone seems like enough to keep me from exploring it too much. Besides, you make it sound so fun >>> [EMAIL PROTECTED] 9/13/2004 12:11:14 PM >>> > I have a question about sharing /usr with multiple vm guests. Is this a > recommended acrchitecture? Are there any benefits to doing this other > than saving space. It seems to me this could be problematic when > applying fixes from yast. I welcome any input on this subject. I do this all the time (both PC class and mainframe class Linux). On the mainframe, you get some storage constraint relief which is theoretically important because you might be running thousands of Linux instances. In any case, you get tremendous installation and maint relief, with some substantial caveats. Any time you have shared filesystem storage (shared disks) and you then perform an installation or upgrade, there will be some pieces that fall into the shared storage and some that fall outside of the shared areas. Bringing these into sanity, between the "master" where you did the maint and the "slaves" (or ... give me a better term) :-S which access that shared storage, is a pain. You can re-run RPM on the sharers (the "slaves"), and you'll get a bazillion errors. You can probably ignore most of the errors. But are you really sure? Besides, it's inelegant. And what about customizations? You might need to distribute your own special config of whatever you installed or upgraded. Re-running RPM on all your sharers would probably not get your custom config kicked out to them. RPM does not deal with read-only volumes. IT WOULD BE NICE if it could/would check the file to be installed (into a R/O directory) and IFF the file to be placed there matches a file already there, ignore the fact that he (RPM) cannot write the file he wants to write. It's already there! Right? But if you can put up with stuff like this, then I recommend sharing /usr (and /opt). No sarcasm here: I really run this way all the time. It's great! -- R; -- For LINUX-390 subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For LINUX-390 subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390
Re: Shared /usr
> I have a question about sharing /usr with multiple vm guests. Is this a > recommended acrchitecture? Are there any benefits to doing this other > than saving space. It seems to me this could be problematic when > applying fixes from yast. I welcome any input on this subject. I do this all the time (both PC class and mainframe class Linux). On the mainframe, you get some storage constraint relief which is theoretically important because you might be running thousands of Linux instances. In any case, you get tremendous installation and maint relief, with some substantial caveats. Any time you have shared filesystem storage (shared disks) and you then perform an installation or upgrade, there will be some pieces that fall into the shared storage and some that fall outside of the shared areas. Bringing these into sanity, between the "master" where you did the maint and the "slaves" (or ... give me a better term) :-S which access that shared storage, is a pain. You can re-run RPM on the sharers (the "slaves"), and you'll get a bazillion errors. You can probably ignore most of the errors. But are you really sure? Besides, it's inelegant. And what about customizations? You might need to distribute your own special config of whatever you installed or upgraded. Re-running RPM on all your sharers would probably not get your custom config kicked out to them. RPM does not deal with read-only volumes. IT WOULD BE NICE if it could/would check the file to be installed (into a R/O directory) and IFF the file to be placed there matches a file already there, ignore the fact that he (RPM) cannot write the file he wants to write. It's already there! Right? But if you can put up with stuff like this, then I recommend sharing /usr (and /opt). No sarcasm here: I really run this way all the time. It's great! -- R; -- For LINUX-390 subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390
Re: Shared /usr
Thanks for the reply. This info will help me make the decision on wether to use shared /usr or not. -Doug >>> [EMAIL PROTECTED] 9/10/2004 3:16:33 PM >>> This is one possible architecture. Whether it's recommended or not depends on why you want to do it. The advantages are 1) saving disk space. Depending on how expensive dasd is in your organization, this can be considerable. 2) Allowing minidisk cacheing to take place, reducing the number of physical I/O's and speeding up response. 3) keeping your users from installing programs or making modifications on their own and then calling you at three in the morning when their server goes down. then you find out after two hours of work that the problem is some modification they made. 4) Creating a "standard" version of Linux that is easily deployable. The disadvantages are: 1) Service is much more difficult. You have to install updates on a test server, then compare before and after with tripwire to see what files were updated on /usr and which were not. You have to route the non-/usr files around then swap /usr disks and reboot. You end up having almost as many /usr disks with different versions on them than you would have if everybody just had their own disk. I've got 38 servers and 6 different shared /usr disks, not to mention 4 or 5 servers with non-shared /usr. 2) you have altercations with users who want to write to the /usr disk. Usually you can get around it by loop-mounting a subdirectory in /home over a /usr subdirectory. Installing WebSphere with a read-only /usr is virtually impossible, as are other program products. I'd say if all of your linux servers are essentially identical, shared /usr makes a lot of sense. If they are all configured differently, question it. We've been using shared /usr for about three years. We are considering going to individual read-write /usr areas with SLES9, just for the ease in maintenance. Disk is cheap here. We bill our customers only $6.14 per gigabyte per month for 3390 dasd storage. A full-pack 3390-3 for /usr is about 80% full and is about 2.2GB. Check out my presentation at SHARE on this topic at http://linuxvm.org/present/SHARE101/S9343GWa.pdf So one elephant says to another, "You'll never believe what happened last night. I was trying on Groucho Marx's pajamas--and he shot me!" Gordon Wolfe, Ph.D. (425)865-5940 VM Technical Services, The Boeing Company > -- > From: Doug Griswold > Reply To: Linux on 390 Port > Sent: Friday, September 10, 2004 11:24 AM > To: [EMAIL PROTECTED] > Subject: Shared /usr > > I have a question about sharing /usr with multiple vm guests. Is this a > recommended acrchitecture? Are there any benefits to doing this other > than saving space. It seems to me this could be problematic when > applying fixes from yast. I welcome any input on this subject. > > > > Thanks, > Doug > > -- > For LINUX-390 subscribe / signoff / archive access instructions, > send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit > http://www.marist.edu/htbin/wlvindex?LINUX-390 > > -- For LINUX-390 subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For LINUX-390 subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390
Re: Shared /usr
*** Reply to note of Fri, 10 Sep 2004 12:16:33 -0700 (MST/PDT) *** by [EMAIL PROTECTED] Using shared disks also helps if you are/want to use linux DCSS support. Then, you have the same problems as a VM's Y-DISK, just much bigger disks and segments ... sal "Wolfe, Gordon W" <[EMAIL PROTECTED]> writes: >This is one possible architecture. Whether it's recommended or not = >depends on why you want to do it. > >The advantages are=20 >1) saving disk space. Depending on how expensive dasd is in your = >organization, this can be considerable. >2) Allowing minidisk cacheing to take place, reducing the number of = >physical I/O's and speeding up response. =20 >3) keeping your users from installing programs or making modifications = >on their own and then calling you at three in the morning when their = >server goes down. then you find out after two hours of work that the = >problem is some modification they made. >4) Creating a "standard" version of Linux that is easily deployable. > >The disadvantages are: >1) Service is much more difficult. You have to install updates on a = >test server, then compare before and after with tripwire to see what = >files were updated on /usr and which were not. You have to route the = >non-/usr files around then swap /usr disks and reboot. You end up = >having almost as many /usr disks with different versions on them than = >you would have if everybody just had their own disk. I've got 38 = >servers and 6 different shared /usr disks, not to mention 4 or 5 servers = >with non-shared /usr. >2) you have altercations with users who want to write to the /usr disk. = >Usually you can get around it by loop-mounting a subdirectory in /home = >over a /usr subdirectory. Installing WebSphere with a read-only /usr is = >virtually impossible, as are other program products. > >I'd say if all of your linux servers are essentially identical, shared = >/usr makes a lot of sense. If they are all configured differently, = >question it. > >We've been using shared /usr for about three years. We are considering = >going to individual read-write /usr areas with SLES9, just for the ease = >in maintenance. Disk is cheap here. We bill our customers only $6.14 = >per gigabyte per month for 3390 dasd storage. A full-pack 3390-3 for = >/usr is about 80% full and is about 2.2GB. > >Check out my presentation at SHARE on this topic at=20 >http://linuxvm.org/present/SHARE101/S9343GWa.pdf > >So one elephant says to another, "You'll never believe what happened = >last night. I was trying on Groucho Marx's pajamas--and he shot me!"=20 >Gordon Wolfe, Ph.D. (425)865-5940 >VM Technical Services, The Boeing Company > >> -- >> From: Doug Griswold >> Reply To: Linux on 390 Port >> Sent: Friday, September 10, 2004 11:24 AM >> To: [EMAIL PROTECTED] >> Subject: Shared /usr >>=20 >> I have a question about sharing /usr with multiple vm guests. Is this = >a >> recommended acrchitecture? Are there any benefits to doing this other >> than saving space. It seems to me this could be problematic when >> applying fixes from yast. I welcome any input on this subject. >>=20 >>=20 >>=20 >> Thanks, >> Doug >>=20 >> -- >> For LINUX-390 subscribe / signoff / archive access instructions, >> send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 = >or visit >> http://www.marist.edu/htbin/wlvindex?LINUX-390 >>=20 >>=20 > >-- >For LINUX-390 subscribe / signoff / archive access instructions, >send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit >http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For LINUX-390 subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390
Re: Shared /usr
This is one possible architecture. Whether it's recommended or not depends on why you want to do it. The advantages are 1) saving disk space. Depending on how expensive dasd is in your organization, this can be considerable. 2) Allowing minidisk cacheing to take place, reducing the number of physical I/O's and speeding up response. 3) keeping your users from installing programs or making modifications on their own and then calling you at three in the morning when their server goes down. then you find out after two hours of work that the problem is some modification they made. 4) Creating a "standard" version of Linux that is easily deployable. The disadvantages are: 1) Service is much more difficult. You have to install updates on a test server, then compare before and after with tripwire to see what files were updated on /usr and which were not. You have to route the non-/usr files around then swap /usr disks and reboot. You end up having almost as many /usr disks with different versions on them than you would have if everybody just had their own disk. I've got 38 servers and 6 different shared /usr disks, not to mention 4 or 5 servers with non-shared /usr. 2) you have altercations with users who want to write to the /usr disk. Usually you can get around it by loop-mounting a subdirectory in /home over a /usr subdirectory. Installing WebSphere with a read-only /usr is virtually impossible, as are other program products. I'd say if all of your linux servers are essentially identical, shared /usr makes a lot of sense. If they are all configured differently, question it. We've been using shared /usr for about three years. We are considering going to individual read-write /usr areas with SLES9, just for the ease in maintenance. Disk is cheap here. We bill our customers only $6.14 per gigabyte per month for 3390 dasd storage. A full-pack 3390-3 for /usr is about 80% full and is about 2.2GB. Check out my presentation at SHARE on this topic at http://linuxvm.org/present/SHARE101/S9343GWa.pdf So one elephant says to another, "You'll never believe what happened last night. I was trying on Groucho Marx's pajamas--and he shot me!" Gordon Wolfe, Ph.D. (425)865-5940 VM Technical Services, The Boeing Company > -- > From: Doug Griswold > Reply To: Linux on 390 Port > Sent: Friday, September 10, 2004 11:24 AM > To: [EMAIL PROTECTED] > Subject: Shared /usr > > I have a question about sharing /usr with multiple vm guests. Is this a > recommended acrchitecture? Are there any benefits to doing this other > than saving space. It seems to me this could be problematic when > applying fixes from yast. I welcome any input on this subject. > > > > Thanks, > Doug > > -- > For LINUX-390 subscribe / signoff / archive access instructions, > send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit > http://www.marist.edu/htbin/wlvindex?LINUX-390 > > -- For LINUX-390 subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390
Shared /usr
I have a question about sharing /usr with multiple vm guests. Is this a recommended acrchitecture? Are there any benefits to doing this other than saving space. It seems to me this could be problematic when applying fixes from yast. I welcome any input on this subject. Thanks, Doug -- For LINUX-390 subscribe / signoff / archive access instructions, send email to [EMAIL PROTECTED] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390
Re: Strange DASD errors on shared /usr
Thanks to Jay Brenneman and Rob Van der Heij. This problem is solved. "You do not need a parachute to skydive. You only need a parachute to skydive twice." -Motto of the Darwin Society Gordon W. Wolfe, Ph.D. (425) 865-5940 VM Technical Services, The Boeing Company > -- > From: Robert J Brenneman > Reply To: Linux on 390 Port > Sent: Monday, April 29, 2002 10:37 AM > To: [EMAIL PROTECTED] > Subject: Re: Strange DASD errors on shared /usr > > I believe there is also a read only switch to be placed in the parmfile > for > dasd which is mounted under VM as read only. > Something like the following: > dasd=7080,7081(ro) root=/dev/dasda1 ro noinitrd > > that will tell the dasd driver that 7081 is really really read only, It > wont update acces times and what not. This would be in addition to the > line > in the fstab which states: > /dev/dasdb1 /usr ext2 ro 0 0 > > Note that if you have your dasd as a range like so: dasd=900-910(ro) > the (ro) applies to the entire range. So you may want to do something like > this: dasd=900-905,906-910(ro) > > You of course have to re run silo / zipl to make the changes stick , as > well as re-IPL the linux image. > > > Jay Brenneman > > > > > > > "Wolfe, Gordon W" >[EMAIL PROTECTED] > oeing.com> cc: > Sent by: Linux onSubject: Re: Strange DASD > errors on shared /usr > 390 Port > <[EMAIL PROTECTED] > IST.EDU> > > > 04/29/02 12:02 PM > Please respond to > Linux on 390 Port > > > > > > Thanks, Rob. > > I'll have to wait for the weekend to give this a try, but I think it will > solve the problem. > > "You do not need a parachute to skydive. You only need a parachute to > skydive twice." -Motto of the Darwin Society > Gordon W. Wolfe, Ph.D. (425) 865-5940 > VM Technical Services, The Boeing Company > > > > -- > > From: Rob van der Heij > > Reply To: Linux on 390 Port > > Sent: Saturday, April 27, 2002 1:24 AM > > To: [EMAIL PROTECTED] > > Subject: Re: Strange DASD errors on shared /usr > > > > >Indeed I do. First thing I checked. Here's my line out of /etc/fstab: > > > > > >/dev/dasdc1 /usr ext2ro 1 > > 0 > > > > You need to do both. On your LinuxA you > > mount /usr -o ro,remount > > sync;sync > > and then you can boot up the others provided they have the > > /etc/fstab as you show above. > > > > If you have not mounted /usr on LinuxA as ro before, the > > ext2 filesystem appears dirty to the other guests and they > > will try to e2fsck the disk before mounting, even ro. > > > > Rob > > > > > >
Re: Strange DASD errors on shared /usr
I believe there is also a read only switch to be placed in the parmfile for dasd which is mounted under VM as read only. Something like the following: dasd=7080,7081(ro) root=/dev/dasda1 ro noinitrd that will tell the dasd driver that 7081 is really really read only, It wont update acces times and what not. This would be in addition to the line in the fstab which states: /dev/dasdb1 /usr ext2 ro 0 0 Note that if you have your dasd as a range like so: dasd=900-910(ro) the (ro) applies to the entire range. So you may want to do something like this: dasd=900-905,906-910(ro) You of course have to re run silo / zipl to make the changes stick , as well as re-IPL the linux image. Jay Brenneman "Wolfe, Gordon W" cc: Sent by: Linux onSubject: Re: Strange DASD errors on shared /usr 390 Port <[EMAIL PROTECTED] IST.EDU> 04/29/02 12:02 PM Please respond to Linux on 390 Port Thanks, Rob. I'll have to wait for the weekend to give this a try, but I think it will solve the problem. "You do not need a parachute to skydive. You only need a parachute to skydive twice." -Motto of the Darwin Society Gordon W. Wolfe, Ph.D. (425) 865-5940 VM Technical Services, The Boeing Company > -- > From: Rob van der Heij > Reply To: Linux on 390 Port > Sent: Saturday, April 27, 2002 1:24 AM > To: [EMAIL PROTECTED] > Subject: Re: Strange DASD errors on shared /usr > > >Indeed I do. First thing I checked. Here's my line out of /etc/fstab: > > > >/dev/dasdc1 /usr ext2ro 1 > 0 > > You need to do both. On your LinuxA you > mount /usr -o ro,remount > sync;sync > and then you can boot up the others provided they have the > /etc/fstab as you show above. > > If you have not mounted /usr on LinuxA as ro before, the > ext2 filesystem appears dirty to the other guests and they > will try to e2fsck the disk before mounting, even ro. > > Rob > >
Re: Strange DASD errors on shared /usr
Thanks, Rob. I'll have to wait for the weekend to give this a try, but I think it will solve the problem. "You do not need a parachute to skydive. You only need a parachute to skydive twice." -Motto of the Darwin Society Gordon W. Wolfe, Ph.D. (425) 865-5940 VM Technical Services, The Boeing Company > -- > From: Rob van der Heij > Reply To: Linux on 390 Port > Sent: Saturday, April 27, 2002 1:24 AM > To: [EMAIL PROTECTED] > Subject: Re: Strange DASD errors on shared /usr > > >Indeed I do. First thing I checked. Here's my line out of /etc/fstab: > > > >/dev/dasdc1 /usr ext2ro 1 > 0 > > You need to do both. On your LinuxA you > mount /usr -o ro,remount > sync;sync > and then you can boot up the others provided they have the > /etc/fstab as you show above. > > If you have not mounted /usr on LinuxA as ro before, the > ext2 filesystem appears dirty to the other guests and they > will try to e2fsck the disk before mounting, even ro. > > Rob > >
Re: Strange DASD errors on shared /usr
>Indeed I do. First thing I checked. Here's my line out of /etc/fstab: > >/dev/dasdc1 /usr ext2ro 1 0 You need to do both. On your LinuxA you mount /usr -o ro,remount sync;sync and then you can boot up the others provided they have the /etc/fstab as you show above. If you have not mounted /usr on LinuxA as ro before, the ext2 filesystem appears dirty to the other guests and they will try to e2fsck the disk before mounting, even ro. Rob
Re: Strange DASD errors on shared /usr
Indeed I do. First thing I checked. Here's my line out of /etc/fstab: /dev/dasdc1 /usr ext2ro 1 0 "You do not need a parachute to skydive. You only need a parachute to skydive twice." -Motto of the Darwin Society Gordon W. Wolfe, Ph.D. (425) 865-5940 VM Technical Services, The Boeing Company > -- > From: Post, Mark K > Reply To: Linux on 390 Port > Sent: Friday, April 26, 2002 3:19 PM > To: [EMAIL PROTECTED] > Subject: Re: Strange DASD errors on shared /usr > > Gordon, > > You say you have the VM minidisks linked read-only. Do you have the Linux > filesystems mounted read-only as well? The command reject message makes > me > think that for some reason Linux is trying to write to the volume, and VM > isn't letting it. > > Mark Post > > -Original Message- > From: Wolfe, Gordon W [mailto:[EMAIL PROTECTED]] > Sent: Friday, April 26, 2002 5:40 PM > To: [EMAIL PROTECTED] > Subject: Strange DASD errors on shared /usr > > > I'm having a very strange problem. Hope someone can help. > > I have a number of Linux servers, all running SuSE SLES7 under VM. For > discussion purposes, let's call them LinuxA, LinuxB, LinuxC, LinuxD and so > on. All of our Linux servers have the /usr filesystem separated out on a > separate VM minidisk. > > LinuxA is a test server. It has its own /usr disk, the 294 disk accessed > r/w as /dev/dasdc . this server is used to build new subsystems and to > apply maintenance. > > LinuxB is basically a server that is logged off most of the time. It is > used as a server from which new servers are cloned. It has its own /usr > disk as 294 - /dev/dasdc1. (It actually has two /usr disks, the 194 and > the > 294. Both are linked read/only, but the 294 is actually used as /usr. 194 > isn't even in the parm file.) > > LinuxC and LinuxD are production servers. They do not have their own /usr > disk, but share the 194 disk of LinuxB, linked r/o as 294, as read-only > /usr > disks. The disk is linked in the CP directory read-only and mounted > read-only in /etc/fstab. > > I want to emphasize that at no time, ever, is the shared /usr disk EVER > linked read-write by any server. When I need to update /usr, I shut down > and log off both LinuxA and LinuxB (so that /usr is in clean shutdown > state) > and DDR LinuxA's 294 to LinuxB's 294. Then each of the production servers > is shut down, logged off and pointed to the LinuxB 294 (remember, LinuxB > is > shut down and logged off) and then rebooted. Eventually, when all > production servers have been switched, I rename 294 to 194 and 194 to 294 > to > start the cycle again. /usr disks should always be in the clean unmounted > mode during this copy. > > Now, here's the problem. On the production servers, on the VM console log > and in /var/log/messages, I frequently get a series of messages like > these: > > dasd_erp(3990): /dev/dasdc(94:8),0294@0x5:EXAMINE 24: Command Reject > detected - - fatal errorSense data: > dasd(eckd):device 0294 on irq 5: I/O status report: > dasd(eckd):in req: 027e0300 CS: 0x00 DS: 0x02 > dasd(eckd):Failing CCW: 027e03a0 > dasd(eckd):Sense(hex) 0- 7: 80 02 00 00 b6 f8 2a 00 > dasd(eckd):Sense(hex) 8-15: 00 00 00 ad 61 00 00 10 > dasd(eckd):Sense(hex) 16-23: 42 00 15 12 93 93 00 00 > dasd(eckd):Sense(hex) 24-31: 00 00 4c e2 00 02 f8 0a > dasd(eckd):24 Byte: 0 MSG 0, no MSGb to SYSOP > dasd_erp(3990): /dev/dasdc(94:8),0294@0x5:(EXAMINE) ERP chain report for > req: 027e0300 dasd_erp(3990): /dev/dasdc(94:8),0294@0x5:027e0300: c5c3d2c4 > > 027e0300 027e0300dasd_erp(3990): /dev/dasdc(94:8),0294@0x5:027e0310: > 004e1800 004ce800 > 027e0390 0302dasd_erp(3990): /dev/dasdc(94:8),0294@0x5:027e0320: > ff00 > 027e0370 dasd_erp(3990): /dev/dasdc(94:8),0294@0x5:027e0330: > > 011e 1a30dasd_erp(3990): /dev/dasdc(94:8),0294@0x5:027e0340: > b789dd2a 17f40483 > b789dd2a 17f41643dasd_erp(3990): /dev/dasdc(94:8),0294@0x5:027e0350: > > /dasdc(94:8),0294@0x5:Failed CCW (027e03a0) already > logged > or, dev 5e:09 (dasd), sector 1578096 > This may happen several times a day on each server. It doesn't seem to > hurt Linux processing at all, it just keeps going with no other errors. > But > it only happens on systems that have dasdc (/usr) mounted read-only. > > I've tried moving the dasd to a different volume. I've tried running > e2fsck > on it. I've even tried running ICKDSF surface checks against it. Nothing > effects it. this looks for all the world like either a hardware e
Re: Strange DASD errors on shared /usr
Gordon, You say you have the VM minidisks linked read-only. Do you have the Linux filesystems mounted read-only as well? The command reject message makes me think that for some reason Linux is trying to write to the volume, and VM isn't letting it. Mark Post -Original Message- From: Wolfe, Gordon W [mailto:[EMAIL PROTECTED]] Sent: Friday, April 26, 2002 5:40 PM To: [EMAIL PROTECTED] Subject: Strange DASD errors on shared /usr I'm having a very strange problem. Hope someone can help. I have a number of Linux servers, all running SuSE SLES7 under VM. For discussion purposes, let's call them LinuxA, LinuxB, LinuxC, LinuxD and so on. All of our Linux servers have the /usr filesystem separated out on a separate VM minidisk. LinuxA is a test server. It has its own /usr disk, the 294 disk accessed r/w as /dev/dasdc . this server is used to build new subsystems and to apply maintenance. LinuxB is basically a server that is logged off most of the time. It is used as a server from which new servers are cloned. It has its own /usr disk as 294 - /dev/dasdc1. (It actually has two /usr disks, the 194 and the 294. Both are linked read/only, but the 294 is actually used as /usr. 194 isn't even in the parm file.) LinuxC and LinuxD are production servers. They do not have their own /usr disk, but share the 194 disk of LinuxB, linked r/o as 294, as read-only /usr disks. The disk is linked in the CP directory read-only and mounted read-only in /etc/fstab. I want to emphasize that at no time, ever, is the shared /usr disk EVER linked read-write by any server. When I need to update /usr, I shut down and log off both LinuxA and LinuxB (so that /usr is in clean shutdown state) and DDR LinuxA's 294 to LinuxB's 294. Then each of the production servers is shut down, logged off and pointed to the LinuxB 294 (remember, LinuxB is shut down and logged off) and then rebooted. Eventually, when all production servers have been switched, I rename 294 to 194 and 194 to 294 to start the cycle again. /usr disks should always be in the clean unmounted mode during this copy. Now, here's the problem. On the production servers, on the VM console log and in /var/log/messages, I frequently get a series of messages like these: dasd_erp(3990): /dev/dasdc(94:8),0294@0x5:EXAMINE 24: Command Reject detected - - fatal errorSense data: dasd(eckd):device 0294 on irq 5: I/O status report: dasd(eckd):in req: 027e0300 CS: 0x00 DS: 0x02 dasd(eckd):Failing CCW: 027e03a0 dasd(eckd):Sense(hex) 0- 7: 80 02 00 00 b6 f8 2a 00 dasd(eckd):Sense(hex) 8-15: 00 00 00 ad 61 00 00 10 dasd(eckd):Sense(hex) 16-23: 42 00 15 12 93 93 00 00 dasd(eckd):Sense(hex) 24-31: 00 00 4c e2 00 02 f8 0a dasd(eckd):24 Byte: 0 MSG 0, no MSGb to SYSOP dasd_erp(3990): /dev/dasdc(94:8),0294@0x5:(EXAMINE) ERP chain report for req: 027e0300 dasd_erp(3990): /dev/dasdc(94:8),0294@0x5:027e0300: c5c3d2c4 027e0300 027e0300dasd_erp(3990): /dev/dasdc(94:8),0294@0x5:027e0310: 004e1800 004ce800 027e0390 0302dasd_erp(3990): /dev/dasdc(94:8),0294@0x5:027e0320: ff00 027e0370 dasd_erp(3990): /dev/dasdc(94:8),0294@0x5:027e0330: 011e 1a30dasd_erp(3990): /dev/dasdc(94:8),0294@0x5:027e0340: b789dd2a 17f40483 b789dd2a 17f41643dasd_erp(3990): /dev/dasdc(94:8),0294@0x5:027e0350: /dasdc(94:8),0294@0x5:Failed CCW (027e03a0) already logged or, dev 5e:09 (dasd), sector 1578096 This may happen several times a day on each server. It doesn't seem to hurt Linux processing at all, it just keeps going with no other errors. But it only happens on systems that have dasdc (/usr) mounted read-only. I've tried moving the dasd to a different volume. I've tried running e2fsck on it. I've even tried running ICKDSF surface checks against it. Nothing effects it. this looks for all the world like either a hardware error or a filesystem error but I can't find anything. LinuxA with the read-write /usr runs just fine without these errors, so I don't think it's a bad file. I'm not adept at reading CCW's or sense bytes. Anyone ever seen anything like this? "You do not need a parachute to skydive. You only need a parachute to skydive twice." -Motto of the Darwin Society Gordon W. Wolfe, Ph.D. (425) 865-5940 VM Technical Services, The Boeing Company
Strange DASD errors on shared /usr
I'm having a very strange problem. Hope someone can help. I have a number of Linux servers, all running SuSE SLES7 under VM. For discussion purposes, let's call them LinuxA, LinuxB, LinuxC, LinuxD and so on. All of our Linux servers have the /usr filesystem separated out on a separate VM minidisk. LinuxA is a test server. It has its own /usr disk, the 294 disk accessed r/w as /dev/dasdc . this server is used to build new subsystems and to apply maintenance. LinuxB is basically a server that is logged off most of the time. It is used as a server from which new servers are cloned. It has its own /usr disk as 294 - /dev/dasdc1. (It actually has two /usr disks, the 194 and the 294. Both are linked read/only, but the 294 is actually used as /usr. 194 isn't even in the parm file.) LinuxC and LinuxD are production servers. They do not have their own /usr disk, but share the 194 disk of LinuxB, linked r/o as 294, as read-only /usr disks. The disk is linked in the CP directory read-only and mounted read-only in /etc/fstab. I want to emphasize that at no time, ever, is the shared /usr disk EVER linked read-write by any server. When I need to update /usr, I shut down and log off both LinuxA and LinuxB (so that /usr is in clean shutdown state) and DDR LinuxA's 294 to LinuxB's 294. Then each of the production servers is shut down, logged off and pointed to the LinuxB 294 (remember, LinuxB is shut down and logged off) and then rebooted. Eventually, when all production servers have been switched, I rename 294 to 194 and 194 to 294 to start the cycle again. /usr disks should always be in the clean unmounted mode during this copy. Now, here's the problem. On the production servers, on the VM console log and in /var/log/messages, I frequently get a series of messages like these: dasd_erp(3990): /dev/dasdc(94:8),0294@0x5:EXAMINE 24: Command Reject detected - - fatal errorSense data: dasd(eckd):device 0294 on irq 5: I/O status report: dasd(eckd):in req: 027e0300 CS: 0x00 DS: 0x02 dasd(eckd):Failing CCW: 027e03a0 dasd(eckd):Sense(hex) 0- 7: 80 02 00 00 b6 f8 2a 00 dasd(eckd):Sense(hex) 8-15: 00 00 00 ad 61 00 00 10 dasd(eckd):Sense(hex) 16-23: 42 00 15 12 93 93 00 00 dasd(eckd):Sense(hex) 24-31: 00 00 4c e2 00 02 f8 0a dasd(eckd):24 Byte: 0 MSG 0, no MSGb to SYSOP dasd_erp(3990): /dev/dasdc(94:8),0294@0x5:(EXAMINE) ERP chain report for req: 027e0300 dasd_erp(3990): /dev/dasdc(94:8),0294@0x5:027e0300: c5c3d2c4 027e0300 027e0300dasd_erp(3990): /dev/dasdc(94:8),0294@0x5:027e0310: 004e1800 004ce800 027e0390 0302dasd_erp(3990): /dev/dasdc(94:8),0294@0x5:027e0320: ff00 027e0370 dasd_erp(3990): /dev/dasdc(94:8),0294@0x5:027e0330: 011e 1a30dasd_erp(3990): /dev/dasdc(94:8),0294@0x5:027e0340: b789dd2a 17f40483 b789dd2a 17f41643dasd_erp(3990): /dev/dasdc(94:8),0294@0x5:027e0350: /dasdc(94:8),0294@0x5:Failed CCW (027e03a0) already logged or, dev 5e:09 (dasd), sector 1578096 This may happen several times a day on each server. It doesn't seem to hurt Linux processing at all, it just keeps going with no other errors. But it only happens on systems that have dasdc (/usr) mounted read-only. I've tried moving the dasd to a different volume. I've tried running e2fsck on it. I've even tried running ICKDSF surface checks against it. Nothing effects it. this looks for all the world like either a hardware error or a filesystem error but I can't find anything. LinuxA with the read-write /usr runs just fine without these errors, so I don't think it's a bad file. I'm not adept at reading CCW's or sense bytes. Anyone ever seen anything like this? "You do not need a parachute to skydive. You only need a parachute to skydive twice." -Motto of the Darwin Society Gordon W. Wolfe, Ph.D. (425) 865-5940 VM Technical Services, The Boeing Company
