Re: Vinum configuration lost at vinum stop / start
Stijn Hoop wrote: Hi, On Thu, Nov 11, 2004 at 04:53:39PM +0200, Kim Helenius wrote: Stijn Hoop wrote: I don't know the state of affairs for 5.2.1-RELEASE, but in 5.3-RELEASE gvinum is the way forward. Thanks again for answering. Agreed, but there still seems to be a long way to go. A lot of 'classic' vinum functionality is still missing and at least for me it still doesn't do the job the way I would find trustworthy. See below. That's absolutely true. While 5.3 is IMHO pretty stable, gvinum is quite new and therefore a bit less well tested than the rest of the system. Fortunately Lukas Ertl, the maintainer of gvinum, is pretty active and responsive to problems. So if you need a critically stable vinum environment you would be better off with 4.x. I tested gvinum with some interesting results. First the whole system froze after creating a concatenated drive and trying to gvinum -rf -r objects (resetconfig command doesn't exist). That's not good. Nothing in dmesg? If you can consistently get this to happen you should send in a problem report. Next, I created the volume, newfs, copied some data on it. The rebooted, and issued gvinum start. This is what follows: 2 drives: D d1State: up /dev/ad4s1d A: 285894/286181 MB (99%) D d2State: up /dev/ad5s1d A: 285894/286181 MB (99%) 1 volume: V vinum0State: down Plexes: 1 Size:572 MB 1 plex: P vinum0.p0 C State: down Subdisks: 2 Size:572 MB 2 subdisks: S vinum0.p0.s0 State: staleD: d1 Size:286 MB S vinum0.p0.s1 State: staleD: d2 Size:286 MB I'm getting a bit confused. Issuing separately 'gvinum start vinum0' does seem to fix it (all states go 'up') but surely it should come up fine with just 'gvinum start'? This is how I would start it in loader.conf. I think I've seen this too, but while testing an other unrelated problem. At the time I attributed it to other factors. Can you confirm that when you restart again, it stays up? Or maybe try an explicit 'saveconfig' when it is in the 'up' state, and then reboot. Have you read http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/current-stable.html Particularly the 19.2.2 section, 'Staying stable with FreeBSD'? I have read it and used -stable in 4.x, and if I read it really carefully I figure out that -stable does not equal "stable" which is way I stopped tracking -stable in the first place. And when knowing I would only need it to fix raid5 init I'm a bit reluctant to do it as I found out I can't even create a concat volume correctly. That I can understand. If I may make a polite suggestion, it sounds like you value stability above all else. In this case where vinum is involved, I would recommend you to stay with 4.x until 5.4 is released. That should take another 6-8 months and probably most of the gvinum issues will have been tackled by then. Although I know that there are a lot of users, myself included, that run gvinum on 5.x, it is pretty new technology and therefore unfortunately includes pretty new bugs. The other option is to bite the bullet now, and fiddle with gvinum for a few days. Since other users are using it, it is certainly possible. This will take you some time however. It will save you time when the upgrade to 5.4 will be though. It is your decision what part of the tradeoff you like the most. HTH, --Stijn Stability is exactly what I'm looking for. However, I begin to doubt there's something strange going on with my setup. I mentioned gvinum freezing - there's indeed a fatal kernel trap message (page fault) on the console. Now, then, thinking of good old FreeBSD 4.x I decided to spend some more time on this issue. Ok... so I tested vinum with FreeBSD 4.10 and amazing things just keep happening. Like with 5.x, I create a small test concat volume with vinum. Newfs, mount, etc, everything works. Now, then, I issue the following commands: vinum stop, then vinum start. Fatal kernel trap -> automatic reboot. So, the root of the problem must lie deeper than (g)vinum in 5.x. More info on my 5.3 setup: Copyright (c) 1992-2004 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 5.3-RELEASE #1: Mon Nov 8 21:43:07 EET 2004 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/KIUKKU Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: AMD Athlon(TM) XP 1600+ (1400.06-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x662 Stepping = 2 Features=0x383f9ff AMD Features=0xc048 real memory = 536788992 (511 MB) avail memory = 519794688 (495 MB) npx0: [FAST] npx0: on motherboard npx0: INT 16 interface acpi0: on
Re: Vinum configuration lost at vinum stop / start
Stijn Hoop wrote: Greetings. I posted earlier about problems with vinum raid5 but it appears it's not restricted to that. Are you running regular vinum on 5.x? It is known broken. Please use 'gvinum' instead. There is one caveat: the gvinum that shipped with 5.3-RELEASE contains an error in RAID-5 initialization. If you really need RAID-5 you either need to wait for the first patch level release of 5.3, or you can build RELENG_5 from source yourself. The fix went in on 2004-11-07. Thank you for your answer. I tested normal concat with both 5.2.1-RELEASE and 5.3-RELEASE with similar results. Plenty of people (at least I get this impression after browsing several mailing lists and websites) have working vinum setups with 5.2.1 (where gvinum doesn't exist) so there's definately something I'm doing wrong here. So my problem is not limited to raid5. I don't know the state of affairs for 5.2.1-RELEASE, but in 5.3-RELEASE gvinum is the way forward. Thanks again for answering. Agreed, but there still seems to be a long way to go. A lot of 'classic' vinum functionality is still missing and at least for me it still doesn't do the job the way I would find trustworthy. See below. I'm aware of gvinum and the bug and actually tried to cvsup & make world last night but it didn't succeed due to some missing files in netgraph dirs. I will try again tonight. I tested gvinum with some interesting results. First the whole system froze after creating a concatenated drive and trying to gvinum -rf -r objects (resetconfig command doesn't exist). Next, I created the volume, newfs, copied some data on it. The rebooted, and issued gvinum start. This is what follows: 2 drives: D d1State: up /dev/ad4s1d A: 285894/286181 MB (99%) D d2State: up /dev/ad5s1d A: 285894/286181 MB (99%) 1 volume: V vinum0State: down Plexes: 1 Size:572 MB 1 plex: P vinum0.p0 C State: down Subdisks: 2 Size:572 MB 2 subdisks: S vinum0.p0.s0 State: staleD: d1 Size:286 MB S vinum0.p0.s1 State: staleD: d2 Size:286 MB I'm getting a bit confused. Issuing separately 'gvinum start vinum0' does seem to fix it (all states go 'up') but surely it should come up fine with just 'gvinum start'? This is how I would start it in loader.conf. OK, I think that will help you out. But the strange thing is, RELENG_5 should be buildable. Are you sure you are getting that? Have you read http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/current-stable.html Particularly the 19.2.2 section, 'Staying stable with FreeBSD'? I have read it and used -stable in 4.x, and if I read it really carefully I figure out that -stable does not equal "stable" which is way I stopped tracking -stable in the first place. And when knowing I would only need it to fix raid5 init I'm a bit reluctant to do it as I found out I can't even create a concat volume correctly. -- Kim Helenius [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Vinum configuration lost at vinum stop / start
On Thu, 11 Nov 2004, Stijn Hoop wrote: > On Thu, Nov 11, 2004 at 12:00:52PM +0200, Kim Helenius wrote: > > Greetings. I posted earlier about problems with vinum raid5 but it > > appears it's not restricted to that. > > Are you running regular vinum on 5.x? It is known broken. Please use > 'gvinum' instead. > > There is one caveat: the gvinum that shipped with 5.3-RELEASE contains an > error in RAID-5 initialization. If you really need RAID-5 you either need > to wait for the first patch level release of 5.3, or you can build > RELENG_5 from source yourself. The fix went in on 2004-11-07. Thank you for your answer. I tested normal concat with both 5.2.1-RELEASE and 5.3-RELEASE with similar results. Plenty of people (at least I get this impression after browsing several mailing lists and websites) have working vinum setups with 5.2.1 (where gvinum doesn't exist) so there's definately something I'm doing wrong here. So my problem is not limited to raid5. I'm aware of gvinum and the bug and actually tried to cvsup & make world last night but it didn't succeed due to some missing files in netgraph dirs. I will try again tonight. > > --Stijn > > -- > Fairy tales do not tell children that dragons exist. Children already > know dragons exist. Fairy tales tell children the dragons can be > killed. > -- G.K. Chesterton > ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Vinum configuration lost at vinum stop / start
On Thu, 11 Nov 2004, Artem Kazakov wrote: > Kim Helenius wrote: > > > Now I can newfs /dev/vinum/vinum0, mount it, use it, etc. But when I do > > vinum stop, vinum start, vinum stop, and vinum start something amazing > > happens. Vinum l after this is as follows: > > > > 2 drives: > > D d2State: up /dev/ad5s1d A: 286181/286181 > > MB (100%) > > D d1State: up /dev/ad4s1d A: 286181/286181 > > MB (100%) > > > > 0 volumes: > > 0 plexes: > > 0 subdisks: > > > > Where did my configuration go?! I can of course recreate it, with no > > data lost, but imagine this on a raid5 where the plex goes into init > > mode after creation. Not a pleasant scenario. Also recreating the > > configuration from a config file after every reboot doesn't sound > > interesting. > you shoud issue a read command to vinum to make it read configureation. > in your case: > vinum read /dev/ad5 /dev/ad4 Thank you for your answer. However, when I mentioned the configuration is lost, it is lost and not stored on the drives anymore. Thus 'vinum read' command cannot read it from the drives. In addition, 'vinum start' scans drives for vinum information so in fact 'vinum read' should not be needed. > -- > Kazakov Artem > > OOO "CompTek" > tel: +7(095) 785-2525, ext.1802 > fax: +7(095) 785-2526 > WWW:http://www.comptek.ru > ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Vinum configuration lost at vinum stop / start
Greetings. I posted earlier about problems with vinum raid5 but it appears it's not restricted to that: Let's make a fresh start with vinum resetconfig. Then vinum create kala.txt which contains: drive d1 device /dev/ad4s1d drive d2 device /dev/ad5s1d volume vinum0 plex org concat sd length 586096s drive d1 sd length 586096s drive d2 Output: 2 drives: D d1State: up /dev/ad4s1d A: 285894/286181 MB (99%) D d2State: up /dev/ad5s1d A: 285894/286181 MB (99%) 1 volumes: V vinum0State: up Plexes: 1 Size:572 MB 1 plexes: P vinum0.p0 C State: up Subdisks: 2 Size:572 MB 2 subdisks: S vinum0.p0.s0 State: up D: d1 Size:286 MB S vinum0.p0.s1 State: up D: d2 Size:286 MB Now I can newfs /dev/vinum/vinum0, mount it, use it, etc. But when I do vinum stop, vinum start, vinum stop, and vinum start something amazing happens. Vinum l after this is as follows: 2 drives: D d2State: up /dev/ad5s1d A: 286181/286181 MB (100%) D d1State: up /dev/ad4s1d A: 286181/286181 MB (100%) 0 volumes: 0 plexes: 0 subdisks: Where did my configuration go?! I can of course recreate it, with no data lost, but imagine this on a raid5 where the plex goes into init mode after creation. Not a pleasant scenario. Also recreating the configuration from a config file after every reboot doesn't sound interesting. -- Kim Helenius [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
A fundamental vinum raid5 problem
I have an absurd problem with vinum. I use the following command: vinum create blob.conf blob.conf: drive d1 device /dev/ad4s1d drive d2 device /dev/ad5s1d drive d3 device /dev/ad6s1d drive d4 device /dev/ad7s1d volume blob plex org raid5 433k sd length 0 drive d1 sd length 0 drive d2 sd length 0 drive d3 sd length 0 drive d4 After this I get the following output: 4 drives: D d1State: up /dev/ad4s1d A: 0/286181 MB (0%) D d2State: up /dev/ad5s1d A: 0/286181 MB (0%) D d3State: up /dev/ad6s1d A: 0/286181 MB (0%) D d4State: up /dev/ad7s1d A: 0/286181 MB (0%) 1 volumes: V blob State: down Plexes: 1 Size:838 GB 1 plexes: P blob.p0R5 State: init Subdisks: 4 Size:838 GB 4 subdisks: S blob.p0.s0State: emptyD: d1 Size:279 GB S blob.p0.s1State: emptyD: d2 Size:279 GB S blob.p0.s2State: emptyD: d3 Size:279 GB S blob.p0.s3State: emptyD: d4 Size:279 GB So far everything looks fine. Even vinum printconfig looks all right to me: # Vinum configuration of e26b.mylly.jyu.fi, saved at Wed Nov 10 23:49:11 2004 drive d1 device /dev/ad4s1d drive d2 device /dev/ad5s1d drive d3 device /dev/ad6s1d drive d4 device /dev/ad7s1d volume blob plex name blob.p0 org raid5 866s vol blob sd name blob.p0.s0 drive d1 plex blob.p0 len 586098408s driveoffset 265s plexoffset 0s sd name blob.p0.s1 drive d2 plex blob.p0 len 586098408s driveoffset 265s plexoffset 866s sd name blob.p0.s2 drive d3 plex blob.p0 len 586098408s driveoffset 265s plexoffset 1732s sd name blob.p0.s3 drive d4 plex blob.p0 len 586098408s driveoffset 265s plexoffset 2598s Well, then, I type vinum stop and vinum start. Now vinum printconfig shows: # Vinum configuration of e26b.mylly.jyu.fi, saved at Wed Nov 10 23:52:06 2004 drive d4 device /dev/ad7s1d drive d3 device /dev/ad6s1d drive d2 device /dev/ad5s1d drive d1 device /dev/ad4s1d Which means my config just disappeared. (Earlier I did vinum init to initialize the raid5 as recommended. I mounted, copied some data on it and rebooted. After reboot it was gone. Init took three hours so I decided to test the short way.) So next I tested a setup that would take a little less time: vinum -> concat -v /dev/ad4s1d /dev/ad5s1d volume vinum0 plex name vinum0.p0 org concat drive vinumdrive0 device /dev/ad4s1d sd name vinum0.p0.s0 drive vinumdrive0 size 0 drive vinumdrive1 device /dev/ad5s1d sd name vinum0.p0.s1 drive vinumdrive1 size 0 V vinum0State: up Plexes: 1 Size:558 GB P vinum0.p0 C State: up Subdisks: 2 Size:558 GB S vinum0.p0.s0 State: up D: vinumdrive0 Size:279 GB S vinum0.p0.s1 State: up D: vinumdrive1 Size:279 GB newfs /dev/vinum/vinum0 ... Mounted and seems to work. Vinum stop, vinum start... gone. I can recreate it with the concat command but surely this is not how it's meant to be? I've actually created a concat vinum volume on another install earlier and it works. Current setup: FreeBSD e26b.mylly.jyu.fi 5.3-RELEASE FreeBSD 5.3-RELEASE #1: Mon Nov 8 21:43:07 EET 2004 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/KIUKKU i386 I've read about problems with vinum in 5.3 (already got the dangling vnode panic and lost some data in the following confusion) but I would be very satisfied if I could start vinum and mount the volume with a script after boot process. But why is the configuration disappearing? vinum saveconfig doesn't help either. Thanks in advance, -- Kim Helenius [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Puzzling NATD problem - revisited
Thank you both for your answers. The campus network uses public ip address space, sorry for not including that information. The fact why I included it in between the internet and the natd gateway is that if there's some weirdness in it, I somehow have to compensate for it in FreeBSD. As I stated, Linux users haven't had any problems with nat in the same network. Even I had working nat in the same network two years ago (on FreeBSD 4.1-4.3 I think) so I'm trying to pinpoint the cause for this extremely peculiar behaviour. Josh Paetzel wrote: >On Tue, Oct 08, 2002 at 03:28:28PM -0400, JoeB wrote: > > >>You state Network topology: >>Internet---Campus Network---(xl0)FreeBSD NATD machine(xl1)---Internal host >> >>Internet is public ip address, if Campus Network private ip address then >>you can not nat them again, if Campus Network is public ip address then you >>should nat x11 for the private ip address on the lan behind the FBSD box. >> >> >That's not correct. I've seen two layers of NATD work just fine in an office >building environment where the gateway to the office was natting ips to the >individual clients, and then clients were natting again to hang multiple >machines off the one ip they got from the office gateway. > >Josh > > "You should nat x11 for the private ip address on the lan behind the FBSD box." I always thought natd should run on the external interface? How can natd work perfectly if I'm running it on a wrong interface? > > >>-Original Message- >>From: [EMAIL PROTECTED] >>[mailto:[EMAIL PROTECTED]]On Behalf Of Kim Helenius >>Sent: Tuesday, October 08, 2002 9:13 AM >>To: [EMAIL PROTECTED] >>Subject: Puzzling NATD problem - revisited >> >>The setting: >> >>Network topology: >>Internet---Campus Network---(xl0)FreeBSD NATD machine(xl1)---Internal host >> >>A custom kernel build including the following options: >>options IPFIREWALL >>options IPDIVERT >>Used the command: >>sysctl net.inet.ip.forwarding=1 >>And started natd with natd -interface xl0 >> >>Then did, straight from the manpage, the following firewall rules: >>/sbin/ipfw -f flush >>/sbin/ipfw add divert natd all from any to any via xl0 >>/sbin/ipfw add pass all from any to any >> >>Now NAT works perfectly for the internal host, but (almost) all TCP >>connections cease to work to/from the NATD machine. AFAIK UDP and ICMP work >>perfectly. I've tried this on two different FreeBSD machines in the same >>network with identical results. If I remove the divert rule, everything >>works perfectly, except of course for the NAT. There have been no similar, >>puzzling effects on any Linux hosts I know of in the same network. Therefore >>I'm sure there's some knob I haven't pushed yet :) >> >>I'm aware this doesn't make much of a firewall but I'd like to get natd >>working before I run the firewall script. >> >>-- >>Kim Helenius >>[EMAIL PROTECTED] >> >> >> >>To Unsubscribe: send mail to [EMAIL PROTECTED] >>with "unsubscribe freebsd-questions" in the body of the message >> >> >>To Unsubscribe: send mail to [EMAIL PROTECTED] >>with "unsubscribe freebsd-questions" in the body of the message >> >> -- Kim Helenius [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-questions" in the body of the message
Puzzling NATD problem - revisited
The setting: Network topology: Internet---Campus Network---(xl0)FreeBSD NATD machine(xl1)---Internal host A custom kernel build including the following options: options IPFIREWALL options IPDIVERT Used the command: sysctl net.inet.ip.forwarding=1 And started natd with natd -interface xl0 Then did, straight from the manpage, the following firewall rules: /sbin/ipfw -f flush /sbin/ipfw add divert natd all from any to any via xl0 /sbin/ipfw add pass all from any to any Now NAT works perfectly for the internal host, but (almost) all TCP connections cease to work to/from the NATD machine. AFAIK UDP and ICMP work perfectly. I've tried this on two different FreeBSD machines in the same network with identical results. If I remove the divert rule, everything works perfectly, except of course for the NAT. There have been no similar, puzzling effects on any Linux hosts I know of in the same network. Therefore I'm sure there's some knob I haven't pushed yet :) I'm aware this doesn't make much of a firewall but I'd like to get natd working before I run the firewall script. -- Kim Helenius [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-questions" in the body of the message
Puzzling NATD problem
The setting: Network topology: Internet---Campus Network---FreeBSD NATD machine---Internal host A custom kernel build including the following options: options IPFIREWALL options IPDIVERT Used the command: sysctl net.inet.ip.forwarding=1 And started natd with natd -interface xl0 Then did, straight from the manpage, the following firewall rules: /sbin/ipfw -f flush /sbin/ipfw add divert natd all from any to any via xl0 /sbin/ipfw add pass all from any to any Now NAT works perfectly for the internal host, but (almost) all TCP connections cease to work to/from the NATD machine. AFAIK UDP and ICMP work perfectly. I've tried this on two different FreeBSD machines in the same network with identical results. If I remove the divert rule, everything works perfectly, except of course for the NAT. There have been no similar, puzzling effects on any Linux hosts I know of in the same network. I'm sure there's some knob I haven't pushed yet :) I'm aware this doesn't make much of a firewall but I'd like to get natd working before I run the firewall script. -- Kim Helenius [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-questions" in the body of the message
