On 02/09/10 09:39, Simon Horman wrote: > On Tue, Feb 09, 2010 at 07:38:29AM +0000, Nigel Kukard wrote: > >> >>>>>>>> Apon upgrading to 2.6.29.6 from 2.6.22 and recompiling ipvsadm 1.25 to >>>>>>>> get ipv6 support I'm getting the following error: >>>>>>>> >>>>>>>> Invalid operation. Possibly wrong module version, address not >>>>>>>> unicast, ... >>>>>>>> >>>>>>>> when running ... >>>>>>>> >>>>>>>> ipvsadm -ln -t SERVICE_HERE >>>>>>>> >>>>>>>> There is absolutely nothing else that changed apart from the kernel and >>>>>>>> recompile of 1.25. >>>>>>>> >>>>>>>> Everything seems to work fine, ipvsadm -ln works fine, setting up >>>>>>>> services works fine as does all functionality except the above. >>>>>>>> >>>>>>>> Can anyone shed some light? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> Hi Nigel, >>>>>>> >>>>>>> I'm not having any luck reproducing this problem. >>>>>>> Is sane output produced when you run "ipvsadm -ln" ? >>>>>>> Could you be more specific about what SERVICE_HERE is, >>>>>>> and if it includes a host name if it resolves to an ipv4 >>>>>>> or ipv6 address, or both? >>>>>>> >>>>>>> Also, is the 2.6.29.6 vanilla, or has it had some patches added? >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> 2.6.29.6 vanilla >>>>>> >>>>>> The problem above was due to some CFLAGS being set in the environment, >>>>>> namely "-O2 --march=pentium-mmx". Without the env being set, there are >>>>>> no more errors. I did a total rm -rf sources and recompile. >>>>>> >>>>>> Which brings me to another problem now... my ipvsadm -ln is not >>>>>> displaying the current number of connections. I've copied the same >>>>>> binary over to 2.6.29.5 and its working. Using the binary on 2.6.29.6 >>>>>> seems to just give zero's. Also statement in the original post was >>>>>> incorrect, -ln did work, but also didn't display the number of >>>>>> connections. >>>>>> >>>>>> # ipvsadm -v >>>>>> ipvsadm v1.25 2008/5/15 (compiled with popt and IPVS v1.2.1) >>>>>> >>>>>> # ipvsadm -ln -t a.b.c.d:25 >>>>>> Prot LocalAddress:Port Scheduler Flags >>>>>> -> RemoteAddress:Port Forward Weight ActiveConn InActConn >>>>>> TCP a.b.c.d:25 wlc >>>>>> -> 10.0.200.131:25 Masq 240 0 75 >>>>>> -> 10.0.200.141:25 Masq 239 0 69 >>>>>> -> 10.0.200.151:25 Masq 238 0 69 >>>>>> -> 10.0.200.161:25 Masq 239 0 67 >>>>>> >>>>>> # ipvsadm -ln -t a.b.c.d:25 --stats >>>>>> Prot LocalAddress:Port Conns InPkts OutPkts InBytes >>>>>> OutBytes >>>>>> -> RemoteAddress:Port >>>>>> TCP a.b.c.d:25 1516 37220 32114 31690896 1836300 >>>>>> -> 10.0.200.131:25 404 13002 10258 10433390 >>>>>> 556410 >>>>>> -> 10.0.200.141:25 360 7119 6485 5986697 >>>>>> 392303 >>>>>> -> 10.0.200.151:25 390 9374 8454 8919481 >>>>>> 485851 >>>>>> -> 10.0.200.161:25 362 7726 6918 6351400 >>>>>> 401884 >>>>>> >>>>>> # ipvsadm -ln -t a.b.c.d:25 --rate >>>>>> Prot LocalAddress:Port CPS InPPS OutPPS InBPS >>>>>> OutBPS >>>>>> -> RemoteAddress:Port >>>>>> TCP a.b.c.d:25 6 96 92 69714 5619 >>>>>> -> 10.0.200.131:25 1 14 16 >>>>>> 3177 1108 >>>>>> -> 10.0.200.141:25 2 38 32 >>>>>> 30077 1911 >>>>>> -> 10.0.200.151:25 1 21 21 >>>>>> 14418 1274 >>>>>> -> 10.0.200.161:25 1 24 23 >>>>>> 22041 1325 >>>>>> >>>>>> # ipvsadm -ln -c | grep ESTABLISHED | wc -l >>>>>> 65 >>>>>> >>>>>> >>>>>> >>>>>> >>>>> Ok ... I found the problem. >>>>> >>>>> * My scripts overrode CFLAGS= which is why it worked before, -D for NL >>>>> was being forcefully removed. >>>>> * I fixed this a few weeks ago, when upgrading .. same time I upgraded >>>>> my kernel, appears that libnl is now used. >>>>> * Below ... >>>>> >>>>> make HAVE_NL=0 POPT_LIB="-lpopt" <= that works fine, I get the correct >>>>> listing below... >>>>> TCP a.b.c.d:25 wlc >>>>> -> 10.0.200.131:25 Masq 238 14 47 >>>>> -> 10.0.200.141:25 Masq 241 14 36 >>>>> -> 10.0.200.151:25 Masq 242 10 109 >>>>> -> 10.0.200.161:25 Masq 242 13 90 >>>>> >>>>> make POPT_LIB="-lpopt" <= that gives wrong details for ActiveConn, below >>>>> ... >>>>> TCP a.b.c.d:25 wlc >>>>> -> 10.0.200.131:25 Masq 240 0 31 >>>>> -> 10.0.200.141:25 Masq 240 0 34 >>>>> -> 10.0.200.151:25 Masq 241 0 94 >>>>> -> 10.0.200.161:25 Masq 238 0 87 >>>>> >>>>> Same results on all kernel versions & servers I've tried. >>>>> >>>>> My problem is that using HAVE_NL=0 seems not to work with v6 support... >>>>> /tmp/ipvsadm.no-nl -A -t [fc00::10]:25 -s rr >>>>> Operation not supported with specified address family >>>>> >>>>> >>>>> >>>> That bit is expected, the NL support was added specifically >>>> because the non-NL support couldn't be expanded to handle ipv6. >>>> So as you need ipv6 you also need NL. And curiously that seems >>>> to result in the connection count being bogus. >>>> >>>> Would it be possible for you to test this against a newer kernel - say >>>> 2.6.32 or 2.6.33-rc4. It would be good to know if the problem is >>>> still present. >>>> >>>> >>> I can with no doubt confirm the problem is still present in 2.6.33-rc5. >>> I now have a test network setup especially for this. >>> >> I've tested this out now on quite a number of systems with different >> kernel versions and at least to me its 100% reproduceable. >> >> Simon, you had any luck your side finding what the problem may be? >> > Sorry, no. I've been super-busy of late. > > >> I've also noticed an issue with a 64bit kernel and 32bit compiled >> userspace, not sure if this is related. >> > Interesting, could you give some more details? >
Lets first get the NL issue fixed, then I'll do more testing and post my findings :) Is there anything else you want me to try from my side which would make finding the problem easier? -N _______________________________________________ Please read the documentation before posting - it's available at: http://www.linuxvirtualserver.org/ LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org Send requests to lvs-users-requ...@linuxvirtualserver.org or go to http://lists.graemef.net/mailman/listinfo/lvs-users