On Tue, Feb 09, 2010 at 07:38:29AM +0000, Nigel Kukard wrote: > > >>>>>> Apon upgrading to 2.6.29.6 from 2.6.22 and recompiling ipvsadm 1.25 to > >>>>>> get ipv6 support I'm getting the following error: > >>>>>> > >>>>>> Invalid operation. Possibly wrong module version, address not > >>>>>> unicast, ... > >>>>>> > >>>>>> when running ... > >>>>>> > >>>>>> ipvsadm -ln -t SERVICE_HERE > >>>>>> > >>>>>> There is absolutely nothing else that changed apart from the kernel and > >>>>>> recompile of 1.25. > >>>>>> > >>>>>> Everything seems to work fine, ipvsadm -ln works fine, setting up > >>>>>> services works fine as does all functionality except the above. > >>>>>> > >>>>>> Can anyone shed some light? > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>> Hi Nigel, > >>>>> > >>>>> I'm not having any luck reproducing this problem. > >>>>> Is sane output produced when you run "ipvsadm -ln" ? > >>>>> Could you be more specific about what SERVICE_HERE is, > >>>>> and if it includes a host name if it resolves to an ipv4 > >>>>> or ipv6 address, or both? > >>>>> > >>>>> Also, is the 2.6.29.6 vanilla, or has it had some patches added? > >>>>> > >>>>> > >>>>> > >>>> 2.6.29.6 vanilla > >>>> > >>>> The problem above was due to some CFLAGS being set in the environment, > >>>> namely "-O2 --march=pentium-mmx". Without the env being set, there are > >>>> no more errors. I did a total rm -rf sources and recompile. > >>>> > >>>> Which brings me to another problem now... my ipvsadm -ln is not > >>>> displaying the current number of connections. I've copied the same > >>>> binary over to 2.6.29.5 and its working. Using the binary on 2.6.29.6 > >>>> seems to just give zero's. Also statement in the original post was > >>>> incorrect, -ln did work, but also didn't display the number of > >>>> connections. > >>>> > >>>> # ipvsadm -v > >>>> ipvsadm v1.25 2008/5/15 (compiled with popt and IPVS v1.2.1) > >>>> > >>>> # ipvsadm -ln -t a.b.c.d:25 > >>>> Prot LocalAddress:Port Scheduler Flags > >>>> -> RemoteAddress:Port Forward Weight ActiveConn InActConn > >>>> TCP a.b.c.d:25 wlc > >>>> -> 10.0.200.131:25 Masq 240 0 75 > >>>> -> 10.0.200.141:25 Masq 239 0 69 > >>>> -> 10.0.200.151:25 Masq 238 0 69 > >>>> -> 10.0.200.161:25 Masq 239 0 67 > >>>> > >>>> # ipvsadm -ln -t a.b.c.d:25 --stats > >>>> Prot LocalAddress:Port Conns InPkts OutPkts InBytes > >>>> OutBytes > >>>> -> RemoteAddress:Port > >>>> TCP a.b.c.d:25 1516 37220 32114 31690896 1836300 > >>>> -> 10.0.200.131:25 404 13002 10258 10433390 > >>>> 556410 > >>>> -> 10.0.200.141:25 360 7119 6485 5986697 > >>>> 392303 > >>>> -> 10.0.200.151:25 390 9374 8454 8919481 > >>>> 485851 > >>>> -> 10.0.200.161:25 362 7726 6918 6351400 > >>>> 401884 > >>>> > >>>> # ipvsadm -ln -t a.b.c.d:25 --rate > >>>> Prot LocalAddress:Port CPS InPPS OutPPS InBPS > >>>> OutBPS > >>>> -> RemoteAddress:Port > >>>> TCP a.b.c.d:25 6 96 92 69714 5619 > >>>> -> 10.0.200.131:25 1 14 16 > >>>> 3177 1108 > >>>> -> 10.0.200.141:25 2 38 32 > >>>> 30077 1911 > >>>> -> 10.0.200.151:25 1 21 21 > >>>> 14418 1274 > >>>> -> 10.0.200.161:25 1 24 23 > >>>> 22041 1325 > >>>> > >>>> # ipvsadm -ln -c | grep ESTABLISHED | wc -l > >>>> 65 > >>>> > >>>> > >>>> > >>> Ok ... I found the problem. > >>> > >>> * My scripts overrode CFLAGS= which is why it worked before, -D for NL > >>> was being forcefully removed. > >>> * I fixed this a few weeks ago, when upgrading .. same time I upgraded > >>> my kernel, appears that libnl is now used. > >>> * Below ... > >>> > >>> make HAVE_NL=0 POPT_LIB="-lpopt" <= that works fine, I get the correct > >>> listing below... > >>> TCP a.b.c.d:25 wlc > >>> -> 10.0.200.131:25 Masq 238 14 47 > >>> -> 10.0.200.141:25 Masq 241 14 36 > >>> -> 10.0.200.151:25 Masq 242 10 109 > >>> -> 10.0.200.161:25 Masq 242 13 90 > >>> > >>> make POPT_LIB="-lpopt" <= that gives wrong details for ActiveConn, below > >>> ... > >>> TCP a.b.c.d:25 wlc > >>> -> 10.0.200.131:25 Masq 240 0 31 > >>> -> 10.0.200.141:25 Masq 240 0 34 > >>> -> 10.0.200.151:25 Masq 241 0 94 > >>> -> 10.0.200.161:25 Masq 238 0 87 > >>> > >>> Same results on all kernel versions & servers I've tried. > >>> > >>> My problem is that using HAVE_NL=0 seems not to work with v6 support... > >>> /tmp/ipvsadm.no-nl -A -t [fc00::10]:25 -s rr > >>> Operation not supported with specified address family > >>> > >>> > >> That bit is expected, the NL support was added specifically > >> because the non-NL support couldn't be expanded to handle ipv6. > >> So as you need ipv6 you also need NL. And curiously that seems > >> to result in the connection count being bogus. > >> > >> Would it be possible for you to test this against a newer kernel - say > >> 2.6.32 or 2.6.33-rc4. It would be good to know if the problem is > >> still present. > >> > > I can with no doubt confirm the problem is still present in 2.6.33-rc5. > > I now have a test network setup especially for this. > > I've tested this out now on quite a number of systems with different > kernel versions and at least to me its 100% reproduceable. > > Simon, you had any luck your side finding what the problem may be?
Sorry, no. I've been super-busy of late. > I've also noticed an issue with a 64bit kernel and 32bit compiled > userspace, not sure if this is related. Interesting, could you give some more details? _______________________________________________ Please read the documentation before posting - it's available at: http://www.linuxvirtualserver.org/ LinuxVirtualServer.org mailing list - [email protected] Send requests to [email protected] or go to http://lists.graemef.net/mailman/listinfo/lvs-users
