Re: [Vserver] netstat -nap --inet kills vserver kernel

2004-10-12 Thread Herbert Poetzl 
On Tue, Oct 12, 2004 at 09:27:09AM -0300, James MacLean wrote:
 James MacLean wrote:
 
 Herbert Poetzl wrote:
 
 On Thu, Sep 16, 2004 at 11:49:51AM -0300, James MacLean wrote:
  
 
 okay could you please go back in steps of 4 bytes as
 described above .. c02ae988, c02ae984, c02ae980 ...
 until you hit 'something' in net/ipv4/tcp_ipv4.c?
 
 
 
 addr2line -e /usr/src/linux/vmlinux c02ae988
 net/ipv4/tcp_ipv4.c:2185
   
 
 
 okay, please try the latest patch and let me know
 if this fixes your issues:
 
 http://vserver.13thfloor.at/Experimental/patch-2.6.9-rc2-vs1.9.2.28.diff
 
 TIA,
 Herbert
  
 
 #!/usr/bin/perl
 $counter = 0;
 while($counter  1000) {
  print `netstat -nap --inet`;
  $counter++;
 }
 
 Did not blow it up. So far much better :). I will be pushing it more 
 over the next while so if it goes I'll be back :).
 
 Now... just be be critical of this service. You must realize that if 
 you are able to fix kernel crashes within 3 hours of having the 
 correct information you are likely to set some very high expectations 
 :). Snicker snicker :).
 
 Thanks a bunch,
 JES
 
 Hi Folks (Herbert),
 
 Noticed just recently that netstat on the main server or context 1 does 
 not show up my Apache process/ports (httpd). I have no apaches running 
 on any vservers.

hmm, but they are there, right?

 There may be others that don't show up too, but that was the obvious one ;).

did you try with 1.9.3-rc2 yet? 
if not, please give it a spin!

 Thought I should pass that along ;),

if it remains, maybe we can investigate
this interactively on the irc channel?
#vserver @ irc.oftc.net

thanks,
Herbert

 Jim


 ___
 Vserver mailing list
 [EMAIL PROTECTED]
 http://list.linux-vserver.org/mailman/listinfo/vserver

___
Vserver mailing list
[EMAIL PROTECTED]
http://list.linux-vserver.org/mailman/listinfo/vserver

Re: [Vserver] netstat -nap --inet kills vserver kernel

2004-10-12 Thread James MacLean 
Herbert Poetzl wrote:
On Tue, Oct 12, 2004 at 09:27:09AM -0300, James MacLean wrote:
 

James MacLean wrote:
   

Herbert Poetzl wrote:
 

On Thu, Sep 16, 2004 at 11:49:51AM -0300, James MacLean wrote:
   

okay could you please go back in steps of 4 bytes as
described above .. c02ae988, c02ae984, c02ae980 ...
until you hit 'something' in net/ipv4/tcp_ipv4.c?
  
   

addr2line -e /usr/src/linux/vmlinux c02ae988
net/ipv4/tcp_ipv4.c:2185
 

okay, please try the latest patch and let me know
if this fixes your issues:
http://vserver.13thfloor.at/Experimental/patch-2.6.9-rc2-vs1.9.2.28.diff
TIA,
Herbert
   

#!/usr/bin/perl
$counter = 0;
while($counter  1000) {
print `netstat -nap --inet`;
$counter++;
}
Did not blow it up. So far much better :). I will be pushing it more 
over the next while so if it goes I'll be back :).

Now... just be be critical of this service. You must realize that if 
you are able to fix kernel crashes within 3 hours of having the 
correct information you are likely to set some very high expectations 
:). Snicker snicker :).

Thanks a bunch,
JES
 

Hi Folks (Herbert),
Noticed just recently that netstat on the main server or context 1 does 
not show up my Apache process/ports (httpd). I have no apaches running 
on any vservers.
   

hmm, but they are there, right?
 

Yes, httpd was running on the master server.
There may be others that don't show up too, but that was the obvious one ;).
   

did you try with 1.9.3-rc2 yet? 
if not, please give it a spin!

 

Tried it and netstat now shows httpd's... That was a quick fix :).
Thought I should pass that along ;),
   

if it remains, maybe we can investigate
this interactively on the irc channel?
#vserver @ irc.oftc.net
thanks,
Herbert
 

Thanks for the offer,
JES

begin:vcard
fn:James B MacLean
n:MacLean;James B
org:Education;ITS Technical Services
adr:;;;Halifax;NS;;Canada
email;internet:[EMAIL PROTECTED]
url:http://www.ednet.ns.ca/~macleajb
version:2.1
end:vcard

___
Vserver mailing list
[EMAIL PROTECTED]
http://list.linux-vserver.org/mailman/listinfo/vserver

Re: [Vserver] netstat -nap --inet kills vserver kernel

2004-09-16 Thread Herbert Poetzl 
On Thu, Sep 16, 2004 at 10:24:31AM -0300, James MacLean wrote:
 Herbert Poetzl wrote:
 
 On Wed, Sep 15, 2004 at 09:53:50PM -0300, James MacLean wrote:
 Herbert Poetzl wrote:
 On Wed, Sep 15, 2004 at 08:28:43PM -0300, James MacLean wrote:
 Herbert Poetzl wrote:
 Don't hold your breath :(. Can not find a null modem and cables :(. 
 Plus the test box is down and didn't auto reboot this time so I'm 
 sending someone in to revive it :).
 I can wait ;) ...
 
 best,
 Herbert
 JES
 Hi James!
 
 okay, could you put the EIP address through addr2line
 with 'addr2line -e vmlinux c02ae990' and see if
 this points to the folowing line of code:
 Just gives ??:00
 
 probably CONFIG_DEBUG_KERNEL=y isn't set in the 
 kernel config. make a copy of the kernel source
 tree, change just that single option, recompile
 the kernel and use the vmlinux for the addr2line
 
 Needed CONFIG_DEBUG_INFO=y.
 
 net/ipv4/tcp_ipv4.c ~2189
 
vxdprintk(VXD_CBIT(net, 6),
sk,req: %p [#%d] (from %d),
req-sk, req-sk-sk_xid, current-xid);
 here -if (!vx_check(req-sk-sk_xid, VX_IDENT|VX_WATCH))
continue;
if (req-class-family == st-family) {
cur = req;
goto out;
}
 
 if the addr2line doesn't produce a line in
 net/ipv4/tcp_ipv4.c, please decrement the addr
 in steps of 4 bytes, like this: 
 c02ae98c, c02ae988, c02ae984, c02ae980 ...
 and see where in net/ipv4/tcp_ipv4.c it ends up
 
 Sorry, but all I get it is ??:00 and after reading the man on addr2line, 
 I also compiled hello.c and got the same result. So obviously I'm not up 
 to snuff with what I should be doing/seeing to get addr2line to 
 function.  Or is there a way to get what you want with gdb vmlinux ?
 
 (for hello.c you'll need the -g option for gcc)
 
 Okay. This time the EIP resulted in :
 addr2line -e /usr/src/linux/vmlinux c02ae98c
 include/linux/vs_base.h:23

okay could you please go back in steps of 4 bytes as
described above .. c02ae988, c02ae984, c02ae980 ...
until you hit 'something' in net/ipv4/tcp_ipv4.c?

 So I tried it for the c02aee1b tcp_seq_next+53/ad 
 line and got welcome: addr2line -e /usr/src/linux/vmlinux c02aee1b
 net/ipv4/tcp_ipv4.c:2395

yep, but that is one call higher in the call stack ;)

TIA,
Herbert

 And here is the latest dump :
 
 ksymoops 2.4.9 on i686 2.6.9-rc2-bk1-vs1.9.2.23b.  Options used
 -v /usr/src/linux/vmlinux (specified)
 -k /proc/ksyms (default)
 -l /proc/modules (default)
 -o /lib/modules/2.6.9-rc2-bk1-vs1.9.2.23b/ (default)
 -m /usr/src/linux/System.map (default)
 
 Error (regular_file): read_ksyms stat /proc/ksyms failed
 ksymoops: No such file or directory
 No modules in ksyms, skipping objects
 No ksyms, skipping lsmod
 Unable to handle kernel NULL pointer dereference at virtual address 001c
 c02ae98c
 *pde = 
 Oops:  [#1]
 CPU:0
 EIP:0060:[c02ae98c]Not tainted VLI
 Using defaults from ksymoops -t elf32-i386 -a i386
 EFLAGS: 00010286   (2.6.9-rc2-bk1-vs1.9.2.23b)
 eax:    ebx:    ecx: ebe22700   edx: f088
 esi: f7c0bac0   edi: f4fcc42c   ebp: ec956000   esp: ec957f20
 ds: 007b   es: 007b   ss: 0068
 Stack: f7c0bac0 f4fcc28c f7c0a780 f7c0bac0 ec957f60 01c2 c02aee1b 
 
   f7c0badc f7c0a780 f7c9c080 c01750ef f7c0a798 001a 03e6 
 b7de601a
   0009  0009  c03fc7c0 f1e63780 0400 
 ec957fac
 Call Trace:
 [c02aee1b] tcp_seq_next+0x53/0xad
 [c01750ef] seq_read+0x1d8/0x268
 [c0159252] vfs_read+0xc6/0x111
 [c01594cb] sys_read+0x47/0x76
 [c0105e1d] sysenter_past_esp+0x52/0x71
 Code: 00 8b 4c 24 04 8b 34 24 8b 91 a4 01 00 00 8b 46 10 8b 4c 82 14 85 
 c9 74 20 b8 00 e0 ff ff 21 e0 8b 00 8b 98 10 05 00 00 8b 41 28 39 58 
 1c 74 4e 83 fb 01 74 49 85 c9 75 ef 8b 14 24 8b 42 10 83
 
 
 EIP; c02ae98c listening_get_next+120/1b7   =
 
 ecx; ebe22700 pg0+2ba1f700/3fbfb400
 edx; f088 pg0+3047d000/3fbfb400
 esi; f7c0bac0 pg0+37808ac0/3fbfb400
 edi; f4fcc42c pg0+34bc942c/3fbfb400
 ebp; ec956000 pg0+2c553000/3fbfb400
 esp; ec957f20 pg0+2c554f20/3fbfb400
 
 Trace; c02aee1b tcp_seq_next+53/ad
 Trace; c01750ef seq_read+1d8/268
 Trace; c0159252 vfs_read+c6/111
 Trace; c01594cb sys_read+47/76
 Trace; c0105e1d sysenter_past_esp+52/71
 
 This architecture has variable length instructions, decoding before eip
 is unreliable, take these instructions with a pinch of salt.
 
 Code;  c02ae961 listening_get_next+f5/1b7
  _EIP:
 Code;  c02ae961 listening_get_next+f5/1b7
   0:   00 8b 4c 24 04 8b add%cl,0x8b04244c(%ebx)
 Code;  c02ae967 listening_get_next+fb/1b7
   6:   34 24 xor$0x24,%al
 Code;  c02ae969 listening_get_next+fd/1b7
   8:   8b 91 a4 01 00 00 mov0x1a4(%ecx),%edx
 Code;  c02ae96f listening_get_next+103/1b7
   e:   8b 46 10  mov0x10(%esi),%eax
 Code;  c02ae972 listening_get_next+106/1b7
  11:   8b 4c 82 14

Re: [Vserver] netstat -nap --inet kills vserver kernel

2004-09-16 Thread James MacLean 
Herbert Poetzl wrote:
On Thu, Sep 16, 2004 at 10:24:31AM -0300, James MacLean wrote:
 

Herbert Poetzl wrote:
   

On Wed, Sep 15, 2004 at 09:53:50PM -0300, James MacLean wrote:
 

Herbert Poetzl wrote:
   

On Wed, Sep 15, 2004 at 08:28:43PM -0300, James MacLean wrote:
 

Herbert Poetzl wrote:
   

Don't hold your breath :(. Can not find a null modem and cables :(. 
Plus the test box is down and didn't auto reboot this time so I'm 
sending someone in to revive it :).
   

I can wait ;) ...
best,
Herbert
 

JES
   

Hi James!
okay, could you put the EIP address through addr2line
with 'addr2line -e vmlinux c02ae990' and see if
this points to the folowing line of code:
 

Just gives ??:00
   

probably CONFIG_DEBUG_KERNEL=y isn't set in the 
kernel config. make a copy of the kernel source
tree, change just that single option, recompile
the kernel and use the vmlinux for the addr2line

 

Needed CONFIG_DEBUG_INFO=y.
   

net/ipv4/tcp_ipv4.c ~2189
vxdprintk(VXD_CBIT(net, 6),
sk,req: %p [#%d] (from %d),
req-sk, req-sk-sk_xid, current-xid);
here -  if (!vx_check(req-sk-sk_xid, VX_IDENT|VX_WATCH))
continue;
if (req-class-family == st-family) {
cur = req;
goto out;
}
if the addr2line doesn't produce a line in
net/ipv4/tcp_ipv4.c, please decrement the addr
in steps of 4 bytes, like this: 
c02ae98c, c02ae988, c02ae984, c02ae980 ...
and see where in net/ipv4/tcp_ipv4.c it ends up

 

Sorry, but all I get it is ??:00 and after reading the man on addr2line, 
I also compiled hello.c and got the same result. So obviously I'm not up 
to snuff with what I should be doing/seeing to get addr2line to 
function.  Or is there a way to get what you want with gdb vmlinux ?
   

(for hello.c you'll need the -g option for gcc)
 

Okay. This time the EIP resulted in :
addr2line -e /usr/src/linux/vmlinux c02ae98c
include/linux/vs_base.h:23
   

okay could you please go back in steps of 4 bytes as
described above .. c02ae988, c02ae984, c02ae980 ...
until you hit 'something' in net/ipv4/tcp_ipv4.c?
 

addr2line -e /usr/src/linux/vmlinux c02ae988
net/ipv4/tcp_ipv4.c:2185
JES
___
Vserver mailing list
[EMAIL PROTECTED]
http://list.linux-vserver.org/mailman/listinfo/vserver

Re: [Vserver] netstat -nap --inet kills vserver kernel

2004-09-16 Thread Herbert Poetzl 
On Thu, Sep 16, 2004 at 11:49:51AM -0300, James MacLean wrote:
 okay could you please go back in steps of 4 bytes as
 described above .. c02ae988, c02ae984, c02ae980 ...
 until you hit 'something' in net/ipv4/tcp_ipv4.c?
  
 addr2line -e /usr/src/linux/vmlinux c02ae988
 net/ipv4/tcp_ipv4.c:2185

okay, please try the latest patch and let me know
if this fixes your issues:

http://vserver.13thfloor.at/Experimental/patch-2.6.9-rc2-vs1.9.2.28.diff

TIA,
Herbert

 JES


___
Vserver mailing list
[EMAIL PROTECTED]
http://list.linux-vserver.org/mailman/listinfo/vserver

Re: [Vserver] netstat -nap --inet kills vserver kernel

2004-09-16 Thread James MacLean 
Herbert Poetzl wrote:
On Thu, Sep 16, 2004 at 11:49:51AM -0300, James MacLean wrote:
 

okay could you please go back in steps of 4 bytes as
described above .. c02ae988, c02ae984, c02ae980 ...
until you hit 'something' in net/ipv4/tcp_ipv4.c?
 

addr2line -e /usr/src/linux/vmlinux c02ae988
net/ipv4/tcp_ipv4.c:2185
   

okay, please try the latest patch and let me know
if this fixes your issues:
http://vserver.13thfloor.at/Experimental/patch-2.6.9-rc2-vs1.9.2.28.diff
TIA,
Herbert
 

#!/usr/bin/perl
$counter = 0;
while($counter  1000) {
 print `netstat -nap --inet`;
 $counter++;
}
Did not blow it up. So far much better :). I will be pushing it more 
over the next while so if it goes I'll be back :).

Now... just be be critical of this service. You must realize that if you 
are able to fix kernel crashes within 3 hours of having the correct 
information you are likely to set some very high expectations :). 
Snicker snicker :).

Thanks a bunch,
JES
___
Vserver mailing list
[EMAIL PROTECTED]
http://list.linux-vserver.org/mailman/listinfo/vserver

Re: [Vserver] netstat -nap --inet kills vserver kernel

2004-09-16 Thread Tor Rune Skoglund 
 On Thu, 16 Sep 2004 13:48:19 -0300, James MacLean [EMAIL PROTECTED]
 if this fixes your issues:
 http://vserver.13thfloor.at/Experimental/patch-2.6.9-rc2-vs1.9.2.28.diff

 Did not blow it up. So far much better :). I will be pushing it more over
 the next while so if it goes I'll be back :).

 You guys rock.

 Bertl, that linux-vserver company must be paying you some serious
 money for you to be giving support like this.
 ;-)

 Just wanting you to know your work is appreciated.

Big applause from here up also. It has been interesting to follow the
thread and see how well such a matter was handled. :)

Btw, anyone know the status of this problem on 2.4.27-rc2 ?
which, btw2, is _still_ just rc2? :)

Keep up the good work,
Tor Rune Skoglund
-- 
DataKompaniet as
Teknobyen Innovasjonssenter, Abelsgt. 5 Tel: +47 73 51 51 51
N-7030 Trondheim, NorwayFax: +47 73 94 38 61
WWW:http://www.datakompaniet.no
E-mail: [EMAIL PROTECTED]

Ved svar på email, fjern all overflødig tekst, men inkluder alltid
nok av gammel email slik at det går klart frem hva saken gjelder.



___
Vserver mailing list
[EMAIL PROTECTED]
http://list.linux-vserver.org/mailman/listinfo/vserver

Re: [Vserver] netstat -nap --inet kills vserver kernel

2004-09-16 Thread Herbert Poetzl 
On Thu, Sep 16, 2004 at 07:44:37PM +0200, Tor Rune Skoglund wrote:
  On Thu, 16 Sep 2004 13:48:19 -0300, James MacLean [EMAIL PROTECTED]
  if this fixes your issues:
  http://vserver.13thfloor.at/Experimental/patch-2.6.9-rc2-vs1.9.2.28.diff
 
  Did not blow it up. So far much better :). I will be pushing it more over
  the next while so if it goes I'll be back :).
 
  You guys rock.

thanks!

  Bertl, that linux-vserver company must be paying you some serious
  money for you to be giving support like this.
  ;-)
 
  Just wanting you to know your work is appreciated.
 
 Big applause from here up also. It has been interesting to follow the
 thread and see how well such a matter was handled. :)

thanks again!

 Btw, anyone know the status of this problem on 2.4.27-rc2 ?
 which, btw2, is _still_ just rc2? :)

not an issue for vs1.2x, and it will probably become
a final release this week ...

 Keep up the good work,
 Tor Rune Skoglund

we'll try!

best,
Herbert

 -- 
 DataKompaniet as
 Teknobyen Innovasjonssenter, Abelsgt. 5 Tel: +47 73 51 51 51
 N-7030 Trondheim, NorwayFax: +47 73 94 38 61
 WWW:http://www.datakompaniet.no
 E-mail: [EMAIL PROTECTED]
 
 Ved svar på email, fjern all overflødig tekst, men inkluder alltid
 nok av gammel email slik at det går klart frem hva saken gjelder.
 
 
 
 ___
 Vserver mailing list
 [EMAIL PROTECTED]
 http://list.linux-vserver.org/mailman/listinfo/vserver
___
Vserver mailing list
[EMAIL PROTECTED]
http://list.linux-vserver.org/mailman/listinfo/vserver

Re: [Vserver] netstat -nap --inet kills vserver kernel

2004-09-15 Thread James MacLean 
Herbert Poetzl wrote:
On Tue, Sep 14, 2004 at 07:18:14PM -0300, James MacLean wrote:
 

Herbert Poetzl wrote:
I can wait ;) ...
best,
Herbert
   

JES
   

Attached,
JES
___
Vserver mailing list
[EMAIL PROTECTED]
http://list.linux-vserver.org/mailman/listinfo/vserver

Re: [Vserver] netstat -nap --inet kills vserver kernel

2004-09-15 Thread Herbert Poetzl 
On Wed, Sep 15, 2004 at 08:28:43PM -0300, James MacLean wrote:
 Herbert Poetzl wrote:
 
 Don't hold your breath :(. Can not find a null modem and cables :(. 
 Plus the test box is down and didn't auto reboot this time so I'm 
 sending someone in to revive it :).

 
 
 I can wait ;) ...
 
 best,
 Herbert
 
 Try again,
 JES

Hi James!

okay, could you put the EIP address through addr2line
with 'addr2line -e vmlinux c02ae990' and see if
this points to the folowing line of code:

net/ipv4/tcp_ipv4.c ~2189

vxdprintk(VXD_CBIT(net, 6),
sk,req: %p [#%d] (from %d),
req-sk, req-sk-sk_xid, current-xid);
 here -if (!vx_check(req-sk-sk_xid, VX_IDENT|VX_WATCH))
continue;
if (req-class-family == st-family) {
cur = req;
goto out;
}

if the addr2line doesn't produce a line in
net/ipv4/tcp_ipv4.c, please decrement the addr
in steps of 4 bytes, like this: 
c02ae98c, c02ae988, c02ae984, c02ae980 ...
and see where in net/ipv4/tcp_ipv4.c it ends up

TIA,
Herbert

 ksymoops 2.4.9 on i686 2.6.9-rc2-bk1-vs1.9.2.23b.  Options used
 -V (default)
 -k /proc/ksyms (default)
 -l /proc/modules (default)
 -o /lib/modules/2.6.9-rc2-bk1-vs1.9.2.23b/ (default)
 -m /usr/src/linux/System.map (default)
 
 Warning: You did not tell me where to find symbol information.  I will
 assume that the log matches the kernel and modules that are running
 right now and I'll use the default options above for symbol resolution.
 If the current kernel and/or modules do not match the log, you can get
 more accurate output by telling me the kernel version and where to find
 map, modules, ksyms etc.  ksymoops -h explains the options.
 
 Error (regular_file): read_ksyms stat /proc/ksyms failed
 No modules in ksyms, skipping objects
 No ksyms, skipping lsmod
 Unable to handle kernel NULL pointer dereference at virtual address 001c
 c02ae990
 *pde = 
 Oops:  [#1]
 CPU:0
 EIP:0060:[c02ae990]Not tainted VLI
 Using defaults from ksymoops -t elf32-i386 -a i386
 EFLAGS: 00010286   (2.6.9-rc2-bk1-vs1.9.2.23b)
 eax:    ebx:    ecx: f30d5800   edx: f3981000
 esi: f7c0ccc0   edi: f4cba42c   ebp: e47a   esp: e47a1f20
 ds: 007b   es: 007b   ss: 0068
 Stack: f7c0ccc0 f4cba28c f7c0cec0 f7c0ccc0 e47a1f60 01c2 c02aee1f 
 
   f7c0ccdc f7c0cec0 f7c86080 c01750eb f7c0ced8 001a 03e6 
 b7de601a
   0009  0009  c03fc7c0 ddc23280 0400 
 e47a1fac
 Call Trace:
 [c02aee1f] tcp_seq_next+0x53/0xad
 [c01750eb] seq_read+0x1d8/0x268
 [c0159252] vfs_read+0xc6/0x111
 [c01594cb] sys_read+0x47/0x76
 [c0105e1d] sysenter_past_esp+0x52/0x71
 Code: 00 8b 4c 24 04 8b 34 24 8b 91 a4 01 00 00 8b 46 10 8b 4c 82 14 85 
 c9 74 20 b8 00 e0 ff ff 21 e0 8b 00 8b 98 10 05 00 00 8b 41 28 39 58 
 1c 74 4e 83 fb 01 74 49 85 c9 75 ef 8b 14 24 8b 42 10 83
 
 
 EIP; c02ae990 listening_get_next+17c/1b7   =
 
 ecx; f30d5800 pg0+32cd2800/3fbfb400
 edx; f3981000 pg0+3357e000/3fbfb400
 esi; f7c0ccc0 pg0+37809cc0/3fbfb400
 edi; f4cba42c pg0+348b742c/3fbfb400
 ebp; e47a pg0+2439d000/3fbfb400
 esp; e47a1f20 pg0+2439ef20/3fbfb400
 
 Trace; c02aee1f tcp_seq_next+ab/ad
 Trace; c01750eb traverse+18/176
 Trace; c0159252 do_sync_write+a5/d6
 Trace; c01594cb sys_pread64+4b/82
 Trace; c0105e1d sysenter_past_esp+52/71
 
 This architecture has variable length instructions, decoding before eip
 is unreliable, take these instructions with a pinch of salt.
 
 Code;  c02ae965 listening_get_next+151/1b7
  _EIP:
 Code;  c02ae965 listening_get_next+151/1b7
   0:   00 8b 4c 24 04 8b add%cl,0x8b04244c(%ebx)
 Code;  c02ae96b listening_get_next+157/1b7
   6:   34 24 xor$0x24,%al
 Code;  c02ae96d listening_get_next+159/1b7
   8:   8b 91 a4 01 00 00 mov0x1a4(%ecx),%edx
 Code;  c02ae973 listening_get_next+15f/1b7
   e:   8b 46 10  mov0x10(%esi),%eax
 Code;  c02ae976 listening_get_next+162/1b7
  11:   8b 4c 82 14   mov0x14(%edx,%eax,4),%ecx
 Code;  c02ae97a listening_get_next+166/1b7
  15:   85 c9 test   %ecx,%ecx
 Code;  c02ae97c listening_get_next+168/1b7
  17:   74 20 je 39 _EIP+0x39
 Code;  c02ae97e listening_get_next+16a/1b7
  19:   b8 00 e0 ff ffmov$0xe000,%eax
 Code;  c02ae983 listening_get_next+16f/1b7
  1e:   21 e0 and%esp,%eax
 Code;  c02ae985 listening_get_next+171/1b7
  20:   8b 00 mov(%eax),%eax
 Code;  c02ae987 listening_get_next+173/1b7
  22:   8b 98 10 05 00 00 mov0x510(%eax),%ebx
 Code;  c02ae98d listening_get_next+179/1b7
  28:   8b 41 28  mov0x28(%ecx),%eax
 
 This decode from eip onwards should be reliable
 
 Code;  c02ae990 listening_get_next+17c/1b7
  _EIP:
 Code;  c02ae990

Re: [Vserver] netstat -nap --inet kills vserver kernel

2004-09-15 Thread Herbert Poetzl 
On Wed, Sep 15, 2004 at 09:53:50PM -0300, James MacLean wrote:
 Herbert Poetzl wrote:
 
 On Wed, Sep 15, 2004 at 08:28:43PM -0300, James MacLean wrote:
 
 Herbert Poetzl wrote:
 
 Don't hold your breath :(. Can not find a null modem and cables :(. 
 Plus the test box is down and didn't auto reboot this time so I'm 
 sending someone in to revive it :).
 
 I can wait ;) ...
 
 best,
 Herbert
 
 Try again,
 JES
 
 Hi James!
 
 okay, could you put the EIP address through addr2line
 with 'addr2line -e vmlinux c02ae990' and see if
 this points to the folowing line of code:
 
 Just gives ??:00

probably CONFIG_DEBUG_KERNEL=y isn't set in the 
kernel config. make a copy of the kernel source
tree, change just that single option, recompile
the kernel and use the vmlinux for the addr2line

 net/ipv4/tcp_ipv4.c ~2189
 
  vxdprintk(VXD_CBIT(net, 6),
  sk,req: %p [#%d] (from %d),
  req-sk, req-sk-sk_xid, current-xid);
 here -  if (!vx_check(req-sk-sk_xid, VX_IDENT|VX_WATCH))
  continue;
  if (req-class-family == st-family) {
  cur = req;
  goto out;
  }
 
 if the addr2line doesn't produce a line in
 net/ipv4/tcp_ipv4.c, please decrement the addr
 in steps of 4 bytes, like this: 
 c02ae98c, c02ae988, c02ae984, c02ae980 ...
 and see where in net/ipv4/tcp_ipv4.c it ends up
 
 Sorry, but all I get it is ??:00 and after reading the man on addr2line, 
 I also compiled hello.c and got the same result. So obviously I'm not up 
 to snuff with what I should be doing/seeing to get addr2line to 
 function.  Or is there a way to get what you want with gdb vmlinux ?

(for hello.c you'll need the -g option for gcc)

 Also I am resending the ksymoops as it might have been off due to the 
 System.map it was pointing to.

TIA,
Herbert

 ksymoops 2.4.9 on i686 2.6.9-rc2-bk1-vs1.9.2.23b.  Options used
 -v vmlinux (specified)
 -k /proc/ksyms (default)
 -l /proc/modules (default)
 -o /lib/modules/2.6.9-rc2-bk1-vs1.9.2.23b/ (default)
 -m /usr/src/linux/System.map (default)
 
 Error (regular_file): read_ksyms stat /proc/ksyms failed
 No modules in ksyms, skipping objects
 No ksyms, skipping lsmod
 Unable to handle kernel NULL pointer dereference at virtual address 001c
 c02ae990
 *pde = 
 Oops:  [#1]
 CPU:0
 EIP:0060:[c02ae990]Not tainted VLI
 Using defaults from ksymoops -t elf32-i386 -a i386
 EFLAGS: 00010286   (2.6.9-rc2-bk1-vs1.9.2.23b)
 eax:    ebx:    ecx: f30d5800   edx: f3981000
 esi: f7c0ccc0   edi: f4cba42c   ebp: e47a   esp: e47a1f20
 ds: 007b   es: 007b   ss: 0068
 Stack: f7c0ccc0 f4cba28c f7c0cec0 f7c0ccc0 e47a1f60 01c2 c02aee1f 
 
   f7c0ccdc f7c0cec0 f7c86080 c01750eb f7c0ced8 001a 03e6 
 b7de601a
   0009  0009  c03fc7c0 ddc23280 0400 
 e47a1fac
 Call Trace:
 [c02aee1f] tcp_seq_next+0x53/0xad
 [c01750eb] seq_read+0x1d8/0x268
 [c0159252] vfs_read+0xc6/0x111
 [c01594cb] sys_read+0x47/0x76
 [c0105e1d] sysenter_past_esp+0x52/0x71
 Code: 00 8b 4c 24 04 8b 34 24 8b 91 a4 01 00 00 8b 46 10 8b 4c 82 14 85 
 c9 74 20 b8 00 e0 ff ff 21 e0 8b 00 8b 98 10 05 00 00 8b 41 28 39 58 
 1c 74 4e 83 fb 01 74 49 85 c9 75 ef 8b 14 24 8b 42 10 83
 
 
 EIP; c02ae990 listening_get_next+120/1b7   =
 
 ecx; f30d5800 pg0+32cd2800/3fbfb400
 edx; f3981000 pg0+3357e000/3fbfb400
 esi; f7c0ccc0 pg0+37809cc0/3fbfb400
 edi; f4cba42c pg0+348b742c/3fbfb400
 ebp; e47a pg0+2439d000/3fbfb400
 esp; e47a1f20 pg0+2439ef20/3fbfb400
 
 Trace; c02aee1f tcp_seq_next+53/ad
 Trace; c01750eb seq_read+1d8/268
 Trace; c0159252 vfs_read+c6/111
 Trace; c01594cb sys_read+47/76
 Trace; c0105e1d sysenter_past_esp+52/71
 
 This architecture has variable length instructions, decoding before eip
 is unreliable, take these instructions with a pinch of salt.
 
 Code;  c02ae965 listening_get_next+f5/1b7
  _EIP:
 Code;  c02ae965 listening_get_next+f5/1b7
   0:   00 8b 4c 24 04 8b add%cl,0x8b04244c(%ebx)
 Code;  c02ae96b listening_get_next+fb/1b7
   6:   34 24 xor$0x24,%al
 Code;  c02ae96d listening_get_next+fd/1b7
   8:   8b 91 a4 01 00 00 mov0x1a4(%ecx),%edx
 Code;  c02ae973 listening_get_next+103/1b7
   e:   8b 46 10  mov0x10(%esi),%eax
 Code;  c02ae976 listening_get_next+106/1b7
  11:   8b 4c 82 14   mov0x14(%edx,%eax,4),%ecx
 Code;  c02ae97a listening_get_next+10a/1b7
  15:   85 c9 test   %ecx,%ecx
 Code;  c02ae97c listening_get_next+10c/1b7
  17:   74 20 je 39 _EIP+0x39
 Code;  c02ae97e listening_get_next+10e/1b7
  19:   b8 00 e0 ff ffmov$0xe000,%eax
 Code;  c02ae983 listening_get_next+113/1b7
  1e:   21 e0 and%esp,%eax
 Code;  c02ae985 listening_get_next+115/1b7
  20:   8b 00 mov(%eax),%eax
 Code;  c02ae987

[Vserver] netstat -nap --inet kills vserver kernel

2004-09-14 Thread James MacLean 
Hi Folks,
Maybe some remember us posting this with the 2.4.23-rc1 kernel? 
Anyway... It's back... At least for me.

. patch-2.6.9-rc1-bk13-vs1.9.2.24.diff
. linux-2.6.9-rc2-bk1
Happened in an rc1 kernel too ;). Hand patched in some of the diff.
After doing an netstat -nap --inet for a while, the whole kernel 
drops. As netconsole is not working for me I am off to find a serial 
cross-over.

Copied from the screen some stuff from the crash:
c02ae9f9 listening_get_idx
c02aecd7 tcp_get_idx
c0174f18 seq_read
c01fabd3 write_chan
c0159162 vfs_read
c01593db sys_read
instructions 00 8b 4c 24 04
Last time Herbert fixed it with :
--- linux-2.4.23-rc1-vs1.1.4/net/ipv4/tcp_ipv4.cTue Nov 18 23:32:58 2003
+++ linux-2.4.23-rc1-vs1.1.4-fix/net/ipv4/tcp_ipv4.cWed Nov 19 09:47:36 2003
@@ -2334,7 +2334,7 @@ skip_listen:
 tw != NULL;
 tw = (struct tcp_tw_bucket *)tw-next, num++) {
if (!TCP_INET_FAMILY(tw-family) ||
-   !vx_check(sk-vx_id, VX_WATCH|VX_IDENT))
+   !vx_check(tw-vx_id, VX_WATCH|VX_IDENT))
continue;
pos += TMPSZ;
Who has the magic this time :)?
JES
if (pos = offset)


___
Vserver mailing list
[EMAIL PROTECTED]
http://list.linux-vserver.org/mailman/listinfo/vserver

Re: [Vserver] netstat -nap --inet kills vserver kernel

2004-09-14 Thread Herbert Poetzl 
On Tue, Sep 14, 2004 at 12:51:09PM -0300, James MacLean wrote:
 Hi Folks,

Hi James!

 Maybe some remember us posting this with the 2.4.23-rc1 kernel? 
 Anyway... It's back... At least for me.
 
 . patch-2.6.9-rc1-bk13-vs1.9.2.24.diff
 . linux-2.6.9-rc2-bk1
 
 Happened in an rc1 kernel too ;). Hand patched in some of the diff.

could you test if patch-2.6.9-rc1-bk12-vs1.9.2.23.diff
(on the appropriate or similar kernel) showes this
behaviour or not?

 After doing an netstat -nap --inet for a while, the whole kernel 
 drops. As netconsole is not working for me I am off to find a serial 
 cross-over.

serial kernel oops dump would be appreciated ...

thanks,
Herbert

 Copied from the screen some stuff from the crash:
 
 c02ae9f9 listening_get_idx
 c02aecd7 tcp_get_idx
 c0174f18 seq_read
 c01fabd3 write_chan
 c0159162 vfs_read
 c01593db sys_read
 
 instructions 00 8b 4c 24 04
 
 Last time Herbert fixed it with :
 
 --- linux-2.4.23-rc1-vs1.1.4/net/ipv4/tcp_ipv4.c  Tue Nov 18 23:32:58 
 2003
 +++ linux-2.4.23-rc1-vs1.1.4-fix/net/ipv4/tcp_ipv4.c  Wed Nov 19 09:47:36 
 2003
 @@ -2334,7 +2334,7 @@ skip_listen:
tw != NULL;
tw = (struct tcp_tw_bucket *)tw-next, num++) {
   if (!TCP_INET_FAMILY(tw-family) ||
 - !vx_check(sk-vx_id, VX_WATCH|VX_IDENT))
 + !vx_check(tw-vx_id, VX_WATCH|VX_IDENT))
   continue;
   pos += TMPSZ;
 
 Who has the magic this time :)?
 
 JES
   if (pos = offset)
 
 
 
 

 ___
 Vserver mailing list
 [EMAIL PROTECTED]
 http://list.linux-vserver.org/mailman/listinfo/vserver

___
Vserver mailing list
[EMAIL PROTECTED]
http://list.linux-vserver.org/mailman/listinfo/vserver

Re: [Vserver] netstat -nap --inet kills vserver kernel

2004-09-14 Thread Herbert Poetzl 
On Tue, Sep 14, 2004 at 07:18:14PM -0300, James MacLean wrote:
 Herbert Poetzl wrote:
 
 On Tue, Sep 14, 2004 at 12:51:09PM -0300, James MacLean wrote:
  
 
 Hi Folks,

 
 
 Hi James!
 
  
 
 Maybe some remember us posting this with the 2.4.23-rc1 kernel?
 Anyway... It's back... At least for me.
 
 . patch-2.6.9-rc1-bk13-vs1.9.2.24.diff
 . linux-2.6.9-rc2-bk1
 
 Happened in an rc1 kernel too ;). Hand patched in some of the diff.
 could you test if patch-2.6.9-rc1-bk12-vs1.9.2.23.diff
 (on the appropriate or similar kernel) showes this
 behaviour or not?
 
 By appropriate, would the patch-2.6.9-rc2-vs1.9.2.27.diff against 
 linux-2.6.9-rc2 suffice? Or is it the particular patch 
 patch-2.6.9-rc1-bk12-vs1.9.2.23.diff you want me to make a kernel with?

yep, the patch right after that one, the .24 you obviously
tested (read your lines above) changed some things regarding
rcu, and I'd like to know if that is the cause ...

 After doing an netstat -nap --inet for a while, the whole kernel
 drops. As netconsole is not working for me I am off to find a serial
 cross-over.
 serial kernel oops dump would be appreciated ...
 
 Don't hold your breath :(. Can not find a null modem and cables :(. Plus 
 the test box is down and didn't auto reboot this time so I'm sending 
 someone in to revive it :).

I can wait ;) ...

best,
Herbert

 JES


___
Vserver mailing list
[EMAIL PROTECTED]
http://list.linux-vserver.org/mailman/listinfo/vserver