Bug#678519: general: after about 1 month of uptime, routing of IPv6 packets is no longer possible, and IPv4 routing becomes slow and unpredictable. Rebooting brings all functionality back, and back to

2012-06-24 Thread Jonathan Nieder
retitle 678519 after about a month, routing gets wedged
# not a general problem affecting a large portion of the archive
reassign 678519 base
tags 678519 + squeeze
quit

Hi Henrique,

Henrique de Moraes Holschuh wrote:
 On Fri, 22 Jun 2012, Rudy Zijlstra wrote:

 let system run with IPv4  IPv6 routing for about 1 month
 IPv6 routing will start to fail
 IPv4 routing becomes slow and unpredictable

 no obvious causes visible in the system. top and friends do not show a cpu 
 hog

 a reboot will bring the system back to normal behaviour. 

 Please use (as root) ip neigh show, and ip route list cache to try to
 track down any weird differences between the box when it is behaving
 normally, and the box when wedged.  You may want to compare it to a healthy
 box on the same network segment.

Thanks for starting to track this down.  Any idea which package might
be responsible?

Rudy, is this a regression, or has this system always behaved this
way?  How many times has it happened?  How reliable is the 1 month
gestation time?  When did it start?

Thanks,
Jonathan



-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20120625022342.GA3712@burratino



Bug#678519: general: after about 1 month of uptime, routing of IPv6 packets is no longer possible, and IPv4 routing becomes slow and unpredictable. Rebooting brings all functionality back, and back to

2012-06-23 Thread Rudy Zijlstra

On 22-06-12 21:38, Henrique de Moraes Holschuh wrote:

On Fri, 22 Jun 2012, Rudy Zijlstra wrote:

let system run with IPv4  IPv6 routing for about 1 month

IPv6 routing will start to fail
IPv4 routing becomes slow and unpredictable

no obvious causes visible in the system. top and friends do not show a cpu hog

a reboot will bring the system back to normal behaviour.

Please use (as root) ip neigh show, and ip route list cache to try to
track down any weird differences between the box when it is behaving
normally, and the box when wedged.  You may want to compare it to a healthy
box on the same network segment.

You can also try to see if ip route flush cache and ip neigh flush can
unwedge the system.  After a flush, ip neigh show and ip route list
cache should return very few, if any, entries.

Thanks, i've stored the current output of these commands, including the 
IPv6 version, so i can compare when trouble hits again in some weeks.




--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4fe58ba3.2010...@grumpydevil.homelinux.org



Bug#678519: general: after about 1 month of uptime, routing of IPv6 packets is no longer possible, and IPv4 routing becomes slow and unpredictable. Rebooting brings all functionality back, and back to

2012-06-23 Thread Henrique de Moraes Holschuh
On Sat, 23 Jun 2012, Rudy Zijlstra wrote:
 On 22-06-12 21:38, Henrique de Moraes Holschuh wrote:
 On Fri, 22 Jun 2012, Rudy Zijlstra wrote:
 let system run with IPv4  IPv6 routing for about 1 month
 IPv6 routing will start to fail
 IPv4 routing becomes slow and unpredictable
 no obvious causes visible in the system. top and friends do not show a cpu 
 hog
 
 a reboot will bring the system back to normal behaviour.
 Please use (as root) ip neigh show, and ip route list cache to try to
 track down any weird differences between the box when it is behaving
 normally, and the box when wedged.  You may want to compare it to a healthy
 box on the same network segment.
 
 You can also try to see if ip route flush cache and ip neigh flush can
 unwedge the system.  After a flush, ip neigh show and ip route list
 cache should return very few, if any, entries.
 
 Thanks, i've stored the current output of these commands, including
 the IPv6 version, so i can compare when trouble hits again in some
 weeks.

You probably want to store their output once a day.  If it is a
neighbour/route cache leak or malfunction of some sort (e.g. routes getting
stuck in the presence of ICMP redirects), you should be able to notice that
old crap is accumulating over time.

If possible, do the same in a box that does not show the same problem
(ideally in the same network segment), so that you have a baseline to
compare to.

Note that it could be something else entirely, don't rule out hardware
malfunction (sometimes cleared if you down the interfaces and then bring
them up again), or driver issues (sometimes cleared if you rmmod + modprobe
the buggy driver).  And make sure the box is running the latest firmware
(BIOS/UEFI, NIC firmware...).

-- 
  One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie. -- The Silicon Valley Tarot
  Henrique Holschuh



-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20120623125311.ga18...@khazad-dum.debian.net



Bug#678519: general: after about 1 month of uptime, routing of IPv6 packets is no longer possible, and IPv4 routing becomes slow and unpredictable. Rebooting brings all functionality back, and back to

2012-06-23 Thread Rudy Zijlstra

On 23-06-12 14:53, Henrique de Moraes Holschuh wrote:

On Sat, 23 Jun 2012, Rudy Zijlstra wrote:

On 22-06-12 21:38, Henrique de Moraes Holschuh wrote:

On Fri, 22 Jun 2012, Rudy Zijlstra wrote:

let system run with IPv4   IPv6 routing for about 1 month

IPv6 routing will start to fail
IPv4 routing becomes slow and unpredictable

no obvious causes visible in the system. top and friends do not show a cpu hog

a reboot will bring the system back to normal behaviour.

Please use (as root) ip neigh show, and ip route list cache to try to
track down any weird differences between the box when it is behaving
normally, and the box when wedged.  You may want to compare it to a healthy
box on the same network segment.

You can also try to see if ip route flush cache and ip neigh flush can
unwedge the system.  After a flush, ip neigh show and ip route list
cache should return very few, if any, entries.


Thanks, i've stored the current output of these commands, including
the IPv6 version, so i can compare when trouble hits again in some
weeks.

You probably want to store their output once a day.  If it is a
neighbour/route cache leak or malfunction of some sort (e.g. routes getting
stuck in the presence of ICMP redirects), you should be able to notice that
old crap is accumulating over time.

If possible, do the same in a box that does not show the same problem
(ideally in the same network segment), so that you have a baseline to
compare to.

Note that it could be something else entirely, don't rule out hardware
malfunction (sometimes cleared if you down the interfaces and then bring
them up again), or driver issues (sometimes cleared if you rmmod + modprobe
the buggy driver).  And make sure the box is running the latest firmware
(BIOS/UEFI, NIC firmware...).

i'll script the commands from cron.daily. To compare with similar box is 
kind of difficult. I run only a single firewall
And although i have several squeeze boxes active, this is the only one 
showing this problem


NIC firmware is on the latest on condition that Squeeze has the latest. 
I do expect that though, as is it pretty old HW. Fully capable of 
firewall though.




--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4fe5cf3d.1080...@grumpydevil.homelinux.org



Bug#678519: general: after about 1 month of uptime, routing of IPv6 packets is no longer possible, and IPv4 routing becomes slow and unpredictable. Rebooting brings all functionality back, and back to

2012-06-22 Thread Rudy Zijlstra
Package: general
Severity: important
Tags: ipv6

let system run with IPv4  IPv6 routing for about 1 month
 IPv6 routing will start to fail
 IPv4 routing becomes slow and unpredictable

no obvious causes visible in the system. top and friends do not show a cpu hog

a reboot will bring the system back to normal behaviour. 

-- System Information:
Debian Release: 6.0.5
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: i386 (i686)

Kernel: Linux 2.6.32-5-686 (SMP w/2 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash



-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/20120622115937.1905.22529.report...@janus.office.romunt.nl



Bug#678519: general: after about 1 month of uptime, routing of IPv6 packets is no longer possible, and IPv4 routing becomes slow and unpredictable. Rebooting brings all functionality back, and back to

2012-06-22 Thread Roberto C . Sánchez
On Fri, Jun 22, 2012 at 01:59:37PM +0200, Rudy Zijlstra wrote:
 Package: general
 Severity: important
 Tags: ipv6
 
 let system run with IPv4  IPv6 routing for about 1 month
  IPv6 routing will start to fail
  IPv4 routing becomes slow and unpredictable
 
 no obvious causes visible in the system. top and friends do not show a cpu hog
 
 a reboot will bring the system back to normal behaviour. 
 
Could this be something to do with connection tracking?

Regards,

-Roberto
-- 
Roberto C. Sánchez
http://people.connexer.com/~roberto
http://www.connexer.com


signature.asc
Description: Digital signature


Bug#678519: general: after about 1 month of uptime, routing of IPv6 packets is no longer possible, and IPv4 routing becomes slow and unpredictable. Rebooting brings all functionality back, and back to

2012-06-22 Thread Rudy Zijlstra

On 22-06-12 15:04, Roberto C. Sánchez wrote:

On Fri, Jun 22, 2012 at 01:59:37PM +0200, Rudy Zijlstra wrote:

Package: general
Severity: important
Tags: ipv6

let system run with IPv4  IPv6 routing for about 1 month

IPv6 routing will start to fail
IPv4 routing becomes slow and unpredictable

no obvious causes visible in the system. top and friends do not show a cpu hog

a reboot will bring the system back to normal behaviour.


Could this be something to do with connection tracking?

Regards,

-Roberto
Both IPv4 and IPv6 are impacted, which have separate iptables. IPv6 
routing gets fully blocked, IPv4 goes slow and unpredictable.


How could i check any relation to connection tracking?

cheers,


Rudy



--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4fe46f3c.9040...@grumpydevil.homelinux.org



Bug#678519: general: after about 1 month of uptime, routing of IPv6 packets is no longer possible, and IPv4 routing becomes slow and unpredictable. Rebooting brings all functionality back, and back to

2012-06-22 Thread Henrique de Moraes Holschuh
On Fri, 22 Jun 2012, Rudy Zijlstra wrote:
 let system run with IPv4  IPv6 routing for about 1 month
  IPv6 routing will start to fail
  IPv4 routing becomes slow and unpredictable
 
 no obvious causes visible in the system. top and friends do not show a cpu hog
 
 a reboot will bring the system back to normal behaviour. 

Please use (as root) ip neigh show, and ip route list cache to try to
track down any weird differences between the box when it is behaving
normally, and the box when wedged.  You may want to compare it to a healthy
box on the same network segment.

You can also try to see if ip route flush cache and ip neigh flush can
unwedge the system.  After a flush, ip neigh show and ip route list
cache should return very few, if any, entries.

-- 
  One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie. -- The Silicon Valley Tarot
  Henrique Holschuh



-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20120622193831.gd32...@khazad-dum.debian.net