[Kernel-packages] [Bug 1715279] Re: 2.6.32 netfliter . nf_conntrack . destroy_conntrack BUG
** Changed in: linux (Ubuntu) Status: Incomplete => Confirmed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1715279 Title: 2.6.32 netfliter . nf_conntrack . destroy_conntrack BUG Status in linux package in Ubuntu: Confirmed Bug description: In my server ,i got a BUG，and i can initially locate where this bug is : nf_conntrack_in-> resolve_normal_ct -> nf_conntrack_find_get -> unlikely(!nf_ct_tuple_equal(tuple, >tuple)) -> nf_ct_put -> destroy_conntrack -> nf_ct_is_confirmed -> BUG_ON but I don't know why this happens，please help me ,THANK YOU! 4>[ cut here ] 15381 <2>kernel BUG at net/netfilter/nf_conntrack_core.c:206! 15382 <4>invalid opcode: [#1] SMP 15383 <4>last sysfs file: /sys/devices/system/cpu/cpu47/topology/thread_siblings_list 15384 <4>CPU 0 15385 <4>Modules linked in: nbd(U) iptable_mangle iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 iptable_filter ip_tables ksl b_ctk(U) fuse tcp_diag inet_diag xfs exportfs kvm_intel kvm vhost_net macvtap macvlan cdc_ether usbnet mii tun openvswitch(U) libcrc32c b onding 8021q garp stp llc toa hookers ipv6 power_meter iTCO_wdt iTCO_vendor_support dcdbas ipmi_devintf lpc_ich mfd_core shpchp igb i2c_a lgo_bit i2c_core ixgbe dca ptp pps_core mdio sg ext4 jbd2 mbcache sd_mod crc_t10dif ahci megaraid_sas wmi dm_mirror dm_region_hash dm_log dm_mod [last unloaded: nbd] 15386 <4> 15387 <4>Pid: 25449, comm: mebs-nbd-core Tainted: G --- H 2.6.32-431.20.3.el6.mt20161028.x86_64 #1 Dell Inc. PowerEdge R 730/0WCJNT 15388 <4>RIP: 0010:[] [] destroy_conntrack+0x11c/0x150 [nf_conntrack] 15389 <4>RSP: 0018:881d10cd99f8 EFLAGS: 00010246 15390 troy_conntrack+0x11c/0x1RAX: RBX: 8813d96dbdc8 RCX: 0001 15391 <4>RDX: b4be RSI: 0006 RDI: 8813d96dbdc8 15392 <4>RBP: 881d10cd9a08 R08: 0002 R09: 00c0 15393 <4>R10: R11: R12: 81b18540 15394 <4>R13: 0003 R14: 0002 R15: 0002 15395 <4>FS: 7f45d2ba1720() GS:88011f60() knlGS: 15396 <4>CS: 0010 DS: ES: CR0: 8005003b 15397 <4>CR2: CR3: 001f0aa3f000 CR4: 001427e0 15398 <4>DR0: DR1: DR2: 15399 <4>DR3: DR6: fffe0ff0 DR7: 0400 15400 <4>Process mebs-nbd-core (pid: 25449, threadinfo 881d10cd8000, task 881f0a9b6aa0) 15401 <4>Stack: 15402 <4> 881d10cd9ab8 81b18540 881d10cd9a18 8148a4c7 15403 <4> 881d10cd9a38 a029bac5 881c47af1b68 81b18540 15404 <4> 881d10cd9b18 a029bdd0 a026d620 a02a7080 15405 <4>Call Trace: 15406 <4> [] nf_conntrack_destroy+0x17/0x30 15407 <4> [] nf_conntrack_find_get+0x85/0x120 [nf_conntrack] 15408 <4> [] nf_conntrack_in+0x270/0xa00 [nf_conntrack] 15409 <4> [] ? ovs_dp_process_received_packet+0x51/0x80 [openvswitch] 15410 <4> [] ipv4_conntrack_local+0x51/0x60 [nf_conntrack_ipv4] 15411 <4> [] nf_iterate+0x69/0xb0 15412 <4> [] ? dst_output+0x0/0x20 15413 <4> [] nf_hook_slow+0x76/0x120 15414 <4> [] ? dst_output+0x0/0x20 15415 <4> [] __ip_local_out+0x9f/0xb0 15416 <4> [] ip_local_out+0x16/0x30 15417 <4> [] ip_queue_xmit+0x190/0x420 15418 <4> [] ? ip_output+0xb8/0xc0 15419 <4> [] tcp_transmit_skb+0x40e/0x7b0 15420 <4> [] tcp_write_xmit+0x230/0xa90 15421 troy_conntrack+0x11c/0x150 [] __tcp_push_pending_frames+0x30/0xe0 15422 <4> [] tcp_send_fin+0x72/0x1d0 15423 <4> [] tcp_close+0x346/0x450 15424 <4> [] inet_release+0x67/0x90 15425 <4> [] sock_release+0x29/0x90 15426 <4> [] sock_close+0x17/0x30 15427 <4> [] __fput+0xf5/0x210 15428 <4> [] fput+0x25/0x30 15429 <4> [] filp_close+0x5d/0x90 15430 <4> [] sys_close+0xa5/0x100 15431 <4> [] system_call_fastpath+0x16/0x1b 15432 <4>Code: 1b ec 1e e1 eb ce f6 05 9c fa d3 e1 20 74 ce 80 3d a0 ad 00 00 00 74 c5 48 89 de 48 c7 c7 00 32 2a a0 31 c0 e8 ca cc 28 e1 eb b2 <0f> 0b eb fe f6 05 73 fa d3 e1 20 0f 84 f6 fe ff ff 80 3d 9b ad 15433 <1>RIP [] destroy_conntrack+0x11c/0x150 [nf_conntrack] 15434 <4> RSP To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1715279/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1715279] Re: 2.6.32 netfliter . nf_conntrack . destroy_conntrack BUG
Thanks for your reply! Joseph. I have fixed this bug in 2.6.32, with the help of this git commit (e53376be) of the kernel. So in the 4.13, it is not a problem. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1715279 Title: 2.6.32 netfliter . nf_conntrack . destroy_conntrack BUG Status in linux package in Ubuntu: Incomplete Bug description: In my server ,i got a BUG，and i can initially locate where this bug is : nf_conntrack_in-> resolve_normal_ct -> nf_conntrack_find_get -> unlikely(!nf_ct_tuple_equal(tuple, >tuple)) -> nf_ct_put -> destroy_conntrack -> nf_ct_is_confirmed -> BUG_ON but I don't know why this happens，please help me ,THANK YOU! 4>[ cut here ] 15381 <2>kernel BUG at net/netfilter/nf_conntrack_core.c:206! 15382 <4>invalid opcode: [#1] SMP 15383 <4>last sysfs file: /sys/devices/system/cpu/cpu47/topology/thread_siblings_list 15384 <4>CPU 0 15385 <4>Modules linked in: nbd(U) iptable_mangle iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 iptable_filter ip_tables ksl b_ctk(U) fuse tcp_diag inet_diag xfs exportfs kvm_intel kvm vhost_net macvtap macvlan cdc_ether usbnet mii tun openvswitch(U) libcrc32c b onding 8021q garp stp llc toa hookers ipv6 power_meter iTCO_wdt iTCO_vendor_support dcdbas ipmi_devintf lpc_ich mfd_core shpchp igb i2c_a lgo_bit i2c_core ixgbe dca ptp pps_core mdio sg ext4 jbd2 mbcache sd_mod crc_t10dif ahci megaraid_sas wmi dm_mirror dm_region_hash dm_log dm_mod [last unloaded: nbd] 15386 <4> 15387 <4>Pid: 25449, comm: mebs-nbd-core Tainted: G --- H 2.6.32-431.20.3.el6.mt20161028.x86_64 #1 Dell Inc. PowerEdge R 730/0WCJNT 15388 <4>RIP: 0010:[] [] destroy_conntrack+0x11c/0x150 [nf_conntrack] 15389 <4>RSP: 0018:881d10cd99f8 EFLAGS: 00010246 15390 troy_conntrack+0x11c/0x1RAX: RBX: 8813d96dbdc8 RCX: 0001 15391 <4>RDX: b4be RSI: 0006 RDI: 8813d96dbdc8 15392 <4>RBP: 881d10cd9a08 R08: 0002 R09: 00c0 15393 <4>R10: R11: R12: 81b18540 15394 <4>R13: 0003 R14: 0002 R15: 0002 15395 <4>FS: 7f45d2ba1720() GS:88011f60() knlGS: 15396 <4>CS: 0010 DS: ES: CR0: 8005003b 15397 <4>CR2: CR3: 001f0aa3f000 CR4: 001427e0 15398 <4>DR0: DR1: DR2: 15399 <4>DR3: DR6: fffe0ff0 DR7: 0400 15400 <4>Process mebs-nbd-core (pid: 25449, threadinfo 881d10cd8000, task 881f0a9b6aa0) 15401 <4>Stack: 15402 <4> 881d10cd9ab8 81b18540 881d10cd9a18 8148a4c7 15403 <4> 881d10cd9a38 a029bac5 881c47af1b68 81b18540 15404 <4> 881d10cd9b18 a029bdd0 a026d620 a02a7080 15405 <4>Call Trace: 15406 <4> [] nf_conntrack_destroy+0x17/0x30 15407 <4> [] nf_conntrack_find_get+0x85/0x120 [nf_conntrack] 15408 <4> [] nf_conntrack_in+0x270/0xa00 [nf_conntrack] 15409 <4> [] ? ovs_dp_process_received_packet+0x51/0x80 [openvswitch] 15410 <4> [] ipv4_conntrack_local+0x51/0x60 [nf_conntrack_ipv4] 15411 <4> [] nf_iterate+0x69/0xb0 15412 <4> [] ? dst_output+0x0/0x20 15413 <4> [] nf_hook_slow+0x76/0x120 15414 <4> [] ? dst_output+0x0/0x20 15415 <4> [] __ip_local_out+0x9f/0xb0 15416 <4> [] ip_local_out+0x16/0x30 15417 <4> [] ip_queue_xmit+0x190/0x420 15418 <4> [] ? ip_output+0xb8/0xc0 15419 <4> [] tcp_transmit_skb+0x40e/0x7b0 15420 <4> [] tcp_write_xmit+0x230/0xa90 15421 troy_conntrack+0x11c/0x150 [] __tcp_push_pending_frames+0x30/0xe0 15422 <4> [] tcp_send_fin+0x72/0x1d0 15423 <4> [] tcp_close+0x346/0x450 15424 <4> [] inet_release+0x67/0x90 15425 <4> [] sock_release+0x29/0x90 15426 <4> [] sock_close+0x17/0x30 15427 <4> [] __fput+0xf5/0x210 15428 <4> [] fput+0x25/0x30 15429 <4> [] filp_close+0x5d/0x90 15430 <4> [] sys_close+0xa5/0x100 15431 <4> [] system_call_fastpath+0x16/0x1b 15432 <4>Code: 1b ec 1e e1 eb ce f6 05 9c fa d3 e1 20 74 ce 80 3d a0 ad 00 00 00 74 c5 48 89 de 48 c7 c7 00 32 2a a0 31 c0 e8 ca cc 28 e1 eb b2 <0f> 0b eb fe f6 05 73 fa d3 e1 20 0f 84 f6 fe ff ff 80 3d 9b ad 15433 <1>RIP [] destroy_conntrack+0x11c/0x150 [nf_conntrack] 15434 <4> RSP To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1715279/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1715279] Re: 2.6.32 netfliter . nf_conntrack . destroy_conntrack BUG
Would it be possible for you to test the latest upstream kernel? Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest v4.13 kernel[0]. If this bug is fixed in the mainline kernel, please add the following tag 'kernel-fixed-upstream'. If the mainline kernel does not fix this bug, please add the tag: 'kernel-bug-exists-upstream'. Once testing of the upstream kernel is complete, please mark this bug as "Confirmed". Thanks in advance. [0] http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.13 ** Changed in: linux (Ubuntu) Importance: Undecided => Medium ** Changed in: linux (Ubuntu) Status: Confirmed => Incomplete -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1715279 Title: 2.6.32 netfliter . nf_conntrack . destroy_conntrack BUG Status in linux package in Ubuntu: Incomplete Bug description: In my server ,i got a BUG，and i can initially locate where this bug is : nf_conntrack_in-> resolve_normal_ct -> nf_conntrack_find_get -> unlikely(!nf_ct_tuple_equal(tuple, >tuple)) -> nf_ct_put -> destroy_conntrack -> nf_ct_is_confirmed -> BUG_ON but I don't know why this happens，please help me ,THANK YOU! 4>[ cut here ] 15381 <2>kernel BUG at net/netfilter/nf_conntrack_core.c:206! 15382 <4>invalid opcode: [#1] SMP 15383 <4>last sysfs file: /sys/devices/system/cpu/cpu47/topology/thread_siblings_list 15384 <4>CPU 0 15385 <4>Modules linked in: nbd(U) iptable_mangle iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 iptable_filter ip_tables ksl b_ctk(U) fuse tcp_diag inet_diag xfs exportfs kvm_intel kvm vhost_net macvtap macvlan cdc_ether usbnet mii tun openvswitch(U) libcrc32c b onding 8021q garp stp llc toa hookers ipv6 power_meter iTCO_wdt iTCO_vendor_support dcdbas ipmi_devintf lpc_ich mfd_core shpchp igb i2c_a lgo_bit i2c_core ixgbe dca ptp pps_core mdio sg ext4 jbd2 mbcache sd_mod crc_t10dif ahci megaraid_sas wmi dm_mirror dm_region_hash dm_log dm_mod [last unloaded: nbd] 15386 <4> 15387 <4>Pid: 25449, comm: mebs-nbd-core Tainted: G --- H 2.6.32-431.20.3.el6.mt20161028.x86_64 #1 Dell Inc. PowerEdge R 730/0WCJNT 15388 <4>RIP: 0010:[] [] destroy_conntrack+0x11c/0x150 [nf_conntrack] 15389 <4>RSP: 0018:881d10cd99f8 EFLAGS: 00010246 15390 troy_conntrack+0x11c/0x1RAX: RBX: 8813d96dbdc8 RCX: 0001 15391 <4>RDX: b4be RSI: 0006 RDI: 8813d96dbdc8 15392 <4>RBP: 881d10cd9a08 R08: 0002 R09: 00c0 15393 <4>R10: R11: R12: 81b18540 15394 <4>R13: 0003 R14: 0002 R15: 0002 15395 <4>FS: 7f45d2ba1720() GS:88011f60() knlGS: 15396 <4>CS: 0010 DS: ES: CR0: 8005003b 15397 <4>CR2: CR3: 001f0aa3f000 CR4: 001427e0 15398 <4>DR0: DR1: DR2: 15399 <4>DR3: DR6: fffe0ff0 DR7: 0400 15400 <4>Process mebs-nbd-core (pid: 25449, threadinfo 881d10cd8000, task 881f0a9b6aa0) 15401 <4>Stack: 15402 <4> 881d10cd9ab8 81b18540 881d10cd9a18 8148a4c7 15403 <4> 881d10cd9a38 a029bac5 881c47af1b68 81b18540 15404 <4> 881d10cd9b18 a029bdd0 a026d620 a02a7080 15405 <4>Call Trace: 15406 <4> [] nf_conntrack_destroy+0x17/0x30 15407 <4> [] nf_conntrack_find_get+0x85/0x120 [nf_conntrack] 15408 <4> [] nf_conntrack_in+0x270/0xa00 [nf_conntrack] 15409 <4> [] ? ovs_dp_process_received_packet+0x51/0x80 [openvswitch] 15410 <4> [] ipv4_conntrack_local+0x51/0x60 [nf_conntrack_ipv4] 15411 <4> [] nf_iterate+0x69/0xb0 15412 <4> [] ? dst_output+0x0/0x20 15413 <4> [] nf_hook_slow+0x76/0x120 15414 <4> [] ? dst_output+0x0/0x20 15415 <4> [] __ip_local_out+0x9f/0xb0 15416 <4> [] ip_local_out+0x16/0x30 15417 <4> [] ip_queue_xmit+0x190/0x420 15418 <4> [] ? ip_output+0xb8/0xc0 15419 <4> [] tcp_transmit_skb+0x40e/0x7b0 15420 <4> [] tcp_write_xmit+0x230/0xa90 15421 troy_conntrack+0x11c/0x150 [] __tcp_push_pending_frames+0x30/0xe0 15422 <4> [] tcp_send_fin+0x72/0x1d0 15423 <4> [] tcp_close+0x346/0x450 15424 <4> [] inet_release+0x67/0x90 15425 <4> [] sock_release+0x29/0x90 15426 <4> [] sock_close+0x17/0x30 15427 <4> [] __fput+0xf5/0x210 15428 <4> [] fput+0x25/0x30 15429 <4> [] filp_close+0x5d/0x90 15430 <4> [] sys_close+0xa5/0x100 15431 <4> [] system_call_fastpath+0x16/0x1b 15432 <4>Code: 1b ec 1e e1 eb ce f6 05 9c fa d3 e1 20 74 ce 80 3d a0 ad 00 00 00 74 c5 48 89 de 48 c7 c7 00 32 2a a0 31 c0 e8 ca cc 28 e1 eb b2 <0f> 0b eb fe f6 05 73 fa d3 e1 20 0f 84 f6 fe ff ff 80 3d 9b ad 15433
[Kernel-packages] [Bug 1715279] Re: 2.6.32 netfliter . nf_conntrack . destroy_conntrack BUG
** Changed in: linux (Ubuntu) Status: Incomplete => Confirmed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1715279 Title: 2.6.32 netfliter . nf_conntrack . destroy_conntrack BUG Status in linux package in Ubuntu: Confirmed Bug description: In my server ,i got a BUG，and i can initially locate where this bug is : nf_conntrack_in-> resolve_normal_ct -> nf_conntrack_find_get -> unlikely(!nf_ct_tuple_equal(tuple, >tuple)) -> nf_ct_put -> destroy_conntrack -> nf_ct_is_confirmed -> BUG_ON but I don't know why this happens，please help me ,THANK YOU! 4>[ cut here ] 15381 <2>kernel BUG at net/netfilter/nf_conntrack_core.c:206! 15382 <4>invalid opcode: [#1] SMP 15383 <4>last sysfs file: /sys/devices/system/cpu/cpu47/topology/thread_siblings_list 15384 <4>CPU 0 15385 <4>Modules linked in: nbd(U) iptable_mangle iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 iptable_filter ip_tables ksl b_ctk(U) fuse tcp_diag inet_diag xfs exportfs kvm_intel kvm vhost_net macvtap macvlan cdc_ether usbnet mii tun openvswitch(U) libcrc32c b onding 8021q garp stp llc toa hookers ipv6 power_meter iTCO_wdt iTCO_vendor_support dcdbas ipmi_devintf lpc_ich mfd_core shpchp igb i2c_a lgo_bit i2c_core ixgbe dca ptp pps_core mdio sg ext4 jbd2 mbcache sd_mod crc_t10dif ahci megaraid_sas wmi dm_mirror dm_region_hash dm_log dm_mod [last unloaded: nbd] 15386 <4> 15387 <4>Pid: 25449, comm: mebs-nbd-core Tainted: G --- H 2.6.32-431.20.3.el6.mt20161028.x86_64 #1 Dell Inc. PowerEdge R 730/0WCJNT 15388 <4>RIP: 0010:[] [] destroy_conntrack+0x11c/0x150 [nf_conntrack] 15389 <4>RSP: 0018:881d10cd99f8 EFLAGS: 00010246 15390 troy_conntrack+0x11c/0x1RAX: RBX: 8813d96dbdc8 RCX: 0001 15391 <4>RDX: b4be RSI: 0006 RDI: 8813d96dbdc8 15392 <4>RBP: 881d10cd9a08 R08: 0002 R09: 00c0 15393 <4>R10: R11: R12: 81b18540 15394 <4>R13: 0003 R14: 0002 R15: 0002 15395 <4>FS: 7f45d2ba1720() GS:88011f60() knlGS: 15396 <4>CS: 0010 DS: ES: CR0: 8005003b 15397 <4>CR2: CR3: 001f0aa3f000 CR4: 001427e0 15398 <4>DR0: DR1: DR2: 15399 <4>DR3: DR6: fffe0ff0 DR7: 0400 15400 <4>Process mebs-nbd-core (pid: 25449, threadinfo 881d10cd8000, task 881f0a9b6aa0) 15401 <4>Stack: 15402 <4> 881d10cd9ab8 81b18540 881d10cd9a18 8148a4c7 15403 <4> 881d10cd9a38 a029bac5 881c47af1b68 81b18540 15404 <4> 881d10cd9b18 a029bdd0 a026d620 a02a7080 15405 <4>Call Trace: 15406 <4> [] nf_conntrack_destroy+0x17/0x30 15407 <4> [] nf_conntrack_find_get+0x85/0x120 [nf_conntrack] 15408 <4> [] nf_conntrack_in+0x270/0xa00 [nf_conntrack] 15409 <4> [] ? ovs_dp_process_received_packet+0x51/0x80 [openvswitch] 15410 <4> [] ipv4_conntrack_local+0x51/0x60 [nf_conntrack_ipv4] 15411 <4> [] nf_iterate+0x69/0xb0 15412 <4> [] ? dst_output+0x0/0x20 15413 <4> [] nf_hook_slow+0x76/0x120 15414 <4> [] ? dst_output+0x0/0x20 15415 <4> [] __ip_local_out+0x9f/0xb0 15416 <4> [] ip_local_out+0x16/0x30 15417 <4> [] ip_queue_xmit+0x190/0x420 15418 <4> [] ? ip_output+0xb8/0xc0 15419 <4> [] tcp_transmit_skb+0x40e/0x7b0 15420 <4> [] tcp_write_xmit+0x230/0xa90 15421 troy_conntrack+0x11c/0x150 [] __tcp_push_pending_frames+0x30/0xe0 15422 <4> [] tcp_send_fin+0x72/0x1d0 15423 <4> [] tcp_close+0x346/0x450 15424 <4> [] inet_release+0x67/0x90 15425 <4> [] sock_release+0x29/0x90 15426 <4> [] sock_close+0x17/0x30 15427 <4> [] __fput+0xf5/0x210 15428 <4> [] fput+0x25/0x30 15429 <4> [] filp_close+0x5d/0x90 15430 <4> [] sys_close+0xa5/0x100 15431 <4> [] system_call_fastpath+0x16/0x1b 15432 <4>Code: 1b ec 1e e1 eb ce f6 05 9c fa d3 e1 20 74 ce 80 3d a0 ad 00 00 00 74 c5 48 89 de 48 c7 c7 00 32 2a a0 31 c0 e8 ca cc 28 e1 eb b2 <0f> 0b eb fe f6 05 73 fa d3 e1 20 0f 84 f6 fe ff ff 80 3d 9b ad 15433 <1>RIP [] destroy_conntrack+0x11c/0x150 [nf_conntrack] 15434 <4> RSP To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1715279/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
