[Kernel-packages] [Bug 1389787] Re: 3.11 memory consumption leads to HANG (not sure if 3.13 suffers from this).
I've hit this bug the second time within one week on a 24/7 Server system. Is there any workaround for this, currently I'm thinking about switching from ext4 to ext3. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1389787 Title: 3.11 memory consumption leads to HANG (not sure if 3.13 suffers from this). Status in linux package in Ubuntu: Confirmed Bug description: It was brought to my attention the following stack traces (occurring several times on different machines): crash> bt PID: 2877 TASK: 881009b42ee0 CPU: 1 COMMAND: "git" #0 [880c0c6bb9d0] machine_kexec at 8104b141 #1 [880c0c6bba40] crash_kexec at 810d5a58 #2 [880c0c6bbb10] oops_end at 81748b38 #3 [880c0c6bbb40] no_context at 8172dd02 #4 [880c0c6bbb90] __bad_area_nosemaphore at 8172dee4 #5 [880c0c6bbbf0] bad_area at 8172df5d #6 [880c0c6bbc20] __do_page_fault at 8174bda8 #7 [880c0c6bbd30] do_page_fault at 8174bde7 #8 [880c0c6bbd60] page_fault at 81747e98 [exception RIP: kmem_cache_alloc_trace+106] RIP: 8119b22a RSP: 880c0c6bbe18 RFLAGS: 00010206 RAX: RBX: 88008407e0c0 RCX: 031b6bc9 RDX: 031b6bc8 RSI: 80d0 RDI: 000173c0 RBP: 880c0c6bbe68 R8: 88081fa373c0 R9: R10: R11: 0202 R12: 88081f403800 R13: 00629050 R14: 811bcfe4 R15: 80d0 ORIG_RAX: CS: 0010 SS: 0018 #9 [880c0c6bbe70] alloc_pipe_info at 811bcfe4 #10 [880c0c6bbe90] get_pipe_inode at 811bd0aa #11 [880c0c6bbeb0] create_pipe_files at 811bd648 #12 [880c0c6bbef0] __do_pipe_flags at 811bd7d2 #13 [880c0c6bbf30] sys_pipe2 at 811bd8f0 #14 [880c0c6bbf70] sys_pipe at 811bd980 #15 [880c0c6bbf80] system_call_fastpath at 8175099d RIP: 7f2d245da7e7 RSP: 7fff6a4cfb18 RFLAGS: 00010283 RAX: 0016 RBX: 8175099d RCX: 001c RDX: 7f2d248aeac0 RSI: RDI: 7fff6a4cfad0 RBP: 0002 R8: R9: R10: R11: 0202 R12: 811bd980 R13: 880c0c6bbf78 R14: R15: 0232efc0 ORIG_RAX: 0016 CS: 0033 SS: 002b AND Oct 15 14:19:08 LGEARND8B5 kernel: [796493.152703] SysRq : Show backtrace of all active CPUs Oct 15 14:19:08 LGEARND8B5 kernel: [796493.152708] sending NMI to all CPUs: Oct 15 14:19:08 LGEARND8B5 kernel: [796493.152714] NMI backtrace for cpu 0 Oct 15 14:19:08 LGEARND8B5 kernel: [796493.152720] CPU: 0 PID: 13579 Comm: python Tainted: GF I 3.11.0-15-generic #25~precise1-Ubuntu Oct 15 14:19:08 LGEARND8B5 kernel: [796493.152721] Hardware name: HP ProLiant BL460c G7, BIOS I27 05/05/2011 Oct 15 14:19:08 LGEARND8B5 kernel: [796493.152724] task: 880085f5 ti: 880a64afa000 task.ti: 880a64afa000 Oct 15 14:19:08 LGEARND8B5 kernel: [796493.152725] RIP: 0010:[] [] __ticket_spin_lock+0x25/0x30 Oct 15 14:19:08 LGEARND8B5 kernel: [796493.152736] RSP: 0018:880a64afb748 EFLAGS: 0293 Oct 15 14:19:08 LGEARND8B5 kernel: [796493.152737] RAX: db25 RBX: 880be8576480 RCX: 00018066002a Oct 15 14:19:08 LGEARND8B5 kernel: [796493.152739] RDX: db2e RSI: 0001 RDI: 880be8576480 Oct 15 14:19:08 LGEARND8B5 kernel: [796493.152740] RBP: 880a64afb748 R08: R09: ea002407db00 Oct 15 14:19:08 LGEARND8B5 kernel: [796493.152741] R10: 8128d8ed R11: 0001 R12: 8807572480b0 Oct 15 14:19:08 LGEARND8B5 kernel: [796493.152743] R13: 8807572481b8 R14: R15: 8805e9c77908 Oct 15 14:19:08 LGEARND8B5 kernel: [796493.152745] FS: 7ff00d4db700() GS:880c0ba0() knlGS: Oct 15 14:19:08 LGEARND8B5 kernel: [796493.152746] CS: 0010 DS: ES: CR0: 80050033 Oct 15 14:19:08 LGEARND8B5 kernel: [796493.152748] CR2: 7fb66906d8e0 CR3: 00053b3dd000 CR4: 07f0 Oct 15 14:19:08 LGEARND8B5 kernel: [796493.152749] Stack: Oct 15 14:19:08 LGEARND8B5 kernel: [796493.152750] 880a64afb758 8174759e 880a64afb778 8128eb02 Oct 15 14:19:08 LGEARND8B5 kernel: [796493.152756] 8807572480b0 880757248200 880a64afb798 81270595 Oct 15 14:19:08 LGEARND8B5 kernel: [796493.152759] 880a64afb798 8807572480b0 880a64afb7c8 81257621 Oct 15 14:19:08 LGEARND8B5 kernel: [796493.152763] Call Trace: Oct 15 14:19:08 LGEARND8B5 kernel: [796493.152770] [] _raw_spin_lock+0xe/0x20 Oct 15 14:19:08 LGEARND8B5 kernel: [796493.152
[Kernel-packages] [Bug 1389787] Re: 3.11 memory consumption leads to HANG (not sure if 3.13 suffers from this).
I'm also affected by this bug on Trusty with the latest kernel (3.13.0-53-generic): [500345.624596] BUG: soft lockup - CPU#0 stuck for 23s! [khugepaged:54] [500345.630989] Modules linked in: ipt_REJECT btrfs ufs qnx4 hfsplus[500345.636604] BUG: soft lockup - CPU#1 stuck for 23s! [kthreadd:2] [500345.636605] Modules linked in: ipt_REJECT btrfs ufs qnx4 hfsplus hfs minix ntfs msdos jfs xt_multiport iptable_filter ip_tables x_tables kvm_amd edac_core kvm k10temp edac_mce_amd i2c_piix4 serio_raw shpchp mac_hid nfsd auth_rpcgss nfs_acl nfs lp binfmt_misc lockd sunrpc fscache parport xfs libcrc32c xts gf128mul dm_crypt raid10 raid0 multipath linear raid456 async_memcpy async_raid6_recov async_pq async_xor async_tx xor raid6_pq raid1 pata_acpi hid_generic usbhid hid radeon psmouse i2c_algo_bit ahci ttm libahci drm_kms_helper pata_atiixp mpt2sas drm r8169 raid_class scsi_transport_sas mii wmi [500345.636627] CPU: 1 PID: 2 Comm: kthreadd Tainted: G D 3.13.0-53-generic #89-Ubuntu [500345.636628] Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./M3A785GMH/128M, BIOS P1.40 11/26/2009 [500345.636628] task: 880214bd9800 ti: 880214458000 task.ti: 880214458000 [500345.636629] RIP: 0010:[] [] _raw_spin_lock+0x35/0x50 [500345.636631] RSP: 0018:8802144599f8 EFLAGS: 0202 [500345.636632] RAX: 79f7 RBX: 8127f5bc RCX: 87ea [500345.636632] RDX: 87ec RSI: 87ec RDI: 8800cdef5c80 [500345.636633] RBP: 8802144599f8 R08: 9038 R09: 0107e5b3481c [500345.636633] R10: feda1a8eb150d207 R11: 03eb R12: 0046 [500345.636634] R13: 8115e67e R14: 880214459a20 R15: [500345.636635] FS: 7fa8ed61e740() GS:88021fc4() knlGS: [500345.636636] CS: 0010 DS: ES: CR0: 8005003b [500345.636636] CR2: 7f8263077000 CR3: 000210041000 CR4: 07e0 [500345.636637] Stack: [500345.636637] 880214459a18 8127ffb9 880107e5b2a8 880107e5b2a8 [500345.636639] 880214459a30 81260091 880107e5b3f8 880214459a50 [500345.636641] 8124614a 880107e5b2a8 880107e5b3b0 880214459a78 [500345.636642] Call Trace: [500345.636643] [] ext4_es_lru_del+0x29/0x70 [500345.636644] [] ext4_clear_inode+0x41/0x90 [500345.636646] [] ext4_evict_inode+0x8a/0x4d0 [500345.636647] [] evict+0xb0/0x1b0 [500345.636649] [] dispose_list+0x39/0x50 [500345.636651] [] prune_icache_sb+0x47/0x60 [500345.636652] [] super_cache_scan+0x105/0x170 [500345.636654] [] shrink_slab+0x1c7/0x370 [500345.636655] [] do_try_to_free_pages+0x3ed/0x540 [500345.636657] [] try_to_free_pages+0xec/0x180 [500345.636658] [] __alloc_pages_nodemask+0x7d5/0xb80 [500345.636660] [] ? load_balance+0x120/0x890 [500345.636662] [] copy_process.part.26+0x143/0x16b0 [500345.636663] [] ? sched_clock_cpu+0xb5/0x100 [500345.636664] [] ? kthread_create_on_node+0x1c0/0x1c0 [500345.63] [] do_fork+0xd5/0x340 [500345.636667] [] kernel_thread+0x26/0x30 [500345.636668] [] kthreadd+0x15a/0x1c0 [500345.636670] [] ? kthread_create_on_cpu+0x60/0x60 [500345.636671] [] ret_from_fork+0x58/0x90 [500345.636673] [] ? kthread_create_on_cpu+0x60/0x60 [500345.636674] Code: 00 00 02 00 f0 0f c1 07 89 c2 c1 ea 10 66 39 c2 75 02 5d c3 83 e2 fe 0f b7 f2 b8 00 80 00 00 eb 0c 0f 1f 44 00 00 f3 90 83 e8 01 <74> 0a 0f b7 0f 66 39 ca 75 f1 5d c3 66 66 66 90 66 66 90 eb da [500345.648614] BUG: soft lockup - CPU#2 stuck for 23s! [aria2c:11547] [500345.648614] Modules linked in: ipt_REJECT btrfs ufs qnx4 hfsplus hfs minix ntfs msdos jfs xt_multiport iptable_filter ip_tables x_tables kvm_amd edac_core kvm k10temp edac_mce_amd i2c_piix4 serio_raw shpchp mac_hid nfsd auth_rpcgss nfs_acl nfs lp binfmt_misc lockd sunrpc fscache parport xfs libcrc32c xts gf128mul dm_crypt raid10 raid0 multipath linear raid456 async_memcpy async_raid6_recov async_pq async_xor async_tx xor raid6_pq raid1 pata_acpi hid_generic usbhid hid radeon psmouse i2c_algo_bit ahci ttm libahci drm_kms_helper pata_atiixp mpt2sas drm r8169 raid_class scsi_transport_sas mii wmi [500345.648637] CPU: 2 PID: 11547 Comm: aria2c Tainted: G D 3.13.0-53-generic #89-Ubuntu [500345.648637] Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./M3A785GMH/128M, BIOS P1.40 11/26/2009 [500345.648638] task: 8800cfb48000 ti: 88021455 task.ti: 88021455 [500345.648639] RIP: 0010:[] [] _raw_spin_lock+0x32/0x50 [500345.648640] RSP: :880214551910 EFLAGS: 0202 [500345.648641] RAX: 5af7 RBX: 88010cbfdec0 RCX: 87ea [500345.648642] RDX: 87ee RSI: 87ee RDI: 8800cdef5c80 [500345.648642] RBP: 880214551910 R08: 2038 R09: 010cbfe0101c [500345.648643] R10: fed540631fdc0407 R11: 03eb R12: 880214551900 [500345.648644] R13: 8802145518d0 R14: f
[Kernel-packages] [Bug 997695] Re: severe performance issue using mdadm raid5 on top of LUKS
I'm sorry, but I've changed my setup in the meantime. Currently I'm using a RAID6 made of 9 Seagate Barracuda 7200.14 (3TB) and a single crypt device on top of the raid. I've also changed my CPU (AMD A8-5500) and am now using AES-NI to accelerate encryption. I've done a raid grow last week, and my IOStats graph shows between 800K an 1.2M blocks per second. I've also done some quick performance tests using dd and get a write performance around 400MB/s using a blocksize of 4K. The IO graph show 300K blocks per second while my CPU is at 100% an all four cores. top shows also four different kworker processes, the dd and a _raid6 process. I guess this bug is fixes (at least for me). If I can provide any additional information, just let me know. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/997695 Title: severe performance issue using mdadm raid5 on top of LUKS Status in “cryptsetup” package in Ubuntu: Invalid Status in “linux” package in Ubuntu: Incomplete Bug description: I've upgraded to 12.04 a few days ago and since this, the performance of my crypto-drive is limited by CPU. I'm running an MDADM RAID5 on top of 7 independendtly encrypted drives (Seagate Barracuda XT 7200.12 2TB each encrypted using crypsetup / LUKS). The problem seems to be, that in 12.04 all encryption is done using only a single CPU core. In 10.04 each crypt-device used its own thread (kcrypt), that was the reason to encrypt each device independently and build a raid on top of the crypto devices. I'll attach a screenshot of munins iostat and CPU graph to illustrate the difference before and after the upgrade --- AlsaDevices: total 0 crw-rw---T 1 root audio 116, 1 Mai 1 15:52 seq crw-rw---T 1 root audio 116, 33 Mai 1 15:52 timer AplayDevices: aplay: device_list:252: no soundcards found... ApportVersion: 2.0.1-0ubuntu7 Architecture: amd64 ArecordDevices: arecord: device_list:252: no soundcards found... AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: Error: command ['iw', 'reg', 'get'] failed with exit code 1: nl80211 not found. DistroRelease: Ubuntu 12.04 HibernationDevice: RESUME= IwConfig: lono wireless extensions. virbr0no wireless extensions. eth0 no wireless extensions. Lsusb: Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 004 Device 002: ID 051d:0002 American Power Conversion Uninterruptible Power Supply MachineType: To Be Filled By O.E.M. To Be Filled By O.E.M. Package: linux (not installed) ProcEnviron: LANGUAGE=de_DE:de:en_US:en TERM=xterm LANG=de_DE.UTF-8 SHELL=/bin/bash ProcFB: 0 nouveaufb ProcKernelCmdLine: root=/dev/mapper/md1_crypt ro quiet splash ProcVersionSignature: Ubuntu 3.2.0-24.37-generic 3.2.14 RelatedPackageVersions: linux-restricted-modules-3.2.0-24-generic N/A linux-backports-modules-3.2.0-24-generic N/A linux-firmware1.79 RfKill: Error: [Errno 2] No such file or directory StagingDrivers: cxd2099 Tags: precise staging Uname: Linux 3.2.0-24-generic x86_64 UpgradeStatus: Upgraded to precise on 2012-04-30 (10 days ago) UserGroups: mythtv dmi.bios.date: 04/30/2010 dmi.bios.vendor: American Megatrends Inc. dmi.bios.version: P2.30 dmi.board.name: K10N78FullHD-hSLI.. dmi.board.vendor: ASRock dmi.chassis.asset.tag: To Be Filled By O.E.M. dmi.chassis.type: 3 dmi.chassis.vendor: To Be Filled By O.E.M. dmi.chassis.version: To Be Filled By O.E.M. dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvrP2.30:bd04/30/2010:svnToBeFilledByO.E.M.:pnToBeFilledByO.E.M.:pvrToBeFilledByO.E.M.:rvnASRock:rnK10N78FullHD-hSLI..:rvr:cvnToBeFilledByO.E.M.:ct3:cvrToBeFilledByO.E.M.: dmi.product.name: To Be Filled By O.E.M. dmi.product.version: To Be Filled By O.E.M. dmi.sys.vendor: To Be Filled By O.E.M. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cryptsetup/+bug/997695/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
