Bug#580050: linux-image-2.6.32-3-amd64: kcryptd crashes under , heavy I/O

[Dekar]

Can be closed since it was a hardware problem.
> Hello,
>
> this turned out to be a hardware problem after all. With different hardware
> everything works just fine.
>
> Regards,
> Juha




-- 
To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4c3f9ca4.8090...@wc3edit.net

Bug#580050: linux-image-2.6.32-3-amd64: kcryptd crashes under heavy I/O

[Juha Koho]

Package: linux-2.6
Version: 2.6.32-9
Severity: critical
Justification: breaks the whole system

Hello,

I have the following setup in my system: 6 x 500GB drives with software RAID6 + 
encryption (luks) + LVM. This is a new installation and I'm having troubles 
with encryption. Every once and a while kcryptd crashes and system becomes 
unresponsive. Well it responds to ping and currently running applications will 
continue running (if they don't need disk access I suppose) but I'm unable to 
ssh to the box anymore or run any new applications.

These crashes (always?) happen when there are lots of I/O going on. Ie. I can 
reproduce these crashes easily.

I have tested this with latest stable kernel version 2.6.33.3 but the problem 
persists.

I don't know if this is related to this or a different bug but I also noticed 
that when I transfer lots of files over nfs from my previous system about 5% of 
these files are corrupted. Ie. checksums do not match. Sometimes I need to 
transfer these files several times before everything is transferred ok. I'm 
able to transfer these files to my other box with no problems. Data gets 
corrupted only when transferring files to the box having this kcryptd problem.

Nothing appears in system logs after the crash but I was able to get the 
following using netconsole:

[ 3295.969539] [ cut here ]
[ 3295.969580] kernel BUG at 
/build/mattems-linux-2.6_2.6.32-9-amd64-NYTFdD/linux-2.6-2.6.32-9/debian/build/source_amd64_none/include/linux/scatterlist.h:63!
[ 3295.969627] invalid opcode:  [#1] SMP 
[ 3295.969654] last sysfs file: /sys/module/nfsd/initstate
[ 3295.969678] CPU 1 
[ 3295.969699] Modules linked in: autofs4 nfsd exportfs nfs lockd fscache 
nfs_acl auth_rpcgss sunrpc ext2 netconsole configfs loop snd_hda_codec_realtek 
snd_hda_intel psmouse snd_hda_codec snd_hwdep parport_pc snd_pcm edac_core 
parport asus_atk0110 snd_timer serio_raw edac_mce_amd snd pcspkr evdev 
i2c_nforce2 soundcore wmi snd_page_alloc i2c_core button processor ext4 mbcache 
jbd2 crc16 sha256_generic cryptd aes_x86_64 aes_generic cbc dm_crypt dm_mod 
raid456 async_raid6_recov async_pq raid6_pq async_xor xor async_memcpy async_tx 
raid1 md_mod sd_mod crc_t10dif ata_generic ide_pci_generic ahci ohci_hcd 
amd74xx floppy ehci_hcd libata forcedeth scsi_mod ide_core usbcore nls_base 
thermal thermal_sys [last unloaded: scsi_wait_scan]
[ 3295.970250] Pid: 450, comm: kcryptd Not tainted 2.6.32-3-amd64 #1 System 
Product Name
[ 3295.970287] RIP: 0010:[]  [] 
crypt_convert+0xe9/0x269 [dm_crypt]
[ 3295.970336] RSP: 0018:88012c39fd80  EFLAGS: 00010206
[ 3295.970360] RAX: 0002 RBX: 8800358533c0 RCX: 0002
[ 3295.970386] RDX: b2901491843e73a7 RSI:  RDI: 26726b70
[ 3295.970412] RBP: 88012b86f330 R08:  R09: 0001
[ 3295.970438] R10: 88012ba548d0 R11:  R12: 88003585fff0
[ 3295.970464] R13: 88012b86f200 R14: 88012c351000 R15: 
[ 3295.970492] FS:  7fb5063f86f0() GS:88000548() 
knlGS:
[ 3295.970530] CS:  0010 DS: 0018 ES: 0018 CR0: 8005003b
[ 3295.970554] CR2: 7f5f81faf000 CR3: 00012054c000 CR4: 06e0
[ 3295.970580] DR0:  DR1:  DR2: 
[ 3295.970606] DR3:  DR6: 0ff0 DR7: 0400
[ 3295.970633] Process kcryptd (pid: 450, threadinfo 88012c39e000, task 
88012cb0cdb0)
[ 3295.970669] Stack:
[ 3295.970687]  880035853408 0003 8801d000 
88012b86f338
[ 3295.970776] <0> 88012ba54800 8800ceaf83c0  
0202
[ 3295.970827] <0> 880035853390 df00  
a0187fec
[ 3295.970892] Call Trace:
[ 3295.970919]  [] ? kcryptd_crypt+0x40f/0x432 [dm_crypt]
[ 3295.970951]  [] ? worker_thread+0x188/0x21d
[ 3295.970979]  [] ? kcryptd_crypt+0x0/0x432 [dm_crypt]
[ 3295.971008]  [] ? autoremove_wake_function+0x0/0x2e
[ 3295.971035]  [] ? worker_thread+0x0/0x21d
[ 3295.971060]  [] ? kthread+0x79/0x81
[ 3295.971085]  [] ? child_rip+0xa/0x20
[ 3295.971110]  [] ? kthread+0x0/0x81
[ 3295.971134]  [] ? child_rip+0x0/0x20
[ 3295.971156] Code: 0c 48 8d 45 08 48 89 5d 00 48 89 c7 48 89 44 24 18 e8 80 
8b 00 e1 49 8b 14 24 41 8b 7c 24 0c 8b 73 30 48 8b 4d 08 f6 c2 03 74 04 <0f> 0b 
eb fe 44 89 f8 4c 8b 7c 24 20 83 e1 03 48 09 ca 48 c1 e0 
[ 3295.971523] RIP  [] crypt_convert+0xe9/0x269 [dm_crypt]
[ 3295.971556]  RSP 
[ 3295.971808] ---[ end trace 568ed39004d975a7 ]---


-- Package-specific info:
** Version:
Linux version 2.6.32-3-amd64 (Debian 2.6.32-9) (m...@debian.org) (gcc version 
4.3.4 (Debian 4.3.4-8) ) #1 SMP Wed Feb 24 18:07:42 UTC 2010

** Command line:
BOOT_IMAGE=/vmlinuz-2.6.32-3-amd64 root=/dev/mapper/vg0-root ro quiet

** Not tainted

** Kernel log:
[   26.285144] processor LNXCPU:03: registered as cooling_device3
[   26.292428] input: Power Bu

Bug#580050: linux-image-2.6.32-3-amd64: kcryptd crashes under heavy I/O

[Ben Hutchings]

On Mon, 2010-05-03 at 14:19 +0300, Juha Koho wrote:
> Package: linux-2.6
> Version: 2.6.32-9
> Severity: critical
> Justification: breaks the whole system
> 
> Hello,
> 
> I have the following setup in my system: 6 x 500GB drives with
> software RAID6 + encryption (luks) + LVM. This is a new installation
> and I'm having troubles with encryption. Every once and a while
> kcryptd crashes and system becomes unresponsive. Well it responds to
> ping and currently running applications will continue running (if they
> don't need disk access I suppose) but I'm unable to ssh to the box
> anymore or run any new applications.
> 
> These crashes (always?) happen when there are lots of I/O going on.
> Ie. I can reproduce these crashes easily.
> 
> I have tested this with latest stable kernel version 2.6.33.3 but the
> problem persists.
[...]

Please report this at  under product
'IO/Storage', component 'LVM2/DM', and let us know their bug number so
we can track it.

Ben.

-- 
Ben Hutchings
Once a job is fouled up, anything done to improve it makes it worse.


signature.asc
Description: This is a digitally signed message part

Bug#580050: linux-image-2.6.32-3-amd64: kcryptd crashes under heavy I/O

[Juha]

On Mon, May 3, 2010 at 4:13 PM, Ben Hutchings  wrote:
> Please report this at  under product
> 'IO/Storage', component 'LVM2/DM', and let us know their bug number so
> we can track it.

Reported. Bug number is 15902.

Regards,
Juha



-- 
To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/h2s1cf9b8bf1005030632m16c396e3p395a48ad7cb9...@mail.gmail.com

Bug#580068: Bug#580050: linux-image-2.6.32-3-amd64: kcryptd crashes under heavy I/O

[Ben Hutchings]

On Mon, 2010-05-03 at 14:19 +0300, Juha Koho wrote:
[...]
> I don't know if this is related to this or a different bug but I also
> noticed that when I transfer lots of files over nfs from my previous
> system about 5% of these files are corrupted. Ie. checksums do not
> match. Sometimes I need to transfer these files several times before
> everything is transferred ok. I'm able to transfer these files to my
> other box with no problems. Data gets corrupted only when transferring
> files to the box having this kcryptd problem.
[...]

This might or might not be related.  I have split it off as bug #580068.

Please use 'reportbug -N 580068' to add information about the network
configuration for your computer.

Ben.

-- 
Ben Hutchings
Once a job is fouled up, anything done to improve it makes it worse.


signature.asc
Description: This is a digitally signed message part

Bug#580068: Bug#580050: linux-image-2.6.32-3-amd64: kcryptd crashes under heavy I/O

[Juha]

On Mon, May 3, 2010 at 4:18 PM, Ben Hutchings  wrote:
> This might or might not be related.  I have split it off as bug #580068.
>
> Please use 'reportbug -N 580068' to add information about the network
> configuration for your computer.

After some more testing I noticed that the same problem occurs also
when transferring files over ssh (tested with rsync). So this is
definitely not a nfs problem.

What information about network settings would you like to see?

Juha



--
To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/s2y1cf9b8bf1005040346v6a814c84t79355b24386c5...@mail.gmail.com

Bug#580068: Bug#580050: linux-image-2.6.32-3-amd64: kcryptd crashes under heavy I/O

[Ben Hutchings]

On Tue, 2010-05-04 at 13:46 +0300, Juha wrote:
> On Mon, May 3, 2010 at 4:18 PM, Ben Hutchings  wrote:
> > This might or might not be related.  I have split it off as bug #580068.
> >
> > Please use 'reportbug -N 580068' to add information about the network
> > configuration for your computer.
> 
> After some more testing I noticed that the same problem occurs also
> when transferring files over ssh (tested with rsync). So this is
> definitely not a nfs problem.
> 
> What information about network settings would you like to see?

The bug script will collect some automatically if you allow it to.

Ben.

-- 
Ben Hutchings
Once a job is fouled up, anything done to improve it makes it worse.


signature.asc
Description: This is a digitally signed message part

Bug#580068: Bug#580050: linux-image-2.6.32-3-amd64: kcryptd crashes under heavy I/O

[Juha]

Hello,

this turned out to be a hardware problem so this bug can be closed.

Regards,
Juha



-- 
To UNSUBSCRIBE, email to debian-kernel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/aanlktikxdaxzxddp5jp_up32wcmunb1egniipzari...@mail.gmail.com