[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
I'd like to follow up because the issue seems to have cleared up for us after installing linux 5.0.1 about 40 days ago. It's hard to say whether everyone is experiencing the same bugs, but give 5.x a shot and let us know how it goes! Just to recap. Every week or so we were seeing R/O file systems with the follow errors, which required reboot & fsck. EXT4-fs error (device vda2): ext4_mb_generate_buddy:757: group 144, block bitmap and bg descriptor inconsistent: 23914 vs 23913 free clusters Aborting journal on device vda2-8. EXT4-fs (vda2): Remounting filesystem read-only We never experienced any corruption on the host itself, only under KVM guests. Host DELL Poweredge 2950III Several KVM Guests: linux OS, distro&kernel doesn't make any difference, all randomly vulnerable during periods of high disk activity. Not sure it matters, but in our case we were using LVM2 volumes on the host and kvm media was configured as follows "media=disk,if=virtio,cache=none,aio=native,format=raw". We initially thought just one guest was affected, but over time we saw it happen with many distros and kernels. It wasn't until we had an extended period of downtime that we decided to reinstall the host with a 5.x kernel. None of the guests experienced any issues since, fingers crossed. At this point, it's hard to recommend Ubuntu 19.04 given that it's only a few months away from EOL, however the 5.x kernel seems promising whereas the Ubuntu 18.04LTS runs an older kernel that is still known to exhibit the corruption. For LTS I'd look into running it under a custom setup with a newer kernel. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
** Tags added: cscc -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
Launchpad has imported 7 comments from the remote bug at https://bugzilla.kernel.org/show_bug.cgi?id=104571. If you reply to an imported comment from within Launchpad, your comment will be sent to the remote bug automatically. Read more about Launchpad's inter-bugtracker facilities at https://help.launchpad.net/InterBugTracking. On 2015-09-15T08:10:58+00:00 linux-ext4 wrote: This bug report is about ext4 metadata corruption on large (>=10TB) ext4 volumes. This was also reported in 2014 [ http://marc.info/?l=linux-ext4&m=139878494527370&w=2 ] I'm getting sporadic FS errors like this one: (More of these i've pasted at https://8n1.org/10745/cc34) | EXT4-fs error (device vdb): ext4_mb_generate_buddy:757: | group 79842, block bitmap and bg descriptor inconsistent: 10073 vs 10071 | free clusters | Aborting journal on device vdb-8. An e2fsck run then shows: | Pass 5: Checking group summary information | Block bitmap differences: +(2616281446--2616281447) | Free blocks count wrong (170942497, counted=129906218). | Free inodes count wrong (670863012, counted=670860975). I've patched my kernel with WARN_ON(1); inserted in tactical places and caught one such situation: | EXT4-fs (vdb): pa 880016544888: logic 982168, phys. 2469410748, len 104 | EXT4-fs error (device vdb): ext4_mb_release_inode_pa:3773: group 75360, free 38, pa_free 36 | Aborting journal on device vdb-8. | EXT4-fs (vdb): Remounting filesystem read-only | [ cut here ] | WARNING: CPU: 1 PID: 1706 at fs/ext4/mballoc.c:3774 ext4_mb_release_inode_pa.isra.27+0x1cb/0x2c0() | Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter xt_tcpudp ip6_tables | nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables | x_tables cirrus ttm drm_kms_helper drm kvm_intel kvm ppdev syscopyarea sysfillrect | 8250_fintek serio_raw i2c_piix4 sysimgblt pvpanic parport_pc mac_hid nfsd auth_rpcgss nfs_acl | lockd grace sunrpc lp parport autofs4 psmouse floppy pata_acpi | CPU: 1 PID: 1706 Comm: deluged Not tainted 3.19.8-ckt4 #1 | Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 | 81ab4fef 8800da1bb978 817c3760 0007 | 8800da1bb9b8 8107696a 8800da1bb9a8 | 0026 3825 3824 880016544888 | Call Trace: | [] dump_stack+0x45/0x57 | [] warn_slowpath_common+0x8a/0xc0 | [] warn_slowpath_null+0x1a/0x20 | [] ext4_mb_release_inode_pa.isra.27+0x1cb/0x2c0 | [] ? ext4_read_block_bitmap_nowait+0x26f/0x5f0 | [] ext4_discard_preallocations+0x30a/0x490 | [] ext4_da_update_reserve_space+0x178/0x1b0 | [] ext4_ext_map_blocks+0xcd9/0xe50 | [] ext4_map_blocks+0x129/0x570 | [] ? ext4_writepages+0x35d/0xca0 | [] ? __ext4_journal_start_sb+0x69/0xe0 | [] ext4_writepages+0x582/0xca0 | [] do_writepages+0x1e/0x30 | [] __filemap_fdatawrite_range+0x59/0x60 | [] filemap_write_and_wait+0x2c/0x60 | [] do_vfs_ioctl+0x3fd/0x4e0 | [] SyS_ioctl+0x81/0xa0 | [] system_call_fastpath+0x16/0x1b | ---[ end trace c7de4d0d78cb95b6 ]--- | EXT4-fs error (device vdb) in ext4_writepages:2412: IO failure | EXT4-fs (vdb): ext4_writepages: jbd2_start: 9223372036854775751 pages, ino 84149503; err -30 After this, the system started logging a lot of this same message: | EXT4-fs error (device vdb): ext4_find_extent:900: inode #84149503: comm deluged: |pblk 225181822 bad header/extent: invalid magic - magic 53fd, entries 37907, |max 27407(0), depth 50401(0) Ran e2fsck and got: | Pass 5: Checking group summary information | Block bitmap differences: +(1431556444--1431556445) +(2469410748-2469410749) | Free blocks count wrong (134030133, counted=57970467). | Free inodes count wrong (670746893, counted=670746452). Which is usually the same output for fsck in these situations. This server is a QEMU KVM virtual machine running on Intel x64 hardware. Reply at: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/comments/34 On 2015-09-28T08:04:48+00:00 linux-ext4 wrote: Hit by this bug again: | EXT4-fs error (device vdb): ext4_mb_generate_buddy:757: group 76916, |block bitmap and bg descriptor inconsistent: 959 vs 957 free |clusters | Aborting journal on device vdb-8. | EXT4-fs (vdb): Remounting filesystem read-only | [ cut here ] | WARNING: CPU: 0 PID: 5000 at fs/ext4/mballoc.c:758 |ext4_mb_generate_buddy+0x1fa/0x340() | Modules linked in: btrfs xor raid6_pq ufs qnx4 hfsplus hfs minix ntfs |msdos jfs xfs libcrc32c xt_tcpudp nf_conntrack_ipv6 nf_defrag_ipv6 |ip6table_filter ip6_tables nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack |nf_conntrack iptable_filter ip_tables x_tables cirrus ttm drm_kms_helper |drm kvm_intel kvm ppdev syscopyarea sysfillrect serio_raw pvpanic |sysimgblt 8250_fintek p
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
Unfortunately the symptom still persists on bionic (Ubuntu 18.04.1 LTS) with kernel 4.15.0-43-generic. This issue may be a symptom that's caused by lice and fleas. I updated https://bugzilla.kernel.org/show_bug.cgi?id=104571 by adding a comment with additional information. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
Chenyuchai, yes, https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?h=7dec5603b6b8dc4c3e1c65d318bd2a5a8c62a424 is the fix. It was integrated in the Trusty kernel 3.13.0-87.133 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
Hi,all: I need your confirmation.Is the Fix: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?h=7dec5603b6b8dc4c3e1c65d318bd2a5a8c62a424 to solve this issue?If not,pls show the patch,thank you very much! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
** Tags removed: verification-needed-trusty ** Tags added: verification-done-trusty -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
** Changed in: linux (Debian) Status: Confirmed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
This bug was fixed in the package linux-lts-utopic - 3.16.0-73.95~14.04.1 --- linux-lts-utopic (3.16.0-73.95~14.04.1) trusty; urgency=low [ Kamal Mostafa ] * CVE-2016-1583 (LP: #1588871) - ecryptfs: fix handling of directory opening - SAUCE: proc: prevent stacking filesystems on top - SAUCE: ecryptfs: forbid opening files without mmap handler -- Andy Whitcroft Thu, 09 Jun 2016 08:46:24 +0100 ** Changed in: linux-lts-utopic (Ubuntu Trusty) Status: Fix Committed => Fix Released ** CVE added: http://www.cve.mitre.org/cgi- bin/cvename.cgi?name=2016-1583 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
This bug was fixed in the package linux - 3.13.0-87.133 --- linux (3.13.0-87.133) trusty; urgency=low [ Kamal Mostafa ] * Release Tracking Bug - LP: #1585315 [ Upstream Kernel Changes ] * Revert "usb: hub: do not clear BOS field during reset device" - LP: #1582864 linux (3.13.0-87.132) trusty; urgency=low [ Kamal Mostafa ] * Release Tracking Bug - LP: #1582398 [ Kamal Mostafa ] * [Config] Drop ozwpan from the ABI [ Luis Henriques ] * [Config] CONFIG_USB_WPAN_HCD=n - LP: #1463740 - CVE-2015-4004 [ Prarit Bhargava ] * SAUCE: (no-up) ACPICA: Dispatcher: Update thread ID for recursive method calls - LP: #1577898 [ Upstream Kernel Changes ] * usbnet: cleanup after bind() in probe() - LP: #1567191 - CVE-2016-3951 * KVM: x86: bit-ops emulation ignores offset on 64-bit - LP: #1423672 * USB: usbip: fix potential out-of-bounds write - LP: #1572666 - CVE-2016-3955 * x86/mm/32: Enable full randomization on i386 and X86_32 - LP: #1568523 - CVE-2016-3672 * Input: gtco - fix crash on detecting device without endpoints - LP: #1575706 - CVE-2016-2187 * atl2: Disable unimplemented scatter/gather feature - LP: #1561403 - CVE-2016-2117 * ALSA: usb-audio: Skip volume controls triggers hangup on Dell USB Dock - LP: #1577905 * fs/pnode.c: treat zero mnt_group_id-s as unequal - LP: #1572316 * propogate_mnt: Handle the first propogated copy being a slave - LP: #1572316 * drm: Balance error path for GEM handle allocation - LP: #1579610 * x86/mm: Add barriers and document switch_mm()-vs-flush synchronization - LP: #1538429 - CVE-2016-2069 * x86/mm: Improve switch_mm() barrier comments - LP: #1538429 - CVE-2016-2069 * net: fix infoleak in llc - LP: #1578496 - CVE-2016-4485 * net: fix infoleak in rtnetlink - LP: #1578497 - CVE-2016-4486 -- Kamal Mostafa Tue, 24 May 2016 11:04:30 -0700 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
This bug was fixed in the package linux - 3.13.0-87.133 --- linux (3.13.0-87.133) trusty; urgency=low [ Kamal Mostafa ] * Release Tracking Bug - LP: #1585315 [ Upstream Kernel Changes ] * Revert "usb: hub: do not clear BOS field during reset device" - LP: #1582864 linux (3.13.0-87.132) trusty; urgency=low [ Kamal Mostafa ] * Release Tracking Bug - LP: #1582398 [ Kamal Mostafa ] * [Config] Drop ozwpan from the ABI [ Luis Henriques ] * [Config] CONFIG_USB_WPAN_HCD=n - LP: #1463740 - CVE-2015-4004 [ Prarit Bhargava ] * SAUCE: (no-up) ACPICA: Dispatcher: Update thread ID for recursive method calls - LP: #1577898 [ Upstream Kernel Changes ] * usbnet: cleanup after bind() in probe() - LP: #1567191 - CVE-2016-3951 * KVM: x86: bit-ops emulation ignores offset on 64-bit - LP: #1423672 * USB: usbip: fix potential out-of-bounds write - LP: #1572666 - CVE-2016-3955 * x86/mm/32: Enable full randomization on i386 and X86_32 - LP: #1568523 - CVE-2016-3672 * Input: gtco - fix crash on detecting device without endpoints - LP: #1575706 - CVE-2016-2187 * atl2: Disable unimplemented scatter/gather feature - LP: #1561403 - CVE-2016-2117 * ALSA: usb-audio: Skip volume controls triggers hangup on Dell USB Dock - LP: #1577905 * fs/pnode.c: treat zero mnt_group_id-s as unequal - LP: #1572316 * propogate_mnt: Handle the first propogated copy being a slave - LP: #1572316 * drm: Balance error path for GEM handle allocation - LP: #1579610 * x86/mm: Add barriers and document switch_mm()-vs-flush synchronization - LP: #1538429 - CVE-2016-2069 * x86/mm: Improve switch_mm() barrier comments - LP: #1538429 - CVE-2016-2069 * net: fix infoleak in llc - LP: #1578496 - CVE-2016-4485 * net: fix infoleak in rtnetlink - LP: #1578497 - CVE-2016-4486 -- Kamal Mostafa Tue, 24 May 2016 11:04:30 -0700 ** Changed in: linux (Ubuntu Trusty) Status: Fix Committed => Fix Released ** CVE added: http://www.cve.mitre.org/cgi- bin/cvename.cgi?name=2015-4004 ** CVE added: http://www.cve.mitre.org/cgi- bin/cvename.cgi?name=2016-2069 ** CVE added: http://www.cve.mitre.org/cgi- bin/cvename.cgi?name=2016-2117 ** CVE added: http://www.cve.mitre.org/cgi- bin/cvename.cgi?name=2016-2187 ** CVE added: http://www.cve.mitre.org/cgi- bin/cvename.cgi?name=2016-3672 ** CVE added: http://www.cve.mitre.org/cgi- bin/cvename.cgi?name=2016-3951 ** CVE added: http://www.cve.mitre.org/cgi- bin/cvename.cgi?name=2016-3955 ** CVE added: http://www.cve.mitre.org/cgi- bin/cvename.cgi?name=2016-4485 ** CVE added: http://www.cve.mitre.org/cgi- bin/cvename.cgi?name=2016-4486 ** Changed in: linux (Ubuntu Trusty) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
Louie, https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=818502#22 provides a simple way to reproduce. If you could give it a try that would be appreciated. FYI, when my hypervisor was running Trusty (3.13), the problem was reproducible on fresh VMs with brand new ext4 FSes, so hopefully that will be easy to reproduce for you. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
I have a "PE 2950III Intel(R) Xeon(R) CPU X5460 @ 3.16GHz" server here and I've been trying to test this out. I'm using an "rsync" copy of an original server exhibiting the problem. So far though I've been unable to reproduce the original error at all. It would seem that using the exact same OS/kernel/binaries, the error doesn't happen on a fresh filesystem, I guess there must have been something about the filesystem image itself that triggered the fault. So my dilemma is that I don't know how to reproduce this fault on a fresh install. So while I can test this update, I'm not sure how valid the test will be on an installation that isn't faulting. Does anyone have a suggestion or have an idea about how to reproduce the conditions? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
Anyone affected by this issue: We're looking for a positive verification that the Trusty kernel version currently available in -proposed (3.13.0-87.132) fixes this problem. If you can provide that confirmation, please do! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed- trusty' to 'verification-done-trusty'. If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you! ** Tags added: verification-needed-trusty -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
I recently reinstalled the affected host to run Xenial so I can no longer test the proposed fix for the 3.13 kernel. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
** Changed in: linux (Ubuntu Trusty) Status: Confirmed => Fix Committed ** Changed in: linux-lts-utopic (Ubuntu Trusty) Status: New => Fix Committed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
Additional positive test result notes: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=818502#27 https://bugzilla.kernel.org/show_bug.cgi?id=102731#c43 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
** Description changed: + SRU Justification: + + Impact: Users of VMs running 3.13/3.16 and ext4 can experience data corruption in the guest. + Fix: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?h=7dec5603b6b8dc4c3e1c65d318bd2a5a8c62a424 + Testcase: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=818502#22 + + -- + I noticed that one of my VM had this "dmesg -T" output: [Tue Feb 17 09:53:27 2015] systemd-udevd[5433]: starting version 204 [Thu Feb 19 06:25:08 2015] EXT4-fs error (device vda1): ext4_mb_generate_buddy:756: group 5, block bitmap and bg descriptor inconsistent: 16446 vs 16445 free clusters [Thu Feb 19 06:25:09 2015] Aborting journal on device vda1-8. [Thu Feb 19 06:25:09 2015] EXT4-fs (vda1): Remounting filesystem read-only [Thu Feb 19 06:25:09 2015] [ cut here ] [Thu Feb 19 06:25:09 2015] WARNING: CPU: 0 PID: 9946 at /build/buildd/linux-3.13.0/fs/ext4/ext4_jbd2.c:259 __ext4_handle_dirty_metadata+0x1a2/0x1c0() [Thu Feb 19 06:25:09 2015] Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables x_tables serio_raw psmouse floppy [Thu Feb 19 06:25:09 2015] CPU: 0 PID: 9946 Comm: logrotate Not tainted 3.13.0-45-generic #74-Ubuntu [Thu Feb 19 06:25:09 2015] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011 [Thu Feb 19 06:25:09 2015] 0009 880003a11aa0 81720eb6 [Thu Feb 19 06:25:09 2015] 880003a11ad8 810677cd 88c41340 [Thu Feb 19 06:25:09 2015] 88000a58e000 81835280 1302 880003a11ae8 [Thu Feb 19 06:25:09 2015] Call Trace: [Thu Feb 19 06:25:09 2015] [] dump_stack+0x45/0x56 [Thu Feb 19 06:25:09 2015] [] warn_slowpath_common+0x7d/0xa0 [Thu Feb 19 06:25:09 2015] [] warn_slowpath_null+0x1a/0x20 [Thu Feb 19 06:25:09 2015] [] __ext4_handle_dirty_metadata+0x1a2/0x1c0 [Thu Feb 19 06:25:09 2015] [] ? ext4_dirty_inode+0x2a/0x60 [Thu Feb 19 06:25:09 2015] [] ext4_free_blocks+0x646/0xbf0 [Thu Feb 19 06:25:09 2015] [] ? wake_up_bit+0x25/0x30 [Thu Feb 19 06:25:09 2015] [] ext4_ext_rm_leaf+0x505/0x8f0 [Thu Feb 19 06:25:09 2015] [] ? __ext4_ext_check+0x197/0x370 [Thu Feb 19 06:25:09 2015] [] ? ext4_ext_remove_space+0xc0/0x7e0 [Thu Feb 19 06:25:09 2015] [] ext4_ext_remove_space+0x31c/0x7e0 [Thu Feb 19 06:25:09 2015] [] ext4_ext_truncate+0xb0/0xe0 [Thu Feb 19 06:25:09 2015] [] ext4_truncate+0x379/0x3c0 [Thu Feb 19 06:25:09 2015] [] ext4_evict_inode+0x408/0x4d0 [Thu Feb 19 06:25:09 2015] [] evict+0xb0/0x1b0 [Thu Feb 19 06:25:09 2015] [] iput+0xf5/0x180 [Thu Feb 19 06:25:09 2015] [] __dentry_kill+0x1a8/0x200 [Thu Feb 19 06:25:09 2015] [] dput+0xa5/0x180 [Thu Feb 19 06:25:09 2015] [] __fput+0x176/0x260 [Thu Feb 19 06:25:09 2015] [] fput+0xe/0x10 [Thu Feb 19 06:25:09 2015] [] task_work_run+0xa7/0xe0 [Thu Feb 19 06:25:09 2015] [] do_notify_resume+0x97/0xb0 [Thu Feb 19 06:25:09 2015] [] int_signal+0x12/0x17 [Thu Feb 19 06:25:09 2015] ---[ end trace ebff9843d81b5c41 ]--- [Thu Feb 19 06:25:09 2015] BUG: unable to handle kernel NULL pointer dereference at 0028 [Thu Feb 19 06:25:09 2015] IP: [] __ext4_error_inode+0x31/0x160 - [Thu Feb 19 06:25:09 2015] PGD 167067 PUD 161067 PMD 0 - [Thu Feb 19 06:25:09 2015] Oops: [#1] SMP + [Thu Feb 19 06:25:09 2015] PGD 167067 PUD 161067 PMD 0 + [Thu Feb 19 06:25:09 2015] Oops: [#1] SMP [Thu Feb 19 06:25:09 2015] Modules linked in: nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner xt_conntrack nf_conntrack iptable_filter ip_tables x_tables serio_raw psmouse floppy [Thu Feb 19 06:25:09 2015] CPU: 0 PID: 9946 Comm: logrotate Tainted: G W 3.13.0-45-generic #74-Ubuntu [Thu Feb 19 06:25:09 2015] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011 [Thu Feb 19 06:25:09 2015] task: 880009ac4800 ti: 880003a1 task.ti: 880003a1 [Thu Feb 19 06:25:09 2015] RIP: 0010:[] [] __ext4_error_inode+0x31/0x160 [Thu Feb 19 06:25:09 2015] RSP: :880003a11a58 EFLAGS: 00010292 [Thu Feb 19 06:25:09 2015] RAX: RBX: RCX: 0086 [Thu Feb 19 06:25:09 2015] RDX: 1302 RSI: 81a6e81f RDI: [Thu Feb 19 06:25:09 2015] RBP: 880003a11ae8 R08: 81a78568 R09: 0005 [Thu Feb 19 06:25:09 2015] R10: ffe2 R11: 880003a117ce R12: 0086 [Thu Feb 19 06:25:09 2015] R13: 81835280 R14: 1302 R15: 81a78568 [Thu Feb 19 06:25:09 2015] FS: 7f74eaca4840() GS:88000b80() knlGS: [Thu Feb 19 06:25:09 2015] CS: 0010 DS: ES
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
** Changed in: linux (Ubuntu Trusty) Assignee: (unassigned) => Chris J Arges (arges) ** Changed in: linux-lts-utopic (Ubuntu Trusty) Assignee: (unassigned) => Chris J Arges (arges) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
Thanks Chris. So far, no regression with the Trusty kernel: $ uname -a Linux xeon 3.13.0-86-generic #130~lp1423672v201604200743 SMP Wed Apr 20 12:44:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Since the issue only happens rarely, more people testing it would be welcome. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
Leo, Also uploaded a lts-utopic build here: http://people.canonical.com/~arges/lp1423672/ It will all debs with '3.16' in the name. ** Also affects: linux-lts-utopic (Ubuntu) Importance: Undecided Status: New ** Changed in: linux-lts-utopic (Ubuntu) Status: New => Invalid -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
Hi Chris, This is also seen on kernel 3.16.0-51-generic. Could you get us a kernel build test for 3.16 as well ? Thank you Leo -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
Here's a test build for trusty 3.13 with https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?h=7dec5603b6b8dc4c3e1c65d318bd2a5a8c62a424 applied: http://people.canonical.com/~arges/lp1423672/ Can someone verify this does fix the issue so this can be SRU'ed into 3.13? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
Good work. We also have PE2950III systems running "Intel(R) Xeon(R) CPU X5460 @ 3.16GHz". If this is indeed the fix, I'm confused why it would only affect certain cpus? https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?h=7dec5603b6b8dc4c3e1c65d318bd2a5a8c62a424 I'll have to come up with a plan to replace debian's stable/jessie kernel with an unmanaged one on the host. I'm not keen on doing that as the DRAC units on these are not very reliable... -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
Here we have this problem on two servers runnning "Intel(R) Xeon(R) CPU X5460 @ 3.16GHz" (and debian 7.0 backport kernel 3.16 on host, 4.x on VM). We doesn't seems to have this problem on others nodes runnning: Intel(R) Xeon(R) CPU E5345 @ 2.33GHz Intel(R) Xeon(R) CPU5160 @ 3.00GHz Intel(R) Xeon(R) CPU E5320 @ 1.86GHz Intel(R) Xeon(R) CPU X3470 @ 2.93GHz We are going to try an upgrade to see if it solves the problem. If yes, it's a pretty good news! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
Thanks Václav, your conclusion about older Intel CPU seems to match my setup since this only happens on a Xeon E3110 (which is in fact a re- branded Core2 Duo E8400). Thanks for bisecting this and figure the fix was: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?h=7dec5603b6b8dc4c3e1c65d318bd2a5a8c62a424 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
** Changed in: linux (Ubuntu) Status: Incomplete => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
This bug is probably what I describe in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=818502 Bug in Debian 3.16 kernel (kernel from current stable Jessie) used as KVM hypervisor on older Intel CPU. ** Bug watch added: Debian Bug tracker #818502 http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=818502 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
I am able to reproduce this bug every single time I suspend and then resume one of my laptops. Which is running xubun tu 15.10 with 4.2.0-16-generic kernel on a lenovo R400. I suspect possible a hardware problem as this only happens on 1 out of 3 R400 that I have. I'll report back if I remember. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
FYI, the same errors happened to me on real hardware (a Raspberry Pi 2 B), based on a custom Debian Jessie image. The image uses a ext2 filesystem created on a x86 Boot2docker host (Kernel "Linux fbd0c1340061 4.1.12-boot2docker #1 SMP Tue Nov 3 06:03:36 UTC 2015 x86_64 GNU/Linux"). While remounting the filesystem as r-w on the Raspberry and writing to it I got: [53406.370524] EXT4-fs (mmcblk0p3): mounting ext2 file system using the ext4 subsystem [53406.575883] EXT4-fs (mmcblk0p3): mounted filesystem without journal. Opts: (null) [53416.394833] EXT4-fs error (device mmcblk0p3): ext4_mb_generate_buddy:757: group 1, block bitmap and bg descriptor inconsistent: 8953 vs 8990 free clusters [53435.245967] EXT4-fs error (device mmcblk0p3): ext4_lookup:1417: inode #2: comm rsync: deleted inode referenced: 46849 [53481.805006] EXT4-fs error (device mmcblk0p3): ext4_lookup:1417: inode #2: comm ls: deleted inode referenced: 46849 The Raspi isn't running the most current Kernel yet (it's "Linux intermodul 3.19.3-v7 #1 SMP PREEMPT Mon Nov 30 08:37:00 UTC 2015 armv7l GNU/Linux"), but perhaps this helps analyzing this bug as it's not a VM... -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
Oops...the above kvm command line is correct but it did not crash with -m 1000, that's what production is using now. It was crashing consistently with -m 512 about a minute into the synthetic FS load. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
chris, Here is what you asked for, sorry for not getting it earlier. I don't use virsh. This is how I started KVM to trigger the problem interactively (curses interface): kvm -drive file=/dev/raid/shared,media=disk,if=none,cache=none,aio=native,format=raw,id=hd0 -device virtio-blk-pci,drive=hd0 -smp 2 -m 1000 -netdev tap,ifname=vm_shared,script=no,downscript=no,id=eth0 -device virtio-net- pci,netdev=eth0,mac=52:54:00:12:34:58 -name shared -runas shared -curses fdisk -l Disk /dev/vda: 100 GiB, 107374182400 bytes, 209715200 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: gpt Disk identifier: AE4BDF3E-0B83-4C17-B104-A5139722F263 Device Start End Sectors Size Type /dev/vda1 2048 3905535 3903488 1.9G Linux swap /dev/vda2 3905536 209713151 205807616 98.1G Linux filesystem It hasn't happened in this particular VM since upping the RAM so the VM doesn't swap. My intention was to reproduce on non-production hardware, and then try different kernels, rule out LVM, virtio, etc. But I'm in the middle of a new assignment, I probably won't have time to do this myself before December. ** Attachment added: "/boot/config-3.16.0-4-amd64 for the VM in question" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+attachment/4518472/+files/config.txt -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
** Changed in: linux (Debian) Status: Unknown => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
Looks like there are some more LKBT issues for this one: https://bugzilla.kernel.org/show_bug.cgi?id=102731 https://bugzilla.kernel.org/show_bug.cgi?id=89621 ** Bug watch added: Linux Kernel Bug Tracker #102731 http://bugzilla.kernel.org/show_bug.cgi?id=102731 ** Bug watch added: Linux Kernel Bug Tracker #89621 http://bugzilla.kernel.org/show_bug.cgi?id=89621 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
** Bug watch added: Debian Bug tracker #772848 http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=772848 ** Changed in: linux (Debian) Importance: Undecided => Unknown ** Changed in: linux (Debian) Status: New => Unknown ** Changed in: linux (Debian) Remote watch: None => Debian Bug tracker #772848 ** Also affects: linux-kernel-headers via http://bugzilla.kernel.org/show_bug.cgi?id=104571 Importance: Unknown Status: Unknown ** Project changed: linux-kernel-headers => linux -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
I'm running LoCo mirror for my country and this affects our file server for over a week already. All our official file services got offline now. We're also providing free VM/Hosting for OpenSource projects, I can't image how the disaster will become. Any suggestion? All VMs are VMware guest. root@ncnu-ftp:/ubuntu/mirror# LANG=C free -m total used free sharedbuffers cached Mem: 7984 1453 6531 0 66352 -/+ buffers/cache: 1033 6950 Swap: 4091 0 4091 root@ncnu-ftp:/ubuntu/mirror# LANG=C touch /lala touch: cannot touch '/lala': Read-only file system root@ncnu-ftp:/ubuntu/mirror# LANG=C mount -o remount,rw / mount: cannot remount block device /dev/mapper/ub--ftp-root read-write, is write-protected root@ncnu-ftp:/ubuntu/mirror# dumpe2fs /dev/mapper/ub--ftp-root /dev/|grep -i Filesystem\ state dumpe2fs 1.42.9 (4-Feb-2014) Filesystem state: clean with errors root@ncnu-ftp:/ubuntu/mirror# LC_ALL=C dmesg -T |grep error [Thu Nov 5 00:35:46 2015] EXT4-fs (dm-0): re-mounted. Opts: errors=remount-ro [Thu Nov 5 00:35:55 2015] EXT4-fs error (device dm-0): ext4_mb_generate_buddy:756: group 371, block bitmap and bg descriptor inconsistent: 25478 vs 25477 free clusters [Thu Nov 5 00:35:55 2015] IP: [] __ext4_error_inode+0x31/0x160 [Thu Nov 5 00:35:55 2015] RIP: 0010:[] [] __ext4_error_inode+0x31/0x160 [Thu Nov 5 00:35:55 2015] RIP [] __ext4_error_inode+0x31/0x160 [Thu Nov 5 00:35:55 2015] EXT4-fs error (device dm-0): ext4_remount:4816: Abort forced by user [Thu Nov 5 00:40:39 2015] EXT4-fs (dm-0): error count since last fsck: 15 [Thu Nov 5 00:40:39 2015] EXT4-fs (dm-0): initial error at time 1445904785: ext4_reserve_inode_write:4928 [Thu Nov 5 00:40:39 2015] EXT4-fs (dm-0): last error at time 1446654956: ext4_remount:4816 [Thu Nov 5 01:18:38 2015] EXT4-fs error (device dm-0): ext4_remount:4816: Abort forced by user [Thu Nov 5 01:18:59 2015] EXT4-fs error (device dm-0): ext4_remount:4816: Abort forced by user [Thu Nov 5 01:28:59 2015] EXT4-fs error (device dm-0): ext4_remount:4816: Abort forced by user [Thu Nov 5 01:58:08 2015] EXT4-fs error (device dm-0): ext4_remount:4816: Abort forced by user [Thu Nov 5 02:20:41 2015] EXT4-fs error (device dm-0): ext4_remount:4816: Abort forced by user root@ncnu-ftp:/ubuntu/mirror# uname -a Linux ncnu-ftp 3.13.0-63-generic #103-Ubuntu SMP Fri Aug 14 21:42:59 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux root@ncnu-ftp:/ubuntu/mirror# lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description:Ubuntu 14.04.3 LTS Release:14.04 Codename: trusty -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
Am 29.10.15 um 03:29 schrieb LouieGosselin: > I'm posting again to add that I conducted some more tests and ext3 does > not encounter corruption under the same conditions. I hope this > information is helpful to others, if anyone needs more information let > me know and I'll see what I can do. I'll probably switch my own VMs to > ext3 so I don't have to worry about these FS crashes. I created a new LVM Volume and a new ext3 FS. I copied the whole FS from ext4 to ext3 and running now the VM on ext3. Today the ext4_mb_generate_buddy bug did hit me agian. With ext3 FS and running the VM with "cache=none" setting. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
Louie, Can you post some additional information that may help in debugging this issue. 1) Can you post the output of 'virsh dumpxml ' of the affected VM? 2) Can you post the /boot/config-`uname -r` of the affected VM's kernel? 3) What type of partitioning layout does your VM have? It seems that it has been reproducible in 3.13 to 3.16 in this report, and the upstream report shows 3.19. I tried this in a local VM with 32GB disk, 2 cpu, 512MB memory, swapfile and could not trivially reproduce with the filebench test case you mentioned above. --chris -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
I'm posting again to add that I conducted some more tests and ext3 does not encounter corruption under the same conditions. I hope this information is helpful to others, if anyone needs more information let me know and I'll see what I can do. I'll probably switch my own VMs to ext3 so I don't have to worry about these FS crashes. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
It's happened again. I've spent several hours on this and I've been able to recreate the failure under some synthetic conditions with a sacrificial VM. The filebench defaults do not cause an ext4 crash for me, but the following do: load workloads/fileserver set $dir=/tmp/ set $nfiles=20 set $meandirwidth=3 run 120 The ext4 error never happens in the filebench'es init phase, only 50s or so into the 50 threaded run phase. Less extreme settings won't produce a consistent crash. Reducing the amount of free memory makes the errors much more likely. This is before running filebench: total used free sharedbuffers cached Mem: 482M99M 382M 300K27M20M -/+ buffers/cache:52M 429M Swap: 1.9G94M 1.8G This is while running filebench one second before the crash: total used free sharedbuffers cached Mem: 482M 476M 5.6M 284K27M18M -/+ buffers/cache: 430M51M Swap: 1.9G 253M 1.6G 2769.63 The error is reproducible in cloned VMs. Moving swap to another disk changes nothing. As far as I can tell, the error never happens with ext4 filesystems other than the root FS where executables are running from. I've tried bonnie, stress-ng, and simple scripts, I have not been able to get these to crash ext4. The sacrificial VM has not crashed after add an extra 500MB to it. Although production was never under such heavy loads, I've added 500MB to the production VM to see if it helps anyways. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
On 10/13/2015 06:00 PW, Simon Déziel wrote: > On 10/13/2015 11:50 AM, Jan Wagner wrote: >> I'm trying now '"cache=none"' for this VM and see how that turns out. > > FYI, all my VMs use cache=none and have small HDDs (lv of 2-4G) yet the > regularly trip on this bug. looks like this didn't fixed the problem for me. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
It happened here again. 8/24 ext4 corruption 9/14 ext4 corruption 9/29 update/reboot 10/16 ext4 corruption This time the corruption was severe. 1743 files from multiple directories got moved into lost+found. It took me almost 2 hours this morning to verify & fix everything. Fortunately every time this has happened, all the files were dated prior to the daily backup and "diff -qr ..." shows exactly what was lost. As far as I can tell this is not a memory issue and the ext4 FS is using 10G out of 100G. Every time the corruption has been in /var/mail. However the VM is mostly used for mail so it may not be significant. The /var/mail branch itself is 4.3G 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1+deb8u4 (2015-09-19) x86_64 GNU/Linux I'm holding back the kernels on other VM's so this is the only VM with the problem. Is anyone able to reproduce this on demand? I really need to do something because this is causing downtime during normal business hours. I'll probably try one of the following: 1. Rebuild the FS from scratch and see if ext4 corruption continues. 2. Use ext3 or something else. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
On 10/13/2015 11:50 AM, Jan Wagner wrote: > I got also hit with the problem on Debian Jessie VM [virtio] (KVM host > is Jessie as well). > > The following bug might be related: > https://bugzilla.kernel.org/show_bug.cgi?id=104571 > > I'm trying now '"cache=none"' for this VM and see how that turns out. FYI, all my VMs use cache=none and have small HDDs (lv of 2-4G) yet the regularly trip on this bug. Simon -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
I got also hit with the problem on Debian Jessie VM [virtio] (KVM host is Jessie as well). The following bug might be related: https://bugzilla.kernel.org/show_bug.cgi?id=104571 I'm trying now '"cache=none"' for this VM and see how that turns out. ** Bug watch added: Linux Kernel Bug Tracker #104571 http://bugzilla.kernel.org/show_bug.cgi?id=104571 ** Attachment added: "Screenshot_service_2015-10-13_17:25:18.png" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+attachment/4493620/+files/Screenshot_service_2015-10-13_17%3A25%3A18.png -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
Adding memory did not helps, but it seems to have fixed the issue by just removing the fstab option errors=remount-ro on the mountpoints having problems. It's usually only set on root fs and we have no problem on root fs, but by mistake this option was added to other mountpoints and triggered the bug. No more ro-remounts since one week and no ext4 messages in logs. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
I'm getting this same issue every day 3.2.0-4-amd64 #1 SMP Debian 3.2.68-1+deb7u4 x86_64 GNU/Linux Guest is running under proxmox 3.4 virtio -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
** Also affects: linux (Debian) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
Same problem here on 2 VMs, appeared 4 times in one week. VMs are running Debian Jessie, on Debian wheezy hypervisors running ceph giant, libvirt 1.2.9-9~bpo70+1, qemu 1:2.1+dfsg-12~bpo70+1. FS is ext4, using virtio and rbd with writeback. After first crash we upgraded kernel to 4.1.6-1~bpo8+1. But it crashed again next day :( Seems to be related to memory usage at least for last crash. Attached the trace. We added more memory to see if that helps to avoid triggering the bug. ** Attachment added: "kern.log with backtrace" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+attachment/4466843/+files/kern.log -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
Same error, on a single VM, virtio driver, ext4: EXT4-fs (vda1): pa 880010c3daf8: logic 512, pyhs. 6750208, len 512 EXT4-fs error (device vda1): ext4_mb_release_inode_pa:3773: group 206, free 143, pa_free 142 Aborting journal on device vda1-8 Disk quota active No memory error (checked) No disk error (checked too) At system reset, filesystem check and correction, quota data corrupted, quotacheck take long time to complete. The error is strictly correlated to filesystem activity, when raise, error is more frequent. ubuntu 14.04 all updated, kernel 3.16.0-49-generic -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
I'm on Debian, but it's happening to me as well. KVM with virtual disks backed by LVM volumes on the host. Both the VM and the host are running Linux version 3.16.0-4-amd64 (debian-ker...@lists.debian.org) (gcc version 4.8.4 (Debian 4.8.4-1) ) #1 SMP Debian 3.16.7-ckt11-1+deb8u2 (2015-07-17) First occurred on August 24th. Manual fsck required, lots of files converted to lost-inodes. [1992712.418275] EXT4-fs error (device vda2): ext4_mb_generate_buddy:757: group 96, block bitmap and bg descriptor inconsistent: 24017 vs 24015 free clusters [1992712.513438] Aborting journal on device vda2-8. [1992712.514007] EXT4-fs (vda2): Remounting filesystem read-only [1992712.514205] EXT4-fs error (device vda2) in ext4_evict_inode:243: Journal has aborted Happened again today September 14th in the same VM. [1489393.753098] EXT4-fs error (device vda2): ext4_mb_generate_buddy:757: group 144, block bitmap and bg descriptor inconsistent: 23914 vs 23913 free clusters [1489393.803865] Aborting journal on device vda2-8. [1489393.804439] EXT4-fs (vda2): Remounting filesystem read-only This is the first syslog activity since I rebooted in August, no block IO errors on the guest or host. Manual fsck required, files were lost, but everything is running again. It has not happened in other VMs running older kernels. It also has not happened on the host, however there's very little file system activity on the host. The fact that it hasn't happened on other VMs leads me to believe the bug is inside the guest rather than with KVM - perhaps ext4 or the virtual disk driver. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
We have the same error on a 14.04 VM running on a XenServer 6.2 (paravirtualized). EXT4-fs error (device dm-1): ext4_mb_generate_buddy:756: group 36, block bitmap and bg descriptor inconsistent: 30504 vs 30149 free clusters Linux xxx 3.13.0-63-generic #103-Ubuntu SMP Fri Aug 14 21:42:59 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux i created a new VM and restored a backup and run the daily cronjobs to reproduce it but it didn't happen again. run-parts -v --exit-on-error /etc/cron.daily output of dmesg -T ... [Mon Sep 14 14:47:00 2015] random: lvm urandom read with 64 bits of entropy available [Mon Sep 14 14:47:00 2015] bio: create slab at 1 [Mon Sep 14 14:47:00 2015] EXT4-fs (dm-1): INFO: recovery required on readonly filesystem [Mon Sep 14 14:47:00 2015] EXT4-fs (dm-1): write access will be enabled during recovery [Mon Sep 14 14:47:01 2015] random: nonblocking pool is initialized [Mon Sep 14 14:47:01 2015] EXT4-fs (dm-1): orphan cleanup on readonly fs [Mon Sep 14 14:47:01 2015] EXT4-fs (dm-1): 16 orphan inodes deleted [Mon Sep 14 14:47:01 2015] EXT4-fs (dm-1): recovery complete [Mon Sep 14 14:47:01 2015] EXT4-fs (dm-1): mounted filesystem with ordered data mode. Opts: (null) ... that seams to be ok. i also tried with the cron.weekly... -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
Same error, virtual machine 14.04 running on kvm, server 14.04. No hardware issue, the entire server was replaced after the second stop. No hard disk issue, the server is on raid hardware disk. The server is running more than one virtual server, only one affected. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
I'm having this exact issue with Debian Jessie. Virtual machines running with LVM storage unpredictably failing after cron.daily. [680406.357587] EXT4-fs error (device vda1): ext4_mb_generate_buddy:757: group 49, block bitmap and bg descriptor inconsistent: 18356 vs 18355 free clusters [680406.361696] Aborting journal on device vda1-8. [680406.453757] EXT4-fs error (device vda1): ext4_journal_check_start:56: [680406.453999] EXT4-fs (vda1): Remounting filesystem read-only [680406.457551] Detected aborted journal [766809.056039] EXT4-fs (vda1): error count since last fsck: 2 [766809.056069] EXT4-fs (vda1): initial error at time 1441383430: ext4_mb_generate_buddy:757 [766809.056082] EXT4-fs (vda1): last error at time 1441383430: ext4_journal_check_start:56 [853316.576033] EXT4-fs (vda1): error count since last fsck: 2 [853316.576052] EXT4-fs (vda1): initial error at time 1441383430: ext4_mb_generate_buddy:757 [853316.576066] EXT4-fs (vda1): last error at time 1441383430: ext4_journal_check_start:56 root@collect1 ~ # uname -a Linux collect1 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1+deb8u3 (2015-08-04) x86_64 GNU/Linux -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
Seems we are also confronted with this error in one of our VMWare guests: [45780.021968] EXT4-fs error (device dm-9): ext4_mb_generate_buddy:756: group 1411, block bitmap and bg descriptor inconsistent: 32757 vs 32768 free clusters [45780.022029] Aborting journal on device dm-9-8. [45780.055657] EXT4-fs (dm-9): Remounting filesystem read-only [45780.055733] EXT4-fs error (device dm-9) in ext4_orphan_add:2609: Journal has aborted It happened four times the last three weeks, pretty annoying. We have tried to "harden" ext4 in some way by using: * ext4 rw,relatime,nodelalloc,errors=remount-ro,commit=2 as mount options. We have also increased hung_task_timeout_secs as we have found some hints in a RedHat KB article. Any ideas on how to fix that ext4 error in the guest? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1423672] Re: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y
And yet again: root@rproxy:~# dmesg -T | tail [Thu Jun 11 15:28:56 2015] type=1400 audit(1434050938.364:24): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="nc" pid=658 comm="apparmor_parser" [Thu Jun 11 15:28:59 2015] init: plymouth-upstart-bridge main process ended, respawning [Thu Jun 11 15:29:02 2015] init: console-font main process (757) terminated with status 71 [Thu Jun 11 15:29:03 2015] init: plymouth-splash main process (815) terminated with status 1 [Thu Jun 11 18:18:40 2015] random: nonblocking pool is initialized [Mon Jun 15 16:53:35 2015] EXT4-fs (vda1): pa 8800061e1000: logic 3293, phys. 469944, len 424 [Mon Jun 15 16:53:35 2015] EXT4-fs error (device vda1): ext4_mb_release_inode_pa:3753: group 14, free 13, pa_free 11 [Mon Jun 15 16:53:35 2015] Aborting journal on device vda1-8. [Mon Jun 15 16:53:36 2015] EXT4-fs (vda1): Remounting filesystem read-only [Mon Jun 15 16:53:36 2015] EXT4-fs error (device vda1): ext4_journal_check_start:56: Detected aborted journal root@rproxy:~# uname -a Linux rproxy.dmz.sdeziel.info 3.13.0-54-generic #91-Ubuntu SMP Tue May 26 19:15:08 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1423672 Title: ext4_mb_generate_buddy:756: group N, block bitmap and bg descriptor inconsistent: X vs Y To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1423672/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs