[Bug 543617] Re: very slow filesystem I/O

2010-05-04 Thread Kees Cook
The sync()-in-kernel work-around only seems to help on an otherwise idle system. I've opened an upstream and Fedora bug now, since I've been able trivially reproduce it both on current Fedora and the latest stable upstream kernel. ** Bug watch added: Linux Kernel Bug Tracker #15906 http://bugz

Re: [Bug 543617] Re: very slow filesystem I/O

2010-04-13 Thread Theodore Ts'o
On Tue, Apr 13, 2010 at 09:13:49PM -, Phillip Susi wrote: > On 4/13/2010 4:30 PM, Launchpad Bug Tracker wrote: > > * SAUCE: sync before umount to reduce time taken by ext4 umount > > - LP: #543617 > > This sounds more like a temporary workaround than a fix of the real bug. > Is that the

Re: [Bug 543617] Re: very slow filesystem I/O

2010-04-13 Thread Phillip Susi
On 4/13/2010 4:30 PM, Launchpad Bug Tracker wrote: > * SAUCE: sync before umount to reduce time taken by ext4 umount > - LP: #543617 This sounds more like a temporary workaround than a fix of the real bug. Is that the case and why? Just can't find the real problem, or it will take too long

[Bug 543617] Re: very slow filesystem I/O

2010-04-13 Thread Launchpad Bug Tracker
This bug was fixed in the package linux - 2.6.32-21.31 --- linux (2.6.32-21.31) lucid; urgency=low [ Andy Whitcroft ] * allow modules.builtin to be optional * d-i: add mpt2sas to the message-modules udeb - LP: #530361 [ Christopher James Halse Rogers ] * SAUCE: Nouvea

[Bug 543617] Re: very slow filesystem I/O

2010-04-13 Thread Andy Whitcroft
** Changed in: linux (Ubuntu Lucid) Status: In Progress => Fix Committed -- very slow filesystem I/O https://bugs.launchpad.net/bugs/543617 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@l

[Bug 543617] Re: very slow filesystem I/O

2010-04-04 Thread Surbhi Palande
I did not read Teds comments above earlier :( ! Here is what i found out too: 1) barrier is called for every inode - but this happens for ext3 code too and ext3 unmount is not taking as much time. 2) journal transaction is also calling (one more) barrier for every inode. 2) putting timing inform

[Bug 543617] Re: very slow filesystem I/O

2010-04-02 Thread Andy Whitcroft
** Changed in: linux (Ubuntu Lucid) Milestone: ubuntu-10.04-beta-2 => ubuntu-10.04 -- very slow filesystem I/O https://bugs.launchpad.net/bugs/543617 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu

[Bug 543617] Re: very slow filesystem I/O

2010-04-01 Thread Surbhi Palande
There is a discussion on ext4 mailing list about improving the jbd2 layer (and not making journalling sync in two phases - however I will like to discuss this solution there.) I have also a few more findings...will try making a good solution out of it and get back here. -- very slow filesystem I/

[Bug 543617] Re: very slow filesystem I/O

2010-04-01 Thread Surbhi Palande
Kees Cook, I debugged this problem and found out that this is happening because of waiting on ext4 journals. I can see currently there is a ext4 thread going on for doing journal commit in two phases: Phase 1) wake up the journal commit threads (Do some other work) Phase 2) wait for the commit to

[Bug 543617] Re: very slow filesystem I/O

2010-04-01 Thread Theodore Ts'o
The other place you'll want to drop a sync call is in whatever path gets used when someone unmounts a file system in Nautilus (or any other desktop file manager) by clicking on the "eject" icon. This is the one that is going to be more visible to users, since a common usage scenario is someone wh

[Bug 543617] Re: very slow filesystem I/O

2010-03-31 Thread Kees Cook
I can confirm that add "sync" at the top of /etc/schroot/setup.d/10mount solves all the build stalls I've been seeing. -- very slow filesystem I/O https://bugs.launchpad.net/bugs/543617 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. --

[Bug 543617] Re: very slow filesystem I/O

2010-03-31 Thread Theodore Ts'o
I can confirm this using a bleeding edge kernel 2.6.34-rc2+, and when using a tmpfs mounted for /tmp, it takes about two minutes on a T400 laptop. Using blktrace, it looks like we're doing a whole ton of journal writes after the umount. Here's the blktrace summary of the reproducer: Total (loop0

[Bug 543617] Re: very slow filesystem I/O

2010-03-31 Thread Surbhi Palande
** Changed in: linux (Ubuntu Lucid) Status: Confirmed => In Progress -- very slow filesystem I/O https://bugs.launchpad.net/bugs/543617 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists

[Bug 543617] Re: very slow filesystem I/O

2010-03-31 Thread Kees Cook
@Surbhi: I can't use lock_stat because the stalled mount triggers lockdep warnings. -- very slow filesystem I/O https://bugs.launchpad.net/bugs/543617 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bu

[Bug 543617] Re: very slow filesystem I/O

2010-03-30 Thread Surbhi Palande
** Changed in: linux (Ubuntu Lucid) Assignee: (unassigned) => Surbhi Palande (csurbhi) -- very slow filesystem I/O https://bugs.launchpad.net/bugs/543617 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list u

[Bug 543617] Re: very slow filesystem I/O

2010-03-29 Thread Kees Cook
Might be upstream bug https://bugzilla.kernel.org/show_bug.cgi?id=14430 ** Bug watch added: Linux Kernel Bug Tracker #14430 http://bugzilla.kernel.org/show_bug.cgi?id=14430 -- very slow filesystem I/O https://bugs.launchpad.net/bugs/543617 You received this bug notification because you are a

[Bug 543617] Re: very slow filesystem I/O

2010-03-29 Thread Kees Cook
# grep LOCK_STAT /boot/config-$(uname -r) # CONFIG_LOCK_STAT is not set -- very slow filesystem I/O https://bugs.launchpad.net/bugs/543617 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubu

[Bug 543617] Re: very slow filesystem I/O

2010-03-29 Thread Surbhi Palande
@Kees, Can you please do the following: 1) echo 1 >/proc/sys/kernel/lock_stat fire the unmount command. 2) less /proc/lock_stat If you could attach the output of the previous command then we could check if there is some lock contention occurring. You could then disable lock_stat as follows: 3) e

[Bug 543617] Re: very slow filesystem I/O

2010-03-29 Thread Surbhi Palande
** Changed in: linux (Ubuntu Lucid) Importance: Undecided => High -- very slow filesystem I/O https://bugs.launchpad.net/bugs/543617 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu

[Bug 543617] Re: very slow filesystem I/O

2010-03-28 Thread Phillip Susi
It seems like it only happens when there are plenty of dirty buffers that need flushed, then it hangs waiting for the sync to complete, even though that is done after maybe 30 seconds. If you wait a while for the system to settle, or sync before umount, then it is fine. It also may be related to

[Bug 543617] Re: very slow filesystem I/O

2010-03-28 Thread Phillip Susi
Seeing the same thing. Also using lvm, and call trace of umount while in D state looks the same. -- very slow filesystem I/O https://bugs.launchpad.net/bugs/543617 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing

[Bug 543617] Re: very slow filesystem I/O

2010-03-23 Thread Kees Cook
This is reproducible after a reboot (9 minutes to umount). I'm seeing the dmesg warnings too. -- very slow filesystem I/O https://bugs.launchpad.net/bugs/543617 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing lis

[Bug 543617] Re: very slow filesystem I/O

2010-03-23 Thread Kees Cook
** Also affects: linux (Ubuntu Lucid) Importance: Undecided Status: Confirmed ** Changed in: linux (Ubuntu Lucid) Milestone: None => ubuntu-10.04-beta-2 -- very slow filesystem I/O https://bugs.launchpad.net/bugs/543617 You received this bug notification because you are a member of

[Bug 543617] Re: very slow filesystem I/O

2010-03-23 Thread Kees Cook
This is getting worse the longer my system runs. Going to reboot... $ time sync real103m5.920s -- very slow filesystem I/O https://bugs.launchpad.net/bugs/543617 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs maili

[Bug 543617] Re: very slow filesystem I/O

2010-03-23 Thread Jamie Strandboge
Also, it took more than 22 minutes to unmount here: $ time umount /mnt/test real22m47.719s user0m0.000s sys 0m0.180s -- very slow filesystem I/O https://bugs.launchpad.net/bugs/543617 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to

[Bug 543617] Re: very slow filesystem I/O

2010-03-23 Thread Jamie Strandboge
The reproducer works here. I use ext4 on MD (raid1) for '/' with /tmp as part of '/'. Additionally, I get the following trace in dmesg when trying to umount: [214920.810112] INFO: task umount:16606 blocked for more than 120 seconds. [214920.810118] "echo 0 > /proc/sys/kernel/hung_task_timeout_sec

[Bug 543617] Re: very slow filesystem I/O

2010-03-23 Thread Jamie Strandboge
** Description changed: I'm seeing huge latency sometimes when unmounting ext4 partitions. Steps that triggered this most recently: sudo -s cd /tmp dd if=/dev/zero of=test.ext4 bs=1 count=1 seek=1G mkfs.ext4 -F test.ext4 mkdir -p /mnt/test - mount -o loop text.ext4 /mnt/test + mou

[Bug 543617] Re: very slow filesystem I/O

2010-03-22 Thread Kees Cook
** Tags added: lucid regression-potential ** Tags removed: kernel-series-unknown -- very slow filesystem I/O https://bugs.launchpad.net/bugs/543617 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@

[Bug 543617] Re: very slow filesystem I/O

2010-03-22 Thread Jeremy Foshee
** Tags added: kernel-series-unknown -- very slow filesystem I/O https://bugs.launchpad.net/bugs/543617 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/ma

[Bug 543617] Re: very slow filesystem I/O

2010-03-21 Thread Kees Cook
Additional note: my /tmp is on ext4, part of my root partition. (Which on on an LV, on MD) -- very slow filesystem I/O https://bugs.launchpad.net/bugs/543617 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list u

[Bug 543617] Re: very slow filesystem I/O

2010-03-21 Thread Kees Cook
** Description changed: I'm seeing huge latency sometimes when unmounting ext4 partitions. Steps that triggered this most recently: sudo -s cd /tmp dd if=/dev/zero of=test.ext4 bs=1 count=1 seek=1G - mkfs.ext4 test.ext4 - mkdir /mnt/test + mkfs.ext4 -F test.ext4 + mkdir -p /mnt/test