Re: Huge instability and insanely large memory footprint in 9.04
On Fri, Sep 25, 2009 at 8:19 AM, Evan eapa...@gmail.com wrote: On Fri, Sep 25, 2009 at 7:40 AM, Marius Gedminas mar...@pov.lt wrote: On Fri, Sep 18, 2009 at 04:46:21AM -0700, J. Lennard wrote: First I'm sorry to write what may appear as a rant, but I hope it is not considered so. I'm writing to express huge instability problems in Ubuntu 9.04. I migrated from ubuntu 8.04 (386) to ubuntu 9.04 (amd64) using a simple clean install. ... I don't really know where to start. During past month, my machine constantly went to trashing mode where the hard-disk light is constantly on and I can't access anything or even swtich to linux console for several *minutes*. This has occured more than four times although all I usually run is a pdf viewer, an mp3 player, emacs, and firefox with simple html pages (not even gmail, flash, etc). Do you have any swap space? I've got an Asus EeePC 900 with 1 GB of RAM. Twice now I've experienced the same thing: constant disk I/O, huge latencies for any desktop task (switching windows, launching terminals). I suspect a bug in the Linux VM subsystem, since *I was not running out of memory*. Usually about 40-50% of my RAM is in disk cache---I keep track of memory usage via a GNOME panel applet. On those occasions cache size was shrinking, completely free memory was increasing, all application pages were being constantly swapped out and back in causing constant disk I/O (which is painful on an SSD). The fix was to create a temporary swap file in /tmp: sudo -s dd if=/dev/zero of=/tmp/swapfile bs=1M count=1024 mkswap /tmp/swapfile swapon /tmp/swapfile Instantly the system became responsible again, after using ~400 *kilobytes* of swap. I can only assume that having 0 swap space confused the VM somehow and kicked it into a full-on panic let's free all the ram we can mode. This swap file disappears after a reboot, and so far the problem hasn't recurred, so I didn't bother setting up a permanent swap partition (I'm afraid to destroy my SSD too quickly with constant writes---this already happened once thanks to ext3's journal). I have had this occur several times to me on a quad-core with 3GB ram (and 6GB swap), but only ever when resuming. If I suspend with all free memory used by cache (according to the Gnome panel applet), then occasionally on resume it spends ~5minutes unresponsive with massive disk IO before coming up. When it does come back everything is fine, except that all of that space which was cache is now completely empty. I set my vm.swappiness value to 0 (because I so rarely need it), so while I'm definitely not running out of swap space, I can see how swappiness=0 might cause the same sort of effect. I haven't found anything odd in the logs so I never filed a bug, but if it's happening to other people in other situations this bears investigating. I will open a bug on this as soon as I have time. This bug finally reoccurred, and I caught something in the log files this time. The bug is at https://bugs.launchpad.net/ubuntu/+source/pm-utils/+bug/438889 Mine appears to be in pm-utils, so I doubt it's the same as yours Marius, but it may be a more fundamental problem with the kernel's memory management. It's worth taking a look at in any case. Evan -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Huge instability and insanely large memory footprint in 9.04
On Fri, Sep 18, 2009 at 04:46:21AM -0700, J. Lennard wrote: First I'm sorry to write what may appear as a rant, but I hope it is not considered so. I'm writing to express huge instability problems in Ubuntu 9.04. I migrated from ubuntu 8.04 (386) to ubuntu 9.04 (amd64) using a simple clean install. ... I don't really know where to start. During past month, my machine constantly went to trashing mode where the hard-disk light is constantly on and I can't access anything or even swtich to linux console for several *minutes*. This has occured more than four times although all I usually run is a pdf viewer, an mp3 player, emacs, and firefox with simple html pages (not even gmail, flash, etc). Do you have any swap space? I've got an Asus EeePC 900 with 1 GB of RAM. Twice now I've experienced the same thing: constant disk I/O, huge latencies for any desktop task (switching windows, launching terminals). I suspect a bug in the Linux VM subsystem, since *I was not running out of memory*. Usually about 40-50% of my RAM is in disk cache---I keep track of memory usage via a GNOME panel applet. On those occasions cache size was shrinking, completely free memory was increasing, all application pages were being constantly swapped out and back in causing constant disk I/O (which is painful on an SSD). The fix was to create a temporary swap file in /tmp: sudo -s dd if=/dev/zero of=/tmp/swapfile bs=1M count=1024 mkswap /tmp/swapfile swapon /tmp/swapfile Instantly the system became responsible again, after using ~400 *kilobytes* of swap. I can only assume that having 0 swap space confused the VM somehow and kicked it into a full-on panic let's free all the ram we can mode. This swap file disappears after a reboot, and so far the problem hasn't recurred, so I didn't bother setting up a permanent swap partition (I'm afraid to destroy my SSD too quickly with constant writes---this already happened once thanks to ext3's journal). I did have a new weird VM problem yesterday: for some reason, despite having 250 megs totally free, the kernel went into OOM-killer mode and killed rhyhmbox and chromium-browser, then, thankfully, stopped. It saddens me that all I have are anecdotes rather than informative bug reports. :( I can't really understand how this can happen. Several times, and after a day or two of use, firefox, with *one* simple html tab open took 340+ MBs; that's insane. Evince took 120MB while only a single pdf file was open. Even Xorg was taking RAM around a hundread megabyte.[1] [1] Resident set memory, not virtual memory. RSS includes shared pages too, giving a skewed picture. Xorg's memory maps include mmaped device address space, giving an even-more skewed picture. The final result? a machine constantly thrashing and basically unusable. I ran this on a core2 laptop with a full 1GB of ram. How come the experience is SO bad in supposedly a *stable* distro? Software is hard. The second problem is that the GUI is *really* slow, and I use *zero* visual effects. Switching between workspaces is very sluggish where I see parts of firefox in my audacious window for about half a second while switching between workspaces. Switching between applications (alt+tab) is not smooth at all. *Using* desktop effects will give you smoother desktop/application switching. That's the primary reason I enable them, it results in the whole desktop being double-buffered and therefore reduces redrawing artifacts such as these. What kind of graphics do you have? My puny little Asus with its Intel 945 was very sluggish until I disabled vertical synchronisation. The third thing, which is disastrous and never occurred to me before using Ubuntu (and I've been using Ubuntu since Ubuntu 5) was constant and *systematic* audio skipping while doing *any* task. Heck, I swear simple switching between workspaces sometimes lead to several audio skipping. If this happens during the thrashing, then it's understandable; if not then *ouch*. I re-installed a Ubuntu 8.04 amd64 and my sanity is back. You said you were running 8.04 386 before? amd64 versions generally need more RAM, and their primary advantage starts showing up on machines with at least 4 GB of RAM. I'm sorry, this is my worst Linux experience ever, but thankfully Ubuntu 8.04 works beautifully here that I'm thankful after all. It's really sad my favourite OS reached this level of instability and bloat, but hey, I at least have 8.04 till 2011, which I couldn't ask for more. Please don't let Ubuntu go to this sad path. Thank you. Marius Gedminas -- Suppose you went back to Ada Lovelace and asked her the difference between a script and a program. She'd probably look at you funny, then say something like: Well, a script is what you give the actors, but a program is what you give the audience. -- Larry Wall signature.asc Description: Digital signature -- Ubuntu-devel-discuss mailing list
Re: Huge instability and insanely large memory footprint in 9.04
On Fri, Sep 25, 2009 at 7:40 AM, Marius Gedminas mar...@pov.lt wrote: On Fri, Sep 18, 2009 at 04:46:21AM -0700, J. Lennard wrote: First I'm sorry to write what may appear as a rant, but I hope it is not considered so. I'm writing to express huge instability problems in Ubuntu 9.04. I migrated from ubuntu 8.04 (386) to ubuntu 9.04 (amd64) using a simple clean install. ... I don't really know where to start. During past month, my machine constantly went to trashing mode where the hard-disk light is constantly on and I can't access anything or even swtich to linux console for several *minutes*. This has occured more than four times although all I usually run is a pdf viewer, an mp3 player, emacs, and firefox with simple html pages (not even gmail, flash, etc). Do you have any swap space? I've got an Asus EeePC 900 with 1 GB of RAM. Twice now I've experienced the same thing: constant disk I/O, huge latencies for any desktop task (switching windows, launching terminals). I suspect a bug in the Linux VM subsystem, since *I was not running out of memory*. Usually about 40-50% of my RAM is in disk cache---I keep track of memory usage via a GNOME panel applet. On those occasions cache size was shrinking, completely free memory was increasing, all application pages were being constantly swapped out and back in causing constant disk I/O (which is painful on an SSD). The fix was to create a temporary swap file in /tmp: sudo -s dd if=/dev/zero of=/tmp/swapfile bs=1M count=1024 mkswap /tmp/swapfile swapon /tmp/swapfile Instantly the system became responsible again, after using ~400 *kilobytes* of swap. I can only assume that having 0 swap space confused the VM somehow and kicked it into a full-on panic let's free all the ram we can mode. This swap file disappears after a reboot, and so far the problem hasn't recurred, so I didn't bother setting up a permanent swap partition (I'm afraid to destroy my SSD too quickly with constant writes---this already happened once thanks to ext3's journal). I have had this occur several times to me on a quad-core with 3GB ram (and 6GB swap), but only ever when resuming. If I suspend with all free memory used by cache (according to the Gnome panel applet), then occasionally on resume it spends ~5minutes unresponsive with massive disk IO before coming up. When it does come back everything is fine, except that all of that space which was cache is now completely empty. I set my vm.swappiness value to 0 (because I so rarely need it), so while I'm definitely not running out of swap space, I can see how swappiness=0 might cause the same sort of effect. I haven't found anything odd in the logs so I never filed a bug, but if it's happening to other people in other situations this bears investigating. I will open a bug on this as soon as I have time. -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Huge instability and insanely large memory footprint in 9.04
Hello, 2009/9/18 J. Lennard lennar...@yahoo.com: I don't really know where to start. During past month, my machine constantly went to trashing mode where the hard-disk light is constantly on and I can't access anything or even swtich to linux console for several *minutes*. As somebody else suggested, have you tried to remove trackerd? It trashed my machine, despite having 4 GiB of RAM with 64 bits Ubuntu. My personal rant: why this trackerd included by default? I always had a bad experience with it, i.e. the trashing you observed. Best regards, david -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Huge instability and insanely large memory footprint in 9.04
On Fri, Sep 18, 2009 at 7:46 AM, J. Lennard lennar...@yahoo.com wrote: Hi list, First I'm sorry to write what may appear as a rant, but I hope it is not considered so. I'm writing to express huge instability problems in Ubuntu 9.04. I migrated from ubuntu 8.04 (386) to ubuntu 9.04 (amd64) using a simple clean install. Mine freezes and eats a massive load of memory on x86-64. I use 'elevator=as' on the command line; something other than the I/O scheduler is broken, and CFQ triggers it MUCH more frequently (hours, sometimes minutes instead of DAYS) than anticipatory. Some freezes come back; some don't; the ones that come back can be triggered by hitting NumLock (sends a specific interrupt? Sysrq sends INT6, numlock should at most trigger a software driver internal state change...). Killing everything (MagicSys E) causes the disk to suddenly thrash, as whatever blocked path suddenly becomes free. This all indicates to me that somewhere exists a racey deadlock in the I/O system that can affect two or more processes, and that the schedulers each fault through that path. It's a kernel issue, I've raised it but let's see if 9.10 fixes the problem. I haven't been watching the kernel bug fix list. -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Huge instability and insanely large memory footprint in 9.04
Conrad Knauer wrote: I'm going to guess, without seeing his machine, that it's something with Firefox... it could be a malfunctioning extension, it could be some script on an otherwise normal page... I would try backing up my ~/.mozilla folder and seeing if running FF fresh solves the problem. At the risk of offending most by stating the obvious, note that firefox -- especially when extensions are loaded -- has a continuously increasing memory footprint; even when you're not doing anything other than leaving it open on a few tabs. And sites using javascript (especially those with lots of ever changing ads) are notorious for memory leaks. On my MS Windows machine, after a couple of weeks of having firefox continuously open, the memory use goes up to about 1.2GB (the hint is that the machine ceases to repeat characters in a timely fashion). Linux isn't quite as bad, but not by much -- below is the output from top: PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 8209 pgoetz20 0 1877m 895m 45m S 10 27.3 576:27.11 firefox 7668 pgoetz20 0 160m 17m 12m S1 0.5 42:56.50 pulseaudio pulseaudio? I'm not even using any audio applications and hardly ever do! The linux kernel is a lean, mean, fighting machine. X windows? Not so much. KDE and gnome? Don't ask. Firefox and friends? See above. The bottom line is if you're not running X, an Intel 486 with 256KB of RAM is probably a perfectly adequate platform. -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Huge instability and insanely large memory footprint in 9.04
On Fri, Sep 25, 2009 at 11:45 AM, Patrick Goetz pgo...@mail.utexas.edu wrote: 7668 pgoetz 20 0 160m 17m 12m S 1 0.5 42:56.50 pulseaudio Note that you can disable PA's mempool implementation. We also cache /usr/share/sounds/ubuntu/stereo/* . Of course, from your top (not really a good indicator of memory use anyhow; use exmap instead) output, that's not really an issue. There are _lots_ of ways to tweak PA for those people who love knobs. -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Huge instability and insanely large memory footprint in 9.04
2009/9/18 J. Lennard lennar...@yahoo.com Hi list, First I'm sorry to write what may appear as a rant, but I hope it is not considered so. I'm writing to express huge instability problems in Ubuntu 9.04. I migrated from ubuntu 8.04 (386) to ubuntu 9.04 (amd64) using a simple clean install. While the installation and boot process was blazingly and beautifully fast, everything else was a disaster, and I'm not exaggerating. First, I'd like to mention that I have *zero* proprietary drivers (nvidia blobs, etc) or extensions (flash) installed. I don't really know where to start. During past month, my machine constantly went to trashing mode where the hard-disk light is constantly on and I can't access anything or even swtich to linux console for several *minutes*. This has occured more than four times although all I usually run is a pdf viewer, an mp3 player, emacs, and firefox with simple html pages (not even gmail, flash, etc). I can't really understand how this can happen. Several times, and after a day or two of use, firefox, with *one* simple html tab open took 340+ MBs; that's insane. Evince took 120MB while only a single pdf file was open. Even Xorg was taking RAM around a hundread megabyte.[1] The final result? a machine constantly thrashing and basically unusable. I ran this on a core2 laptop with a full 1GB of ram. How come the experience is SO bad in supposedly a *stable* distro? The second problem is that the GUI is *really* slow, and I use *zero* visual effects. Switching between workspaces is very sluggish where I see parts of firefox in my audacious window for about half a second while switching between workspaces. Switching between applications (alt+tab) is not smooth at all. You don't have enough RAM. 1 GB is pretty low by today's standards. You may be happier with xubuntu on a lower spec machine, but as you've got a fast CPU then you should get more RAM as it is a clear bottleneck. You may be happier without compiz at all - but your system will run a lot better if you sort your RAM problem out. Caroline -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Huge instability and insanely large memory footprint in 9.04
On Fri, Sep 18, 2009 at 7:46 AM, J. Lennard lennar...@yahoo.com wrote: Hi list, First I'm sorry to write what may appear as a rant, but I hope it is not considered so. I'm writing to express huge instability problems in Ubuntu 9.04. I migrated from ubuntu 8.04 (386) to ubuntu 9.04 (amd64) using a simple clean install. It's not a rant as long as you detail your problems and don't yell Ubuntu Sucks every sentence just because :) First, I'd like to mention that I have *zero* proprietary drivers (nvidia blobs, etc) or extensions (flash) installed. Good to know. I don't really know where to start. During past month, my machine constantly went to trashing mode where the hard-disk light is constantly on and I can't access anything or even swtich to linux console for several *minutes*. This has occured more than four times although all I usually run is a pdf viewer, an mp3 player, emacs, and firefox with simple html pages (not even gmail, flash, etc). It sounds like you've run out of ram and started swapping, but with only those apps open there's no way you should be using 1 GB of memory. I can't really understand how this can happen. Several times, and after a day or two of use, firefox, with *one* simple html tab open took 340+ MBs; that's insane. Evince took 120MB while only a single pdf file was open. Even Xorg was taking RAM around a hundread megabyte.[1] Firefox has some memory issues, however I believe they were actually worse in the 2.0 version that shipped with Ubuntu 8.04. The good news is that 9.10 will ship with Firefox 3.5, which has resolved 99% of these issues. Ram usage for X really depends on the driver. On my system, X takes ~120MB on boot, but never grows significantly beyond that (proprietary ATI driver). Without the proprietary driver it starts much lower, but climbs slowly over time. The intel drivers in 9.04 are known to be generally terrible for various reasons (also fixed in 9.10), so if you have an intel card then that's the likely culprit. I can't speak for evince. The second problem is that the GUI is *really* slow, and I use *zero* visual effects. Switching between workspaces is very sluggish where I see parts of firefox in my audacious window for about half a second while switching between workspaces. Switching between applications (alt+tab) is not smooth at all. Again, if you have an intel card, I'd be tempted to blame it on the graphics drivers. I wasn't closely following that part of the 9.04 cycle, so I can't speak for why we ended up shipping a somewhat broken driver in a supposedly stable release, but if someone with a bit more knowledge of what happened wants to step in and explain, please do. The third thing, which is disastrous and never occurred to me before using Ubuntu (and I've been using Ubuntu since Ubuntu 5) was constant and *systematic* audio skipping while doing *any* task. Heck, I swear simple switching between workspaces sometimes lead to several audio skipping. That's weird. Pulseaudio has been around since 8.04, so this is definitely a regression. I guess all you can do is file a bug (include your audio card model) and hope it's fixed for Karmic. I'm sorry, this is my worst Linux experience ever, but thankfully Ubuntu 8.04 works beautifully here that I'm thankful after all. It's really sad my favourite OS reached this level of instability and bloat, but hey, I at least have 8.04 till 2011, which I couldn't ask for more. This is not a typical experience: 9.04 is the first Ubuntu release *ever *which I am staying with for more than one cycle, simply because it has been so stable for me :) I would say you simply had really bad luck with your combination of hardware. It doesn't excuse the fact that regressions shouldn't happen, but please don't assume that the distro as a whole has reached that level of instability. I hope you have better luck with 9.10 or 10.04. Evan -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Huge instability and insanely large memory footprint in 9.04
Hey Caroline, On Thu, 2009-09-24 at 18:58 +0100, Caroline Ford wrote: You don't have enough RAM. 1 GB is pretty low by today's standards. You may be happier with xubuntu on a lower spec machine, but as you've got a fast CPU then you should get more RAM as it is a clear bottleneck. I don't think I agree, I do refurbished computers which are normally P4 CPUs with 512MB of RAM. According to tests an Ubuntu machine will work reasonably well on 390MB of RAM or more, but less and it won't work very well at all. XUbuntu should be used on PIII machines with 128-256MB of RAM and we normally have enough 128MB SDRAM sticks to upgrade them. But anything less and it's scrap metal. At first I took offence at your idea that 1GB is low by today's standards, but then I took a moment and thought that perhaps it's a degree of ignorance on your part about who Ubuntu is being developed for. It's not being developed (as Windows 7 is) for the latest and greatest computers being sold right this second. It's being developed for all computers ranging from the old P4s mentioned to the sub-powered Atom and Arm netbooks all the way up to the monster 16GB RAM Gaming rigs for playing Savage2. It it's not working on a computer with 1GB of RAM, then we need to fix it. This is clearly not a case of user error. Best Regards, Martin Owens -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Huge instability and insanely large memory footprint in 9.04
Sorry to followup a followup, but: On Thu, 24 Sep 2009, George Farris wrote: On Thu, 2009-09-24 at 15:08 -0400, Martin Owens wrote: XUbuntu should be used on PIII machines with 128-256MB of RAM and we normally have enough 128MB SDRAM sticks to upgrade them. But anything less and it's scrap metal. I'm running 9.04 on and old P3 500 with 256MB of RAM: total used free sharedbuffers cached Mem:250672 208192 42480 0 21280 132880 -/+ buffers/cache: 54032 196640 Swap: 1020088 11841018904 It's not doing much (a home office gateway running DHCP, Bind, Postfix, PPPoE, etc), but has no memory problems at all (It's not of course running X). I recently upgraded it from 192MB. It used to run Gentoo (builds took a while). -Ronan -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Huge instability and insanely large memory footprint in 9.04
On Thu, Sep 24, 2009 at 12:43 PM, Onkar Shinde onkarshi...@gmail.com wrote: You don't have enough RAM. 1 GB is pretty low by today's standards. You may be happier with xubuntu on a lower spec machine, but as you've got a fast CPU then you should get more RAM as it is a clear bottleneck. 1 GB may be less by today's standards but not everyone is using a machine bought today. Most are running a machine bought before 6 months or an year or even before that. And actually most netbooks come with 1 GB standard right now. I've refurbished old machines and only 512 MB RAM will run Ubuntu quite happily. And 1 GB is no way insufficient for the applications that user is running. There is something else wrong on his machine. I'm going to guess, without seeing his machine, that it's something with Firefox... it could be a malfunctioning extension, it could be some script on an otherwise normal page... I would try backing up my ~/.mozilla folder and seeing if running FF fresh solves the problem. CK -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
