[gentoo-amd64] Re: Emerge and Tmpfs
"Arttu V." posted fecdbac60907080910p1ac75f11pa686a0bb36dad...@mail.gmail.com, excerpted below, on Wed, 08 Jul 2009 19:10:12 +0300: > Or at least that's what it did to me last week when I was experimenting > something utterly stupid with kernel memory settings on a box with 2GB > physical RAM, 8 GB swap, /var/tmp/portage mounted on tmpfs with > size=9000M, and trying to emerge openoffice -- while only having > compiled a kernel which could handle a maximum of 4GB of memory ... :) You had me laughing, but in all seriousness, that's a good test of conditions many of us aren't likely to see very often. So at least we know the failure mode, now, and know it doesn't fail the entire system, which is more than at least I knew before. =:^) -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
Re: [gentoo-amd64] Re: Emerge and Tmpfs
Peter Humphrey wrote: On Wednesday 08 July 2009 14:51:04 Frank Peters wrote: Or is the system designed to extend the tmpfs through swapping? Sort of. If the tmpfs becomes full, part of it that isn't needed at the moment is swapped to disk, exactly as if it had been program space. This is not completely accurate. The size limit you set on the tmpfs is the size of the filesystem. df will show that many bytes free, and if you write one byte more you'll get an error that the filesystem is full. Completely independent from this, the kernel will swap memory out to disk when needed. If I have 300GB of RAM, and a 1GB tmpfs, and 5GB of resident programs, most likely nothing will be swapped up until the time the tmpfs is full. At that point the tmpfs will report that it is full, and still nothing will have swapped (most likely - the kernel can still swap stuff out in favor of cache even if the RAM is 99.999% unutilized). If I have 2GB of RAM and 10GB of tmpfs, then the system is guaranteed to swap long before that tmpfs fills up. In general, the guideline is to set your tmpfs up to be big enough to handle a large emerge (such as openoffice/firefox) - at least a few GB. Be sure to have enough swap space to handle the size of your full tmpfs + anything you're likely to be running - amount of physical RAM. If you run out of swap before you run out of tmpfs space, then the next write to the tmpfs is going to trigger the OOM killer. I'm not sure how tmpfs is prioritized by the OOM killer, but suffice it to say you're either going to lose tmpfs data or running processes. If you can't make your swap big enough, then you could set a smaller tmpfs and then just unmount it before emerging large programs. Note that more than just portage ends up in /var/tmp - I'm not sure what kde does if its cache disappears mid-session. Again, unless you're running in unusual circumstances I don't think you need to worry about the number of tmpfs inodes.
Re: [gentoo-amd64] Re: Emerge and Tmpfs
On 7/8/09, Frank Peters wrote: > However, I am not clear on how tmpfs will fail. If the tmpfs mount > becomes filled or exceeds the file limit, since it is essentially just > another disk partition shouldn't it produce a "No more space left on device" > error? Yes it does. Or at least that's what it did to me last week when I was experimenting something utterly stupid with kernel memory settings on a box with 2GB physical RAM, 8 GB swap, /var/tmp/portage mounted on tmpfs with size=9000M, and trying to emerge openoffice -- while only having compiled a kernel which could handle a maximum of 4GB of memory ... :) -- Arttu V.
[gentoo-amd64] Re: Emerge and Tmpfs
Paul Hartman posted 58965d8a0907080737s2c4a71dah3c9b54e55f02d...@mail.gmail.com, excerpted below, on Wed, 08 Jul 2009 09:37:01 -0500: > In other words if you have a 2 gig tmpfs it's not going to eat up 2gb of > your RAM unless you have actually put 2gb of files onto it. Good point. I neglected to mention that in my reply, taking it as a given. However, if someone didn't understand that tmpfs could swap, it's unlikely they'd understand this bit either, so it's certainly worth mentioning. =:^) -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
Re: [gentoo-amd64] Re: Emerge and Tmpfs
On Wednesday 08 July 2009 14:51:04 Frank Peters wrote: > If the tmpfs mount becomes filled or exceeds the file limit, since it is > essentially just another disk partition shouldn't it produce a "No more > space left on device" error? No, it isn't a disk partition. It's a file system in RAM, with no reference to the disk at all, which is why it's so much faster than file systems on disk. > Or is the system designed to extend the tmpfs through swapping? Sort of. If the tmpfs becomes full, part of it that isn't needed at the moment is swapped to disk, exactly as if it had been program space. -- Rgds Peter
[gentoo-amd64] Re: Emerge and Tmpfs
Frank Peters posted 20090708095104.39a8a53e.frank.pet...@comcast.net, excerpted below, on Wed, 08 Jul 2009 09:51:04 -0400: > However, I am not clear on how tmpfs will fail. If the tmpfs mount > becomes filled or exceeds the file limit, since it is essentially just > another disk partition shouldn't it produce a "No more space left on > device" error? Or is the system designed to extend the tmpfs through > swapping? The latter option doesn't seem right, but everyone still > refers to such behavior. tmpfs is specifically /not/ pinned in real-memory, and only gets the "no space left on device" error when it maxes out. (Note that the tmpfs default max side if not specified is half of real memory, the default max inodes are half of the memory page-count, which @ 4KB pages, means 256/MB or 262,144/gig.) So yes, tmpfs swaps just fine, because it's not pinned memory and running out of real memory is not no-space-on-device, as long as there's swap. For purposes of memory tracking, BTW, (unswapped) tmpfs is reported as cache, tho in some ways it really behaves more like (dirty) app memory in that it can't be simply dropped as the disk-read-cache can (since for cache the data is already on disk and can simply be read back from there if it's needed), but must be actually written out to swap, like dirty app memory. But regardless, pointing portage's working dir at tmpfs is still generally faster than having it on a normal disk filesystem, because the worst-case for tmpfs, is the normal-case for a normal disk filesystem -- that it actually has to write the stuff to disk for a bit, before reading it back in and ultimately deleting it. Of course, the same basic effect could be had by pointing portage at an ext4fs, configured with huge (several gig) max write buffers and long (like an hour) sync timeouts. Except even then ext4 wouldn't be quite as good, because there's shortcuts the kernel can take with the explicitly temporary data of a tmpfs that it can't take with ext4. That's the theory, anyway. It'd be interesting to benchmark it. Oh, and if the system /does/ crash in the middle of something, with tmpfs it's all "magically" cleaned up, while with ext4, whatever files had been written to disk would have to be cleaned up after the reboot. So even if the benchmarks don't quite live up to the theory, avoiding that cleanup hassle is very likely worth it. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
Re: [gentoo-amd64] Re: Emerge and Tmpfs
On Wed, Jul 8, 2009 at 8:51 AM, Frank Peters wrote: > However, I am not clear on how tmpfs will fail. If the tmpfs mount > becomes filled or exceeds the file limit, since it is essentially just > another disk partition shouldn't it produce a "No more space left on device" > error? Or is the system designed to extend the tmpfs through swapping? > The latter option doesn't seem right, but everyone still refers to such > behavior. I'm certainly not an expert but my understanding is that it will not exceed the size given (or defaulted - half of system RAM?) - but the part about resizing and using swap I believe has more to do with how it uses the available RAM in the system. In other words if you have a 2 gig tmpfs it's not going to eat up 2gb of your RAM unless you have actually put 2gb of files onto it. So it dynamically uses RAM when necessary. If the files in the tmpfs exceed the available RAM on the system, then it'll do what any other out-of-memory situation would do: swap.
Re: [gentoo-amd64] Re: Emerge and Tmpfs
On Wed, 08 Jul 2009 08:55:43 -0400 Richard Freeman wrote: > > I occassionally get swaps, but that is no big deal. In the worst case > swapping is no worse than not using tmpfs at all, and in the typical or > best cases it is far better. It really is a no-lose scenario. If > something leaves junk lying around in /var/tmp then it just gets swapped > out and never gets swapped back in - again no worse than if it were > written to disk in the first place. Just make sure you have plenty of > swap space if your physical RAM is limited. > My swap space is set for 8G, and it is also located at about the center of the disk platter, which supposedly will give the fastest I/O performance. However, I am not clear on how tmpfs will fail. If the tmpfs mount becomes filled or exceeds the file limit, since it is essentially just another disk partition shouldn't it produce a "No more space left on device" error? Or is the system designed to extend the tmpfs through swapping? The latter option doesn't seem right, but everyone still refers to such behavior. Frank Peters
Re: [gentoo-amd64] Re: Emerge and Tmpfs
Duncan wrote: Personally, I'd just go with the default nr_inodes. People with 2 gig or less of real RAM may need to worry about it, especially if they do a lot of parallel makes (tho with 2 gig I'd crimp on parallel makes way more than I do, too, so may not have to, but as I've said before, I'm not sure of the affect at a gig, someone would have to test and see), but with 4 gig and a million inodes by default, I think you should be fine. I only have 2GB of RAM and have no problem with the defaults. I run two parallel merges - I don't think anybody with a computer this old would run more than that anyway. I occassionally get swaps, but that is no big deal. In the worst case swapping is no worse than not using tmpfs at all, and in the typical or best cases it is far better. It really is a no-lose scenario. If something leaves junk lying around in /var/tmp then it just gets swapped out and never gets swapped back in - again no worse than if it were written to disk in the first place. Just make sure you have plenty of swap space if your physical RAM is limited. smime.p7s Description: S/MIME Cryptographic Signature
[gentoo-amd64] Re: Emerge and Tmpfs
Frank Peters posted 20090707103623.17cd03be.frank.pet...@comcast.net, excerpted below, on Tue, 07 Jul 2009 10:36:23 -0400: > After reading through the thread on this list about the benefits of > mounting PORTAGE_TMPDIR as tmpfs, I decided that I will be doing this > mount from now on. My total RAM is 4G, but since I only "emerge world" > about once per week, I can easily set aside temporarily 2G or more for > this purpose. > > However, after doing some more reading, I find that some people > recommend mounting tmpfs using the option nr_inodes=1M, while others > just accept the default value for nr_inodes. > > To me, setting maximum inodes to 1M seems a bit high. Can emerge > actually generate that many temporary files? Would it be safer and also > avoid swapping to set nr_inodes=1M or is the default good enough?. FWIW, I've never had the slightest issue with the defaults, and that's running up to 7+ emerges at once, the way portage's auto-scheduler is handling it here, and about that handling it myself based on --tree, before that. However, the mount (1) manpage says the default nr_inodes is half the number of physical RAM pages. At a 4k pgsize, that's 1M for 4 gig anyway, and 2M for the 8 gig I'm running, so I /shouldn't/ be running out and should be able to lower it. Personally, I'd just go with the default nr_inodes. People with 2 gig or less of real RAM may need to worry about it, especially if they do a lot of parallel makes (tho with 2 gig I'd crimp on parallel makes way more than I do, too, so may not have to, but as I've said before, I'm not sure of the affect at a gig, someone would have to test and see), but with 4 gig and a million inodes by default, I think you should be fine. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
