Re: [gentoo-amd64] Re: Emerge and Tmpfs
Duncan wrote: Personally, I'd just go with the default nr_inodes. People with 2 gig or less of real RAM may need to worry about it, especially if they do a lot of parallel makes (tho with 2 gig I'd crimp on parallel makes way more than I do, too, so may not have to, but as I've said before, I'm not sure of the affect at a gig, someone would have to test and see), but with 4 gig and a million inodes by default, I think you should be fine. I only have 2GB of RAM and have no problem with the defaults. I run two parallel merges - I don't think anybody with a computer this old would run more than that anyway. I occassionally get swaps, but that is no big deal. In the worst case swapping is no worse than not using tmpfs at all, and in the typical or best cases it is far better. It really is a no-lose scenario. If something leaves junk lying around in /var/tmp then it just gets swapped out and never gets swapped back in - again no worse than if it were written to disk in the first place. Just make sure you have plenty of swap space if your physical RAM is limited. smime.p7s Description: S/MIME Cryptographic Signature
Re: [gentoo-amd64] Re: Emerge and Tmpfs
On Wed, 08 Jul 2009 08:55:43 -0400 Richard Freeman r...@thefreemanclan.net wrote: I occassionally get swaps, but that is no big deal. In the worst case swapping is no worse than not using tmpfs at all, and in the typical or best cases it is far better. It really is a no-lose scenario. If something leaves junk lying around in /var/tmp then it just gets swapped out and never gets swapped back in - again no worse than if it were written to disk in the first place. Just make sure you have plenty of swap space if your physical RAM is limited. My swap space is set for 8G, and it is also located at about the center of the disk platter, which supposedly will give the fastest I/O performance. However, I am not clear on how tmpfs will fail. If the tmpfs mount becomes filled or exceeds the file limit, since it is essentially just another disk partition shouldn't it produce a No more space left on device error? Or is the system designed to extend the tmpfs through swapping? The latter option doesn't seem right, but everyone still refers to such behavior. Frank Peters
Re: [gentoo-amd64] Re: Emerge and Tmpfs
On Wed, Jul 8, 2009 at 8:51 AM, Frank Petersfrank.pet...@comcast.net wrote: However, I am not clear on how tmpfs will fail. If the tmpfs mount becomes filled or exceeds the file limit, since it is essentially just another disk partition shouldn't it produce a No more space left on device error? Or is the system designed to extend the tmpfs through swapping? The latter option doesn't seem right, but everyone still refers to such behavior. I'm certainly not an expert but my understanding is that it will not exceed the size given (or defaulted - half of system RAM?) - but the part about resizing and using swap I believe has more to do with how it uses the available RAM in the system. In other words if you have a 2 gig tmpfs it's not going to eat up 2gb of your RAM unless you have actually put 2gb of files onto it. So it dynamically uses RAM when necessary. If the files in the tmpfs exceed the available RAM on the system, then it'll do what any other out-of-memory situation would do: swap.
[gentoo-amd64] latest k3b has no menu entry
I just upgraded to 1.0.5-r5 of k3b and now I have no menu option in Multimedia. Is anyone seeing this as well? Where do I look in KDE4 to fix it? -- Interdum feror cupidine partium magnarum Europae vincendarum Mark Haney Sr. Systems Administrator ERC Broadband (828) 350-2415
[gentoo-amd64] Re: Emerge and Tmpfs
Frank Peters frank.pet...@comcast.net posted 20090708095104.39a8a53e.frank.pet...@comcast.net, excerpted below, on Wed, 08 Jul 2009 09:51:04 -0400: However, I am not clear on how tmpfs will fail. If the tmpfs mount becomes filled or exceeds the file limit, since it is essentially just another disk partition shouldn't it produce a No more space left on device error? Or is the system designed to extend the tmpfs through swapping? The latter option doesn't seem right, but everyone still refers to such behavior. tmpfs is specifically /not/ pinned in real-memory, and only gets the no space left on device error when it maxes out. (Note that the tmpfs default max side if not specified is half of real memory, the default max inodes are half of the memory page-count, which @ 4KB pages, means 256/MB or 262,144/gig.) So yes, tmpfs swaps just fine, because it's not pinned memory and running out of real memory is not no-space-on-device, as long as there's swap. For purposes of memory tracking, BTW, (unswapped) tmpfs is reported as cache, tho in some ways it really behaves more like (dirty) app memory in that it can't be simply dropped as the disk-read-cache can (since for cache the data is already on disk and can simply be read back from there if it's needed), but must be actually written out to swap, like dirty app memory. But regardless, pointing portage's working dir at tmpfs is still generally faster than having it on a normal disk filesystem, because the worst-case for tmpfs, is the normal-case for a normal disk filesystem -- that it actually has to write the stuff to disk for a bit, before reading it back in and ultimately deleting it. Of course, the same basic effect could be had by pointing portage at an ext4fs, configured with huge (several gig) max write buffers and long (like an hour) sync timeouts. Except even then ext4 wouldn't be quite as good, because there's shortcuts the kernel can take with the explicitly temporary data of a tmpfs that it can't take with ext4. That's the theory, anyway. It'd be interesting to benchmark it. Oh, and if the system /does/ crash in the middle of something, with tmpfs it's all magically cleaned up, while with ext4, whatever files had been written to disk would have to be cleaned up after the reboot. So even if the benchmarks don't quite live up to the theory, avoiding that cleanup hassle is very likely worth it. -- Duncan - List replies preferred. No HTML msgs. Every nonfree program has a lord, a master -- and if you use the program, he is your master. Richard Stallman
Re: [gentoo-amd64] Re: Emerge and Tmpfs
On Wednesday 08 July 2009 14:51:04 Frank Peters wrote: If the tmpfs mount becomes filled or exceeds the file limit, since it is essentially just another disk partition shouldn't it produce a No more space left on device error? No, it isn't a disk partition. It's a file system in RAM, with no reference to the disk at all, which is why it's so much faster than file systems on disk. Or is the system designed to extend the tmpfs through swapping? Sort of. If the tmpfs becomes full, part of it that isn't needed at the moment is swapped to disk, exactly as if it had been program space. -- Rgds Peter
[gentoo-amd64] Re: Emerge and Tmpfs
Paul Hartman paul.hartman+gen...@gmail.com posted 58965d8a0907080737s2c4a71dah3c9b54e55f02d...@mail.gmail.com, excerpted below, on Wed, 08 Jul 2009 09:37:01 -0500: In other words if you have a 2 gig tmpfs it's not going to eat up 2gb of your RAM unless you have actually put 2gb of files onto it. Good point. I neglected to mention that in my reply, taking it as a given. However, if someone didn't understand that tmpfs could swap, it's unlikely they'd understand this bit either, so it's certainly worth mentioning. =:^) -- Duncan - List replies preferred. No HTML msgs. Every nonfree program has a lord, a master -- and if you use the program, he is your master. Richard Stallman
Re: [gentoo-amd64] Re: Emerge and Tmpfs
On 7/8/09, Frank Peters frank.pet...@comcast.net wrote: However, I am not clear on how tmpfs will fail. If the tmpfs mount becomes filled or exceeds the file limit, since it is essentially just another disk partition shouldn't it produce a No more space left on device error? Yes it does. Or at least that's what it did to me last week when I was experimenting something utterly stupid with kernel memory settings on a box with 2GB physical RAM, 8 GB swap, /var/tmp/portage mounted on tmpfs with size=9000M, and trying to emerge openoffice -- while only having compiled a kernel which could handle a maximum of 4GB of memory ... :) -- Arttu V.
Re: [gentoo-amd64] Re: Emerge and Tmpfs
Peter Humphrey wrote: On Wednesday 08 July 2009 14:51:04 Frank Peters wrote: Or is the system designed to extend the tmpfs through swapping? Sort of. If the tmpfs becomes full, part of it that isn't needed at the moment is swapped to disk, exactly as if it had been program space. This is not completely accurate. The size limit you set on the tmpfs is the size of the filesystem. df will show that many bytes free, and if you write one byte more you'll get an error that the filesystem is full. Completely independent from this, the kernel will swap memory out to disk when needed. If I have 300GB of RAM, and a 1GB tmpfs, and 5GB of resident programs, most likely nothing will be swapped up until the time the tmpfs is full. At that point the tmpfs will report that it is full, and still nothing will have swapped (most likely - the kernel can still swap stuff out in favor of cache even if the RAM is 99.999% unutilized). If I have 2GB of RAM and 10GB of tmpfs, then the system is guaranteed to swap long before that tmpfs fills up. In general, the guideline is to set your tmpfs up to be big enough to handle a large emerge (such as openoffice/firefox) - at least a few GB. Be sure to have enough swap space to handle the size of your full tmpfs + anything you're likely to be running - amount of physical RAM. If you run out of swap before you run out of tmpfs space, then the next write to the tmpfs is going to trigger the OOM killer. I'm not sure how tmpfs is prioritized by the OOM killer, but suffice it to say you're either going to lose tmpfs data or running processes. If you can't make your swap big enough, then you could set a smaller tmpfs and then just unmount it before emerging large programs. Note that more than just portage ends up in /var/tmp - I'm not sure what kde does if its cache disappears mid-session. Again, unless you're running in unusual circumstances I don't think you need to worry about the number of tmpfs inodes.
[gentoo-amd64] Recover files on ext4 partition
Hi, does anyone of you have some experience in recovering a wiped out ext4 partition? I accidentially ran mkfs.ext4 on a ext4 partition for which I do not have a backup (I wanted to format sdg but instead I accidentially took sde :( ). Is there a way to recover the data in this case? Rgds Bernhard
Re: [gentoo-amd64] Recover files on ext4 partition
On Wed, Jul 8, 2009 at 3:16 PM, Bernhard Auzingernordpolcam...@gmail.com wrote: Hi, does anyone of you have some experience in recovering a wiped out ext4 partition? I accidentially ran mkfs.ext4 on a ext4 partition for which I do not have a backup (I wanted to format sdg but instead I accidentially took sde :( ). Is there a way to recover the data in this case? Rgds Bernhard Oh man, I'm really sorry to hear that. The first advice before trying anything is to get another hard drive or partition and make a clone or image of the partiton, in case attempting to repair it actually damages it worse. I don't know if ext4 writes its superblocks and other data in the same place every time or if it randomizes it. If it's random, then maybe the old data can still be found. I would try things like dumpe2fs, fsck, testdisk to see if it can find some files from your formatted partition. I don't know if ext4 is so similar to ext3 that perhaps ext3 tools might be able to find something. I don't know of any ext4-specific tools (other than fsck) I suspect a total recovery is unlikely (unless you had a backup). Good luck!
[gentoo-amd64] Re: Recover files on ext4 partition
Paul Hartman paul.hartman+gen...@gmail.com posted 58965d8a0907081341i22045daasaef92f2bd21d4...@mail.gmail.com, excerpted below, on Wed, 08 Jul 2009 15:41:53 -0500: On Wed, Jul 8, 2009 at 3:16 PM, Bernhard Auzingernordpolcam...@gmail.com wrote: does anyone of you have some experience in recovering a wiped out ext4 partition? I accidentially ran mkfs.ext4 on a ext4 partition for which I do not have a backup (I wanted to format sdg but instead I accidentially took sde :( ). Is there a way to recover the data in this case? Oh man, I'm really sorry to hear that. The first advice before trying anything is to get another hard drive or partition and make a clone or image of the partiton, in case attempting to repair it actually damages it worse. I'd reemphasize that again. The first rule in such a recovery is take an image and work with it, so you don't make the problem worse. dd or dd-rescue (which works better in the case of physically damaged hardware, but shouldn't matter so much here) is the usual imaging tool. Just be SURE you're pointing it at the correct write location. Beyond that, I'd suggest contacting the maintainer (Ted Ts'o) himself, or more accurately, the ext4 list (much better than mailing an individual for something like this), as ext4 is new enough and different enough from ext3, it's unlikely there's standard recovery tools out there for it yet. If it was a conversion from ext3, you can probably still get most of the original ext3 files back at least, using ext3 recovery tools to do so, but may not get anything updated since then. If it was originally ext4, that list is very likely to have tools they used in testing that will be quite helpful. They'll almost certainly have some technique hints for you, as well. From the /usr/src/linux/MAINTAINERS file (snipped and reformatted for posting): P: Person M: Mail patches to L: Mailing list that is relevant to this area W: Web-page with status/info T: SCM tree type and location. Type is one of: git, hg, quilt, stgit. S: Status, one of the following: Maintained: Someone actually looks after it. F: Files and directories with wildcard patterns. A trailing slash includes all files and subdirectory files. F: drivers/net/ all files in and below drivers/net F: drivers/net/* all files in drivers/net, but not below F: */net/* all files in any top level directory/net EXT4 FILE SYSTEM P: Theodore Ts'o M: ty...@mit.edu P: Andreas Dilger M: adil...@sun.com L: linux-e...@vger.kernel.org W: http://ext4.wiki.kernel.org S: Maintained F: Documentation/filesystems/ext4.txt F: fs/ext4/ -- Duncan - List replies preferred. No HTML msgs. Every nonfree program has a lord, a master -- and if you use the program, he is your master. Richard Stallman
[gentoo-amd64] Re: latest k3b has no menu entry
Mark Haney mha...@ercbroadband.org posted 4a54bbb9.1080...@ercbroadband.org, excerpted below, on Wed, 08 Jul 2009 11:31:05 -0400: I just upgraded to 1.0.5-r5 of k3b and now I have no menu option in Multimedia. Is anyone seeing this as well? Where do I look in KDE4 to fix it? I don't know the direct answer to that, but here, I never look in the menu for it anyway. I always start it from the open dialog (krunner in kde4), because I already know the name, it's unique enough I'm not likely to forget it, and it's short enough it's easier to hotkey the dialog and type in k3benter than it is to go looking for it on the menu. -- Duncan - List replies preferred. No HTML msgs. Every nonfree program has a lord, a master -- and if you use the program, he is your master. Richard Stallman
[gentoo-amd64] Re: Emerge and Tmpfs
Arttu V. arttu...@gmail.com posted fecdbac60907080910p1ac75f11pa686a0bb36dad...@mail.gmail.com, excerpted below, on Wed, 08 Jul 2009 19:10:12 +0300: Or at least that's what it did to me last week when I was experimenting something utterly stupid with kernel memory settings on a box with 2GB physical RAM, 8 GB swap, /var/tmp/portage mounted on tmpfs with size=9000M, and trying to emerge openoffice -- while only having compiled a kernel which could handle a maximum of 4GB of memory ... :) You had me laughing, but in all seriousness, that's a good test of conditions many of us aren't likely to see very often. So at least we know the failure mode, now, and know it doesn't fail the entire system, which is more than at least I knew before. =:^) -- Duncan - List replies preferred. No HTML msgs. Every nonfree program has a lord, a master -- and if you use the program, he is your master. Richard Stallman
[gentoo-amd64] Re: Recover files on ext4 partition
Duncan 1i5t5.dun...@cox.net posted pan.2009.07.09.00.51...@cox.net, excerpted below, on Thu, 09 Jul 2009 00:51:32 +: Beyond that, I'd suggest contacting the maintainer (Ted Ts'o) himself, or more accurately, the ext4 list (much better than mailing an individual for something like this), as ext4 is new enough and different enough from ext3, it's unlikely there's standard recovery tools out there for it yet. I forgot this... Be sure to check the list archive tho, and look for a FAQ. It may indeed be a FAQ, that they're tired of seeing on the list. (Years ago, that went without saying as it was just common netiquette, but unfortunately, common netiquette isn't so common, any more, and it way too often needs to be made explicit now days.) -- Duncan - List replies preferred. No HTML msgs. Every nonfree program has a lord, a master -- and if you use the program, he is your master. Richard Stallman
