Returned mail: Returned mail: see transcript for details
User's mailbox is full: <[EMAIL PROTECTED]> Unable to deliver mail.
Re: [PATCH] reiserfs: on-demand bitmap loading (testing only)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 David Masover wrote: > Pierre Etchemaïté wrote: >> Le Thu, 7 Jul 2005 13:59:35 -0400, studdugie <[EMAIL PROTECTED]> a >> écrit : >> >> >>> I agree w/ Jeff 100%. I'm not a kernel hacker, simply a user. As a >>> matter of fact, I was one of those people that Jeff aluded to when he >>> said: "There have been reports of large filesystems taking an >>> unacceptably long time to mount." >> >> >> That also makes reiserfs uncomfortable with automount devices, specially >> if they're bandwidth limited like external USB or firewire disks... > > USB and firewire disks already take a little long to mount anyway. > > But, it is definitely a performance enhancement, or at least a tweak. > I'd like to see it happen -- it takes 10-15 seconds to mount my 200 gig > Reiser4 partition, which is unacceptabe for a desktop machine -- at > least, for a *linux* desktop machine. > > To keep Hans happy about the "default case", can we load the bitmap in > the background during boot/mount? Basically, if it's loaded on demand, > then we pretend to demand each part of it, one by one. Would that > considerably slow normal FS operation? Could we defer it to when the > disk is idle? (*disk*, not FS) I believe there are two possible methods of delayed loading. The first is to issue all the bitmap read requests on mount and then when we need that particular bitmap later we can wait on it. The second is to issue the read request the first time it's needed, and don't let go of it. For the sake of exploring other options, I decided to implment the first one. The results were surprising. The disk I've been testing on lately is a 40 GB ATA/100 disk mounted in a USB2 enclosure. I tested with a range of block sizes so that the number of bitmaps would increase without needing a larger disk. I realize the results won't be identical to filesystems that large, but it's the best I can do with my storage constraints. Realistically, the times will be even longer on the larger filesystems. The results showed that delayed allocation was only slightly faster than waiting on the buffers. The reason for this is that the majority of the time is actually spent issuing the block read requests, not actually waiting for their results. The amount of time waiting on the blocks appears not to change radically, though the amount of time issuing the read requests does. Here are the actual numbers from the test runs. Between each mount attempt, I attempted to clear the system caches by allocating and writing to all the memory on the system, as well as the disk caches by reading 50 MB from disk. I performed the tests with four block sizes in order to increase the number of bitmap blocks that need to be loaded at mount time. Note that each decrease in block size increases the number of bitmaps fourfold. This is because when the block size is halved, it not only doubles the number of blocks, but also halves the capacity of each bitmap block. 4k block size: 2k block size: 10036464 blocks,20072928 blocks, 307 bitmaps (~= 39 GB) 1226 bitmaps (~= 153 GB @ 4k) - -opin_bitmaps -opin_bitmaps sb_getblk loop: 0.0ssb_getblk loop: 0.3999643s ll_rw_block: 1.435871744s ll_rw_block: 8.143272619s wait_on_buffer: 0.513519144swait_on_buffer: 1.990925198s real0m4.551sreal0m10.906s user0m0.000suser0m0.000s sys 0m0.060ssys 0m0.028s - -opin_bitmaps,delayed_bitmaps -opin_bitmaps,delayed_bitmaps sb_getblk loop: 0.0ssb_getblk loop: 0.3999643s ll_rw_block: 1.443871029s ll_rw_block: 8.97839s real0m2.128sreal0m8.630s user0m0.000suser0m0.000s sys 0m0.016ssys 0m0.020s - -odyn_bitmaps -odyn_bitmaps real0m0.626sreal0m0.850s user0m0.000suser0m0.000s sys 0m0.008ssys 0m0.016s 1k block size: 512b block size: 40145856 blocks,80291712 blocks, 4901 bitmaps (~= 612 GB @ 4k) 19603 bitmaps (~= 2.4 [EMAIL PROTECTED]) - -opin_bitmaps -opin_bitmaps sb_getblk loop: 0.19998214s sb_getblk loop: 0.95991426s ll_rw_block: 33.727900516s ll_rw_block: 110.98165711s wait_on_buffer: 1.423872816swait_on_buffer: 0.749324905s real0m36.052s real1m51.423s user0m0.000suser0m0.000s sys 0m0.124ssys 0m0.256s - -opin_bitmaps,delayed_bitmaps -opin_bitmaps,delayed_bitmaps sb_getblk loop: 0.23997856s sb_getblk loop: 0.95991426s ll_rw_block: 33.644994731
corrupted inode directory
Greetings, I was able to recover the hard drive partitions, thank you for your advise. Now I have 1 folder that I need, that is not accessible... ls -l states permission denied (as Root) chown & chmod states the same: Permission Denied, as Root User. I believe the inode is corrupt? not sure. Is there anyway to recover/brute this folder? or determine if inode is corrupt? Thanks! Johnny
Re: [PATCH] reiserfs: on-demand bitmap loading (testing only)
Jeffrey Mahoney wrote: David Masover wrote: Pierre Etchemaïté wrote: Le Thu, 7 Jul 2005 13:59:35 -0400, studdugie <[EMAIL PROTECTED]> a écrit : I agree w/ Jeff 100%. I'm not a kernel hacker, simply a user. As a matter of fact, I was one of those people that Jeff aluded to when he said: "There have been reports of large filesystems taking an unacceptably long time to mount." That also makes reiserfs uncomfortable with automount devices, specially if they're bandwidth limited like external USB or firewire disks... USB and firewire disks already take a little long to mount anyway. But, it is definitely a performance enhancement, or at least a tweak. I'd like to see it happen -- it takes 10-15 seconds to mount my 200 gig Reiser4 partition, which is unacceptabe for a desktop machine -- at least, for a *linux* desktop machine. To keep Hans happy about the "default case", can we load the bitmap in the background during boot/mount? Basically, if it's loaded on demand, then we pretend to demand each part of it, one by one. Would that considerably slow normal FS operation? Could we defer it to when the disk is idle? (*disk*, not FS) Hi David - The main issue I have with this is that I don't think bitmaps should be treated specially. They are metadata, pure and simple. This is the philosophical reason why I like on-demand loading. There's also the immediate speedup of boot. But, I think Hans has a point that it may be better for performance to pre-cache them. I would rather the default behavior be to load them on demand, but I can see situations where people would choose to pre-cache them, or even (as we do now) force them to stay in kernel memory as long as the FS is mounted. But that's not a sane default. Big hard drives on desktop machines are getting more and more common, and even the lowly Linux gamer doesn't want to waste the RAM he bought for Doom 3 on a 200 gig filesystem when he's only using 1.5 gigs of it at the moment. I would guess that this is a lot more common of a scenario than massive 2TB arrays where people can throw money (RAM) at the system to make it faster, in any way they can. But even if it's not, people with 2TB arrays are much more likely to discover the precaching feature and turn it on than gamers / desktop users woud be to discover it and turn it off. And besides, for the average desktop machine, it's latency that matters more than throughput. The most noticeable latency that we can optimize for is boot time, the next most noticeable is launching a new app / changing apps. For the average desktop user, it doesn't matter if it takes an extra half second to load a chunk of the bitmap in order to load apps, and it certainly doesn't matter if it takes an extra tenth of a second to load a file, but it does matter if RAM wasteage pushes an app into swap and it takes 5-10 seconds (at least) to switch apps, and it does matter if it takes 5-10-20 seconds longer to boot. That is why I think on-demand should be default. The root block/node sees far more activity than any bitmap and it is not treated any differently than any other bit of metadata. It's simply requested when it is needed. If the vm has determined that the root block is frequently used, it stays in memory. Why should the bitmaps be any different? Because bitmaps are harder to seek to? I think that's the argument -- the bitmap is going to be pushed out of memory because it's used less frequently, but it'll take much longer to load than anything else because it's spread over the disk. But, you could make a similar argument about most files. I second the request for benchmarks.
Re: reiser4 plugins
Did you mean to reply to the list? I'm taking the liberty of sending my reply to the list. On 2005-07-06 17:50:07 -0400 Horst von Brand <[EMAIL PROTECTED]> wrote: Hubert Chan <[EMAIL PROTECTED]> wrote: On Wed, 06 Jul 2005 16:33:23 -0400, Horst von Brand <[EMAIL PROTECTED]> said: Hubert Chan <[EMAIL PROTECTED]> wrote: If you can store the parents, then finding cycles (relatively) quickly is pretty easy: before you try to make A the parent of B, walk up the parent pointers starting from A. If you ever reach B, you have a cycle. If not, you don't have a cycle. (Hmmm. Do I need a proof of correctness for this? It's just depth-first-search, which everybody learned in their first algorithms course.) Correct. And you need space for potentially a huge lot of up pointers for each file. Which space are you talking about? Disk space, or memory space? Anyways, for disk space, you're basically just doubling the size of the tree -- two pointers per link instead of one. As for memory space, the DFS only takes a couple of words times the depth of the tree. People (that I know of) don't normally have a huge lot of hardlinks to a single file. Can't rely on that... And then there is the (very minor) problem is that meanwhile /nothing/ can touch the filesystem to do any change... If the DFS is quick, it shouldn't make much difference. Lock nodes as you walk up the tree, and people can change unlocked nodes without harming anything. All you need to do is make sure that no up pointers get added to nodes that you've already looked at. If somebody meanwhile tries to link an ancestor of (one of) your node(s) to a descendent, you've got two DFSes going at once through the same paths... a single lock won't help. Good point. You would need to use a counter instead of just a flag for the lock. There's another problem with my proposal though (finding it is left as an exercise for the reader ;-) ). I think it can be avoided, but I don't have the time right now to properly write it up. (No time to properly write out what the problem is either. I'm already behind on my work as it is!) Maybe next week... (Out of town this weekend.) Anyways, you only really need to prevent creating other hard links while doing the DFS. Unliking won't cause a problem -- worst that can happen is that the DFS gives you a false positive. (OK, you need to make sure that the new parent that you're linking the child node to doesn't disappear from under you.) Creating new files/directories won't case a problem -- they're new files/directories; they can't cause cycles. Besides, you lost the nice property of top-down lock ordering that makes the traversal of Unix filesystems naturally deadlock-free. Yes, I don't know much about what locks the filesystem/VFS does already. I'll need to look into that. -- Hubert Chan <[EMAIL PROTECTED]> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred.
Re: reiser4 plugins
Hubert Chan wrote: On Wed, 06 Jul 2005 23:42:50 -0700, Hans Reiser <[EMAIL PROTECTED]> said: Oh no, don't store the whole path, store just the parent list. This will make fsck more robust in the event that the directory gets clobbered by hardware error. I don't think it affects the cost of detecting cycles though, I think it only makes fsck more robust. It doesn't affect the worst-case cost of detecting cycles; by necessity, any (static [1]) directed cycle detection algorithm must take O(n) time, n being the size of the tree (nodes + links). However, under "reasonable assumptions" (where the reasonableness of those assumptions may be questioned, but I think they're reasonable), I think that doing a depth-first-search through the parent links makes the algorithm not-too-terrible. Namely, the "reasonable assumptions" are that a node doesn't have "too many" parents, and the filesystem hierarchy is not "too deep". And, remember, there are other things affected by depth of the tree anyway. For that matter, most of the time, when you push a system past "reasonable assumptions", you hit performance issues, if not stability ones. Most everything but Reiser assumes that you won't have "too many" files in a directory, or "too many" small files in the FS as a whole. I think it's fair to assume people will keep things "reasonable", especially if we tell them what "reasonable" means. As in, "This is the kind of organization which Reiser4 handles really well, and this is the kind of organization which will absolutely kill your performance."
Re: how to use compression with reiser4
Louis-David Mitterrand wrote: Hello, I just converted a server to reiser4 for a big speed gain. Thanks, this looks really promising. How can I activate compression on certain parts of the filesystem? (I digged google for this without finding anything). Hello. Reiser4 will support a special kind of regular files - so called cryptcompress objects. Unfortunately this is not for product using for a while, but benchmarks really show a speed gain for some conditions (if cpu is powerful, compression algorithm is fast, and data is compressible). Thanks, Edward.
how to use compression with reiser4
Hello, I just converted a server to reiser4 for a big speed gain. Thanks, this looks really promising. How can I activate compression on certain parts of the filesystem? (I digged google for this without finding anything).
surprise your woman
"I just wanted to write and thank you for Spur-M. I suffered from poor sperm count and motility. I found your site and ordered Spur-M Fertility Blend for Men. I have wondered for years what caused low semen and sperm count, and how I could improve my fertility and help my wife conceive. Spur-M seems to have done just that! Thank you for your support." Andrew H., London, UK "Spur-M really does help improve fertility and effectiveness of sperm and semen motility. I used it for the past few months, and not only does it work - I also feel better to. I have more energy. This is an excellent counter to low sperm count and motility. I'll be buying more!!!" Franz K., Bonn, Germany http://Lorraine.medianly.com/spur/?sheep
