Re: silent semantic changes with reiser4
> Spam <[EMAIL PROTECTED]> wrote: >> >> >> >> > Spam <[EMAIL PROTECTED]> wrote: >> >> >> >> >> >> >> >> > Spam <[EMAIL PROTECTED]> wrote: >> >> >> >> >> >>Yes, for example documents, image files etc. The multiple data >> >> >>streams can contain thumbnails, info about who is editing the file >> >> >>(useful for networked files) etc. Could be used for version handling >> >> >>and much more. >> >> >> >> > All of which can be handled in userspace library code. >> >> >> >> > What compelling reason is there for doing this in the kernel? >> >> >> >> >> >> Because having user space tools and code will make it not work with >> >> everything. Keeping stuff in the kernel should make the new features >> >> transparent to the applications. >> >> >> >> Applications that support the new features will benefit, all others >> >> will continue to work without destroying data. >> >> > Sorry, but that all sounds a bit fluffy. Please provide some examples. >> >> We already had the examples with cp and mv. Both should continue to >> work and the files will still be copied. The same with Konqueror and >> Nautilus. Files and their meta-files/streams/attributes will be >> retained as long as applications are using the OS API. >> >> However, if things are to implemented in user-space, then old >> applications will not work correctly and there is risk that all the >> extra information will be lost or corrupted. >> >> You said it would be socially hard. I think it would be very much >> close to impossible to get it right. Imagine that Gnome and Nautilus >> would implement support for these. I doubt that cp, mv, KDE, mc, >> app-xyz would implement this anytime soon and in the meantime the >> data is at risk. >> > No. All of the applications which you initially identified can be > implemented by putting the various bits of data into a single file and > getting applications to agree on the format of that file. > For example, some image file formats already support embedded metadata, do > they not? Yes, JPEG, TIFF and PNG files for example. But, if you modify any of these with an application that doesn't support the extensions then you will loose them. Also, you are thinking _very_ narrowly now. There are thousands of file formats. Implementing the support for meta-data/ streams/ attributes in the kernel will make it possible to use this generically for all files. I use this in Windows quite much. I put information description fields for lots of different files. These descriptions are then searchable etc. I could use the command prompt to copy the files and the descriptions would still be there. Secondly, do you expect file managers like Nautilus and Konqueror to support every piece of file format on the planet so they could read information directly from the documents? ~S
Re: silent semantic changes with reiser4
> On Thu, Aug 26, 2004 at 11:53:30AM +0100, Jamie Lokier wrote: >> Hans Reiser wrote: >> > being able to cat dirname/pseudos/cat and get a >> > concatenation of all of the files is nice, and being able to cat >> > dirname/pseudos/tar and get an archive of the directory is nice >> >> Yes. Being able to cd into filename.tar.gz and filename.iso is also >> nice, but all of these features should be supported by the VFS >> generically, not in any specific filesystem, and there should be a >> hook to invoke the various fun filesystem-independent handlers by name. > It doesn't belong into the kernel at all. If at all it belongs into a > userspace filesystems, but even in that case the magic detection of > which one to use is kinda hard. You absoutely don't want to hardcode > file formats in the kernel. Do you mean user-level file system as a VFS handled by user applications, or a intermediate file system layer between any application and the real file system? The latter would be good enough as it would still be transparent to the applications. ~S
Re: silent semantic changes with reiser4
> Chris Wedgwood wrote: >> On Thu, Aug 26, 2004 at 02:00:49AM +0100, Jamie Lokier wrote: >> >> > One of the big potential uses for file-as-directory is to go inside >> > archive files, ELF files, .iso files and so on in a convenient way. >> >> Arguably this belongs in userspace --- and people have put it there. > I agree that these belong in userspace, and that there's plenty* of > userspace code doing a similar thing already. I don't think there's > any argument over it. > However, as far as I know it's not accessible in a file-as-directory > form as yet. In my opinion that is the most natural form and it would > be very intuitive to use. I hope we can pick a useful semantics for > them, and also provide filesystem-independent plugins with GNU > Hurd-like per-user extensibility. > -- Jamie > * plenty == too much. > Gnome, KDE, Emacs and Bash all see different virtual filesystems. > (All but Bash implement their own virtual filesystem extensions). > That makes them much less useful than they could be. Exactly, and I doubt they have the ability to join together and create a common uniform standard out of it either. They have just far to different views on Linux and how they want things to be done.
Re: silent semantic changes with reiser4
> Spam <[EMAIL PROTECTED]> writes: >> Keeping stuff in the kernel should make the new features >> transparent to the applications. > No, it will make just one special case, rename within the same > filesystem, work. (Well, two special cases, rm will also delete > the other forks). > Unless we add a new copy(2) syscall (which would be nice for other > reasons) all applicatons that copy files will fail to copy the > streams. How so? The whole idea is that the underlaying OS that handles the copying should also know to copy everything, otherwise you can implement everything into applications and just skip the whole filesystem part. Of course, it won't work if applications themselves are actually transferring the data and create the new files directly onto the filesystem. This just seem so wrong. ~S > So no working cp command, no nautilus nor konqueror without > changes to the application. And if you have to change the > applications anyway, isn't it much better to agree on a shared library > in userspace that everyone uses? Which has the added bonus that it > can be made to work on top of Linux, Windows, Ultrix and VMS? > /Christer
Re: silent semantic changes with reiser4
> Previously Spam wrote: >> How so? The whole idea is that the underlaying OS that handles the >> copying should also know to copy everything, otherwise you can >> implement everything into applications and just skip the whole >> filesystem part. > UNIX doesn't have a copy systemcall, applications copy the data > manually. Oh, this is very unfortunate and should be a bigger issue to fix. > Wichert.
Re: silent semantic changes with reiser4
Hello Markus, Friday, August 27, 2004, 11:21:31 AM, you wrote: > On Thu, Aug 26, 2004 at 04:57:26PM -0400, Lee Revell wrote: >>On Thu, 2004-08-26 at 16:50, Christophe Saout wrote: >>> are read-only and system-wide and the user-overridden changes. I don't >>> know if all of these things would really make sense inside the kernel. >>True. FWIW, I never use most of those features. It's just too damn >>slow. Windows seems to implement all of the useful features of >>GnomeVFS, and they are 10x faster. > Are they in the kernel in Windows? Some are, some aren't :) Many of the features are between the FS and the application layers. Therefore they will work with all programs that use the Windows API to access files etc. I do not think that many programs actually access disks directly at the FS or below level. Exceptions would perhaps be file-recovery tools etc. Others are simply plugins to Windows Explorer. I don't know if all, any, or none, are installed at the kernel level. I doubt they are. But the problem with Linux seem to be that programs actually do access the FS directly. Therefore everything will break if you change the FS structures to much. Like I said, this is IMHO something that Linux should somehow solve. Applications should really not access the filesystems directly. Perhaps Grub and stuff like it should, but not many other. ~S
Re: silent semantic changes with reiser4
I am thinking that is there was a proper API for accessing the filesystem then this problem wouldn't arise because things could be done behind the curtains inside the API, instead of having all the tools to be rewritten to know. Think of FAT32, for example, the new filenames will be retained even if you use old tools on them. It is only if you mount it as a FAT16 volume they will be lost. Even if this is fixed for Reiser4 now. Next time someone wants to make changes like this we have the exact same problem yet again. > Just have a special name instead of a special boundary, or, better, have > a filename/pseudos/backup method that outputs everything needed to > backup the object "filename". > Hans > Linus Torvalds wrote: >>On Fri, 27 Aug 2004, Rik van Riel wrote: >> >> >>>Thing is, there is no way to distinguish between what are >>>virtual files and what are actual streams hidden inside a >>>file. You don't know what should and shouldn't be backed >>>up... >>> >>> >> >>I think that lack of distinguishing poiwer is more serious for >>directories. The more I think I think about it, the more I wonder whether >>Solaris did things right - having a special operation to "cross the >>boundary". >> >>I suspect Solaris did it that way because it's a hell of a lot easier to >>do it like that, but regardless, it would solve the issue of real >>directories having both real children _and_ the "extra streams". >> >> Linus >> >> >> >>
Re: silent semantic changes in reiser4 (brief attempt to document the idea of what reiser4 wants to do with metafiles and why
>>>>>> "Spam" == Spam <[EMAIL PROTECTED]> writes: V13>> The only thing that changes (from the userland POV) is the way V13>> someone can enter the 'metadata directory'. This way you don't have V13>> to have a special name, just a special function and no existing V13>> application (like tar) can possibly break because it will not know V13>> how to enter this 'metadata directory'. >>> tar won't be able to backup the metadata. That's the major breakage >>> of tar that we're worried about. Spam>> However, if we do a "cp fileA fileB" then the metadata and Spam>> streams ought to be copied too, even if "cp" does not support Spam>> them. > Huh? How do you plan on pulling that off? Unless cp uses the > not-yet-existing copy syscall, if cp can't see the metadata or streams, > how is it going to copy it? My first thought would change the API that cp uses to copy the file. But in an earlier response on this message thread I have found out that there is no such API (well there is, but it is linked into the program) and cp in fact itself is doing the copying. this is also what I objected against before. It is a bad design and should be attended much higher interest to change than just adding adding a new type of file-as-dir. Spam>> This is the real challenge. Backup tools like tar can be patched Spam>> just like it has so many times before. > Yes. And if we can get file-as-dir, then we only need to patch tar > once, since everything can be exported through that interface. Yes. This seem to be an acceptable way to do things. But next time someone comes and want to do changes like this we need to start patching things again. If there was an API that was separate from the programs then new features could be included much more easily as things could be done behind the scenes. ie the "cp fileA fileB" would succeed and all the extended attributes, metas, streams etc would be copied too. Nothing would ever be lost unless copying to a filesystem that doesn't support the special stuff. (as with NTFS file streams). ~S
Re: silent semantic changes with reiser4
> Linus Torvalds <[EMAIL PROTECTED]> writes: >> In a graphical environment, the "icon" stream is a good example of this. >> It literally has _nothing_ to do with the data in the main stream. The >> only linkage is a totally non-technical one, where the user wanted to >> associate a secondary stream with the main stream _without_ altering the >> main one. THAT is where named streams make sense. > I think that the "icon" argument for named streams is a silly > argument, since different users may want to have different icons for > the same file. Say that I want /usr/bin/emacs to have the enterprise > icon and someone else wants the gnu head icon. And besides, root owns > the file anyways, so neither of us mortal users should be able to add > a stream to it. Yet again are we thinking in blocking ways. Firstly this was an example. Usually, though, most users accept the default icon for a file. If they do not they can still change the icon for the link they make on their start-menu/home folder/etc. > Another reason for named streams that usually crops up is the ability > set a "preferred application" for a certain file, so that when I > double click on a document I want to open it with antiword instead of > openoffice. But the same contra-argument applies here, different > users have different preferences. I can make the same argument as for the icons. > I can see the argument for having the equivalent of Content-type or > Content-transfer-encoding as a named stream though. That would be a nice thing. > /Christer
Re: The argument for fs assistance in handling archives
> Linus Torvalds wrote: >> But _my_ point is, no user program is going to take _advantage_ of >> >>anything that only one filesystem on one system offers. >> >> > Apple does not have this problem > and yes, the apps will take advantage of it, which is different from > depending on it. If you use the wrong fs you will lose some of the > features of the app. > For 30 years nothing much has happened in Unix filesystem semantics > because of sheer cowardice (excepting Clearcase, which priced itself > into a niche market). It is 25 years past time for someone to change > things. That someone will have first mover advantage, and the more > little semantic features possessed the more lure there will be to use it > which will increase market share which will lure more apps into > depending on it and in a few years the other filesystems will > (deservedly) have only a small market share because the apps won't all > work on them. > Besides, there are enhancements which are simply compelling. You can > write a dramatically better performance version control system with a > much simpler design if the FS is atomic.Our transaction manager > first draft was written by a version control guy, and he would probably > be happy to tell you how lack of atomicity other than rename makes > version control software design hideous. Btw, version control for ordinary files would be a great feature. I think something like it is available through Windows 2000/3 server. Isn't it called "Shadow Copies". It works over network shares. :) It allows you to restore previous versions of the file even if you delete or overwrite it. Features like this do make a good point and helps protecting data - something that is important IMHO. http://www.microsoft.com/resources/documentation/windowsserv/2003/standard/proddocs/en-us/overview_snapshot.asp > We have the performance lead. By next year we will be stable enough for > mission critical servers, and then we start the serious semantic > enhancements. > If you don't embrace progress, then you doom Linux to following behind, > because the guys at Apple are pretty aggressive now that Jobs is back, > and they WILL change the semantics, and they will do so in compelling > ways, and Linux will be reduced to aping them when it should be leading > them. > Hans
Re: The argument for fs assistance in handling archives
>> Depends on how the forks eventually get implemented. >> With the file-as-directory concept, all you need is to >> look at the file's directory part to see what is there. (The forks, >> implemented as files in a subdirectory.) It is done the same way >> as for an ordinary directory, so nothing "new". >> > This still doesn't solve the problem of how we distinguish between a > multi-fork file > and a directory, for those old programs(ie. almost all that currently > exist), to be able to access meaningful data we would need to either > return just the primary fork, a serialized version of all forks in the > file, or make the file look more or less identical to a directory. Well. wasn't the idea that unless programs specifically tried to open the file-as-dir as a directory it would look like a file? ls -F would show it as file. Or have I understood wrong? > The first option could cause problems when transfering files between > different filesystems, The meta-data should be deleted if it the file is copied or moved to a medium that doesn't support it. However a _warning_ may be shown to the user if there is risk to loose data. I do not think we should not implement something like this because most other filesystems in Linux doesn't support it. Think of other limitations there is in various ones. Large file support. Limited amount of files, length of file names, etc. > the second results in programs getting metadata they can't handle, and > the third option results in few of the advantages over, well, > directories... And even those applications that could handle the fork > information nicely would need to fs type to find out whether they were > handling a directory or a multi-forked file... No, if an application supported the file-as-dir then it could simply try to open the file as a directory. It would fail on systems that didn't support it. The meta-data and file streams would be seen as ordinary files. If applications support the contents or not isn't really relevant, is it?. > As I say I like the idea, but I can't see anyway of implementing it in > a way that is useful without first putting considerable effort into at > least the VFS if not all the actual fs drivers. Indeed. It is important that something like this gets implemented as a transparent way as possible. If it could be done in a general way so other filesystems like ext3/4 can eventually support it then that would be wonderful. I do not, however, think that we should block it in reiser4 because no other filesystems support it. ~S > --Oliver Hunt
Re: The argument for fs assistance in handling archives
> On Thu, 2004-09-02 at 11:22 -0700, Martin J. Bligh wrote: >> > For 30 years nothing much has happened in Unix filesystem semantics >> > because of sheer cowardice >> >> Or because it works fine, and isn't broken. > OK. I'm not a kernel hacker. I'm not a crack C programmer. Nor am I a > designer of filesystems. I'm just a guy that puts together and supports > Linux systems for my customers. > In following this thread, I may be missing huge chunks of concept. > However, a few things are becoming clear to me: > 1. The file as directory thing adds complexity that the administrator > has to deal with. Symlinks are useful, but it's still aggravating to > tar off a directory structure, take it somewhere, and then realize that > all you have is links to something not in the archive because you didn't > get your tar switches just right. Now we're talking about adding > another set of "files which are not really files" to the semantics. > More complexity. I'll take simplicity over some ivory tower ideal of > "unified name space" any day. Are you afraid to learn something new? ;) Just joking. But really, it doesn't have to be very difficult. The extra streams etc would just be saved as files. If tar is patched then it would be no problem and no extra stuff but perhaps a switch --save-metas. > 2. The use of multiple streams within files by Linux apps would make > Linux as cross-platform unfriendly as MS is trying to be. Say these > features start getting used and you copy an OO.org document from a Linux > box to a BSD box. It's broken. Of course, OO.org wouldn't use the > streams in the first place because it would destroy their cross platform > portability. So what's the point? No one who cares about cross > platform portability can use it. Everyone who doesn't care about cross > platform portability please raise your hand. Actually MacOS and Windows support multiple streams, only Linux doesn't. But of course there are BSD's etc too. I'd say to leave them behind. The file streams would make my day a lot easier. The idea to split up contents of OO.org files into streams is bad. But that doesn't make the file streams bad. I see many uses that would make my every day life easier. It isn't about cross plat form compatibility, but to add features that are useful and meta-data and file streams are. Also. No one forces you to use either meta-data or streams, just as no one forces you to use ACLs or other things. > 3. MS does require attributes and multiple streams, which makes these > features important (even essential) to Samba, and Samba alone. Samba is > important to Linux, so this can't be ignored. (Here I am implicitly > assuming that Samba will need kernel support for this to do it right.) I do not think the Samba would really require the streams support but it would certainly make life easier for Samba. Not to mention that these files would also be natively viewed on the Linux host. > So it seems to me that the only real consideration is giving Samba what > it needs without making the semantics one bit more complex than > absolutely necessary. It might even be wise to discourage use of these > ambiguous new objects by the casual application programmer. > Then again, maybe I just have tunnel vision... I would agree with tunnel vision. The kernel should provide the tools and options. Users and developers can then invent new things to use them. :) ~S > -Steve Bergman
Re: silent semantic changes with reiser4
> On Iau, 2004-09-02 at 20:41, Spam wrote: >> > It is trivial to implement this by looking inside the files. I.e., the way >> > mc has done this for ages. >> >> Difference is that you can't do "locate" or "find" or "Search".. You >> would have to open the files in an archive-supporting application >> such as mc. > And would you rather that logic was running swappable in shared library > space or privileged and unswappable in kernel ? I would rather have it as a filesystem/vfs plugin that would allow all my programs to use the features the plugin gives, even if it would mean being totally in kernel space. > Alan
Re: silent semantic changes with reiser4
> On Thu, Sep 02, 2004 at 04:01:17PM -0400, Lee Revell wrote: >> Better to have the contents accessible via a separate stream, in the >> same namespace. Fix it once in the kernel vs. fix it in umpteen >> apps. > This is ridiculous. We have shared libraries, 99% of applications > manage to use libc for example --- this isn't that different. Yes if that was the case then programs wouldn't brake if libc and its API was fixed. Sadly, I think it isn't. ~S > --cw
Re: silent semantic changes with reiser4
> On Iau, 2004-09-02 at 21:07, Spam wrote: >> > And would you rather that logic was running swappable in shared library >> > space or privileged and unswappable in kernel ? >> >> I would rather have it as a filesystem/vfs plugin that would allow >> all my programs to use the features the plugin gives, even if it >> would mean being totally in kernel space. > Thats not what I asked. I failed to understand your question then. :( ~S
Re: The argument for fs assistance in handling archives
> -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > [EMAIL PROTECTED] wrote: > | On Thu, Sep 02, 2004 at 11:48:06PM +0200, Frank van Maarseveen wrote: > | |>>mount is nice for root, clumsy for user. And a rather complicated |>>way of accessing data the kernel has knowledge about in the first |>>place. For filesystem images, cd'ing into the file is the most |>>obvious concept for file-as-a-dir IMHO. > | > | > | The hell it is. > | > | a) kernel has *NO* *FUCKING* *KNOWLEDGE* of fs type contained on a device. > reiser4 kernel will contain knowledge of fs type contained in a file. > "file/..metas/type" might contain a mime type. Mime type might have to > be guessed, but at least if it's made by a local "mkisofs" then we're fine. > Indeed, that's not the only interface that's been discussed. > "file/..metas/is_isofs" might be consulted. What you are talking about isn't the kernel or such, but plugins that could extend the filesystem. Plugins could store information about contents, encodings, formatting, filesystems, etc, as meta-info. If you have a plugin that would allow you to traverse files as disk images then it could read those meta-data. But before those plugins exist then there is no such standard for info stored as meta-data and the kernel wouldn't know anything about this to begin with. > | b) kernel has no way to guess which options to use > | c) fs _type_ is a fundamental part of mount - device(s) (if any) involved > | are arguments to be interpreted by that particular fs driver. > Unless there's some severe security issues with "mount this iso as fat > and you get root access", this should work fine. > I see no reason why there can't be a global setting of the mount > commandline to use. > And it doesn't all have to be in the kernel. Only it'd be nice to have > some of it there because the kernel knows how to deal with an isofs, > even if it won't know what it looks like. > | d) permissions required for that lovely operation (and questions like > | whether we force nosuid/noexec, etc.) are nightmare to define. > They are quite simple, actually. Just set them globally -- some admins > would force nosetuid/noexec, some wouldn't. And the operation happens > transparently -- you need no "permissions" other than to read the > directory which would contain the mount. > | Frankly, the longer that thread grows, the more obvious it becomes that > | file-as-a-dir is a solution in search of problem. Desperate search, at > | that. > Actually, the longer this thread grows, the more obvious it is how when > there's a hot issue, everyone has an opinion, even if the same opinion > has been expressed ten or twenty times already. > File-as-a-dir has numerous advantages, but enough have been discussed. > Short list is image mounts, tarballs, streams, metas, and namespace > unification. Longer list and explanations can be found if you RTFA. > -BEGIN PGP SIGNATURE- > Version: GnuPG v1.2.4 (GNU/Linux) > Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org > iQIVAwUBQTe19XgHNmZLgCUhAQJdow/+Knsw1GgpauqDUcg8sKtxzgXZ18OxMQ3Q > By8sRrSTuKAzI5A3BtYIzndsj1veP+7wndG7nYPz8NS1fU2+xWSIhoGq/YMaQsu4 > 70uMLu448PFXZua4hZMk2w4mkULXbGyYHJ1Bf+2Z7QkQ/8W08hozC8QQynxMXIkX > SrcWCS5hK8Nh7Ol691sDpPqexH7F1GwUyoslNGj63U5r6ViLAawt2ZKDYdT7ZPo8 > 0a/pWUHoHMPbv/KwqZZxRr1/qncA9QYQo6JqQBPPCr+tWNJs/ei3nAKGi58iOt1M > DK1TEKd2lpbmwiK5pWDwGz+nwWmaFTAyfTEEEcP4gZedSJtRXaxyNh0jRl1iLATB > SCO5Eb4jkQs8hdjHqQcQ1q7XKFX9eSXWeDdrGrtWaYC/QYOHxT+ci3lnKBKCG99Y > YTqg3sNEZlV1N0jIcNvFSDEYbbX12v1Y6xbwvUx48+sMyUj3suT76niTRbwEydfO > MA9y+wE2k4wF+h+sJCbTjimCNFvvuFTTJuBCbQTpfY4eOYBAFalxnWmrpTEfVzka > 4iAqAYygWObGDkFFy/rp1HEVZPIKM0NwGLOsRwJsgyUMOsccBrEc0bg8sgMECVfs > 5qNIb27tLokh8NBR6RodAv2NZYKC+foM0T+PC5bZMFD/Q7f6yDklqK4C4RCIYaXj > xO9z1C6FPcM= > =/gmK > -END PGP SIGNATURE-
Re: The argument for fs assistance in handling archives
> -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > Martin J. Bligh wrote: |>>For 30 years nothing much has happened in Unix filesystem semantics |>>because of sheer cowardice > | > | > | Or because it works fine, and isn't broken. > Ok, maybe it wasn't cowardice. Maybe it was laziness. > Because cowardice tells me that data corruption is bad. Data corruption > kills kittens. End data corruption now! > Unfortunately, the only way to allow files to survive a crash is to > implement transactions. The simplest way is to do it in the filesystem > and to export an interface to userland. This is what Hans is doing, > though AFAIK it's not done yet. I thought reiser4 had its journaling and atomic commits. Am I mistaken? I run reiser4 as primary fs on my test systems and it seem to work as expected. > Laziness tells me that I should just use Windows anyway, because "it > works". Switching to Linux requires training, porting apps requires > work, and using Linux instead of Windows is a hell of a lot more work > than just patching programs to do something like > + start_transaction() > ~ write() > ~ blah() > ~ write() > + end_transaction() > How'd you like to be able to tell users "If a file gets corrupted, it's > either a bug in OpenOffice or your hardware eats data." > -BEGIN PGP SIGNATURE- > Version: GnuPG v1.2.4 (GNU/Linux) > Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org > iQIVAwUBQTe9pngHNmZLgCUhAQLTZQ/5AZq31sPKOWB/eI4WtVhM76V2+pmQdfNk > aZkPtwhZlVrGDJscp45OVNthwMiSskD07om4obUMac79BCmA5805clZ7i0X1uzQB > SecyX/7XITWz3iwF4VIaxJgDoo7YPFfbpWS0NTmhDJPt6h5Z+goEccLdTABj5kVk > gbex4xNgXzPrnopvjnWx+K2K3ydvXiX+8bWV200F9S+j0uCObcfSZ357uQPqlReP > gKQhEs0pCcoQQua81rXJfUhP8hCbegCpUmhJNI0MwoTUvHTnBU/+99+cCsTLR4gN > 1XlbS9SIaaBXG7/AV9L7iQO5GF1t8mTMSe163EOzZ7ypLDb2km5e7zZ7t2DrA6SE > Jgd0xT3cHjDgp1b3qhdeWYvt0QVfz5CGaiHukkT4pEOpsrM09BOhDhY72B74swuV > byDIJ+y4X3J08i/a0zt15hBlVz1FUq7ac7doZZDt3orj93t3B/y4Xpb7tgzVGoWP > T5AkR+uS5dGRUoMSI/btnPIK4ERlhbwFzCB5lgkas5kjp4dnawIKAk1wxPdhvEle > g3SsNqSfBsA9hs6tzVw341GaWj6AWV7oOfg3j0c9I+7nt8SX5UeuTCwBWtCuwd/H > +nnBulusiMnAm3tzQxq8aMpSVAqsZDywjgXSler1mNik4Ow5I5JdXl+Bf2ZWHF+G > VHk4JVxMt/c= > =3z3Z > -END PGP SIGNATURE-
Re: The argument for fs assistance in handling archives
> -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > Oliver Neukum wrote: > | Am Donnerstag, 2. September 2004 11:52 schrieb Spam: > | |>> Btw, version control for ordinary files would be a great feature. I |>> think something like it is available through Windows 2000/3 server. |>> Isn't it called "Shadow Copies". It works over network shares. :) |>> |>> It allows you to restore previous versions of the file even if you |>> delete or overwrite it. > | > | > | There's no need to do that in kernel, unless you want to be able > | to force it unto users. > And on apps. Should I teach OpenOffice.org to do version control? > Seems a lot easier to just do it in the kernel, and teach everything to > do version control in one fell swoop. Do you mean in the kernel or as a filesystem/VFS plugin that would extend the functionality to include version control? ~S
Re: The argument for fs assistance in handling archives
> On Fri, Sep 03, 2004 at 09:30:38PM +0200, Spam wrote: >> >> >> >> > Helge Hafting <[EMAIL PROTECTED]> said: >> >> > [...] >> >> >> The only new thing needed is the ability for something to be both >> >> file and directory at the same time. >> >> > Then why have files and directories in the first place? >> >> Good point, we don't need them :) Directories are just a visible >> grouping of files to make it easier for the user to manage. But some >> things aren't really that intuitive with todays layout - especially >> for non-unix users. >> >> Just an example where the user needs to edit config file for some >> program. Where should he look? >> /etc/app.conf ? >> /etc/app/app.conf ? >> /etc/conf.d/app.conf ? >> /var/lib/app/app.conf ? >> and so on >> >> Using file-as-dir isn't really that much of a change. It isn't those >> that will confuse people anyway. >> > If the non-computer literate user is expected to edit the config file, > a pretty interface that hides the internals from the user should be provided. > For the sake of power users, users that believe themselves to be power users, > and configuration changes on multiple systems. > 1) Track the location of the config file with the native package management > system. > 2) Keep a human readable/editable format. > 3) Provide APIs or schema so that the files may be parsed for correctness, > or edited or created programatically. > 4) Provide a tool that not only checks for a parseable file, but does > sanity checks such as (Does file exist). > I'm at a loss as to how any of this has anything to do with filesystems. > The closest I see is one implementation of the above (gconf2) happens to > use the filesystem for it's default hierarchical database backend. The relevance was as to the presentation of meta-data, streams and such and their implication on users. I just used the example of config files because there are not so much of a standard among them. Administrators still need to learn and remember where and what to backup. This wouldn't be so much of a change if they decide to introduce some usage for meta-data or named streams. Remember, even if the functionality exist you do not have to use everything of it. > As for developers that are too stupid/lazy to provide the GUI and meet > the other criteria... I suggest extensive use of cattle prods on > them as they attend an automata class. Lets all start with the bash(ing) ;). But really. What you say is true. The users do not care so much about what is going on below the curtains. Here is where the clash between different groups of Linux users and developers appear most vivid. Sure KDE and Gnome are on their way, but their efforts still have a long way to go. They still don't talk to eachother to strive for a comon goal - but competes instead (which may even be a good thing). ~S
Re: silent semantic changes with reiser4
> Hi! >> >> What if I do not use emacs, but vim, mcedit, gedit, or some other >> >> editor? It doesn't seem logical to have to patch every application >> >> that uses files. >> >> > We would have to do that in either case, so let's patch them to do it >> > in a nonintrusive way. And as to reading and writing inside tar files, >> > write and/or use a really nice userspace library to do it. (As does >> > MacOS/X, as does KDE, etc.) >> >> The problem with the userspace library is standardization. What >> would be needed is a userspace library that has a extensible plugin >> interface that is standardized. Otherwise we would need lots of >> different libraries, and I seriously doubt that 1) this will happen >> and 2) we get all Linux programs to be patched to use it. > libvfs from midnight commander (and anything build on the top of it) > already is very extensible. This wasn't my point. My point was about all applications using it. Most aren't. > Pavel
Re: The argument for fs assistance in handling archives
> -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > Frank van Maarseveen wrote: |>> |>>We have the mount command for that. :^) > | > | > | mount is nice for root, clumsy for user. And a rather complicated > | way of accessing data the kernel has knowledge about in the first > | place. For filesystem images, cd'ing into the file is the most > | obvious concept for file-as-a-dir IMHO. > thats why we have automount. Which still needs to be setup in fstab, right? > lg, clemens > -BEGIN PGP SIGNATURE- > Version: GnuPG v1.2.5 (GNU/Linux) > Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org > iD8DBQFBPF3djBz/yQjBxz8RAmVhAKCODKNKOV9V0I59SUfQ1pp+qk88xwCg6+4p > Q+JcbO+W0+6cUfSEf3z/Iwk= > =IwZQ > -END PGP SIGNATURE-
Re: The argument for fs assistance in handling archives
> On Mon, Sep 06, 2004 at 09:59:25PM +0900, Clemens Schwaighofer wrote: >> -BEGIN PGP SIGNED MESSAGE- >> Hash: SHA1 >> >> Spam wrote: >> >> | thats why we have automount. >> | >> | >> |> Which still needs to be setup in fstab, right? >> >> no, in /etc/automount* actually. I have no fstab entry here and still I >> can mount various samba shares, cdroms, etc ... > but can you access images that way? Probably, but only predefined ones.. IE, no random automount? ~S
Re: The argument for fs assistance in handling archives (was: silent semantic changes with reiser4)
> Salut, > On Fri, Sep 03, 2004 at 01:43:02AM +0200, Spam wrote: >> Yes why not? If there was any filesystem drivers for the AudioCD >> format then it could. >> >> I had such a driver for Windows 9x which would display several >> folders and files for inserted AudioCD's: >> >> D: (cdrom) >> Stereo >> 22050 >> Track01.wav >> Track02.wav >> ... >> 44100 >> Track01.wav >> ... >> Mono >> 22050 >> Track01.wav >> ... >> 44100 >> Track01.wav >> ... > So you'd like the kernel to know about raw CD PCM and RIFF PCM format > and conversion? Great.. That's really Solarisish! If it could be done in userland, and still be accessible for most applications then that is great. If the user things that such a plugin is of value for him then I do not see why he should not be able to use it? Again these are example of stuff that could be done if there was a plugin/extensible interface. You make it sound like everything on the planet should be predefined and included in the kernel. This is not what I was saying or wanted. From what I understood from Hans, there will be a way to load plugins without having to recompile reiser4 module (or the kernel). This is what I'd like to see. Some things/plugins may even be partially user-space. ~S > Tonnerre
Re: silent semantic changes with reiser4
> Spam <[EMAIL PROTECTED]> writes: >> > Additionally, files-as-directores does not solve the problem of >> > "cp a b" losing named streams. There is curently no copyfile syscall >> > in the Linux kernel, "cp a b" essentially does "cat a >b". So unless >> > cp is modified we don't gain anything. If cp is modified to know >> > about named streams, it really does not matter if named streams are >> > accessed as file-as-directories, via openat(3) or via a shared library >> > with some other interface. >> >> One suggestion is missed. It is to provide system calls for copy. >> That would also solve the problem. Named streams and metas would >> then be handled correctly. It also allows further changes to >> filesystems without having to patch applications yet again. > But this still solves only part of the problem. A backup application > won't have any use for a copyfile syscall, it will need to be taught > about streams. Yes, but backup programs always needed to be taught about new features. Be it new type of files, attributes or meta-data. I think that teaching backup applications is far better than teaching every application. >> A copy system call would also be large beneficial for networked >> filesystems (NFS, Samba, etc) as data wouldn't have to be >> transferred over the network and back. > Definitely. >> Can we make a plugin infrastructure that will let user-space plugins >> to be loaded for certain directories or files? If we can, then it >> would present a much cleaner and easier way for the user to access >> data he wants. In this particular example it was a tar file. > In that case I'd argue that: > mount -t userfs -o driver=tarfs foo /tmp/foo > is a rather good kernel interface for plugins. userfs (or something > based on userfs) is the plugin API and tarfs is a plugin. :-) > To make this efficient, well have to allow non-root users to perform > the mount syscall (with the limitation that they can only mount on top > of directories they own and that the mounts have the nosuid and nodev > flags set). Yes, this seem to be one solution. It isn't very dynamic in usage though. You can't use this directly from applications wihout manually doing the mount. This is only a solution to browsing contents of files. It doesn't provide a solution for using meta-data streams or other things like this. ~S > /Christer
Re: silent semantic changes with reiser4
> Spam <[EMAIL PROTECTED]> wrote: >> One suggestion is missed. It is to provide system calls for copy. >> That would also solve the problem. > No, it would not. If you read the POSIX.1 specification for cp > carefully <http://www.unix.org/version3/online.html>, you will > notice that the process for copying a regular file is carefully > standardized. A POSIX.1-conforming cp implementation would not > be allowed to copy additional streams, unless either additional > options are given or the type of the file being copied is other > than S_IFREG. And cp is just one example of a standardized file > handling program. It would solve the problem in Linux. However, it may not be POSIX.1 compatible. On the other hand I read that NTFS 5.0 is POSIX.1 compliant - and Windows uses copy system call. NTFS also has streams support using special character in the file names to select the streams. Surely there must be a solution in Linux that will allow things like streams and meta-data(meta-streams) be visible to the user. ~S > Gunnar
Re: silent semantic changes with reiser4
> Hans Reiser <[EMAIL PROTECTED]> said: >> Horst von Brand wrote: >> >Hans Reiser <[EMAIL PROTECTED]> said: >> >>Horst von Brand wrote: >> >>>Spam <[EMAIL PROTECTED]> said: >> >>>>Christer Weinigel <[EMAIL PROTECTED]> said: > [...] >> >>>>>2. How do we want to expose named streams? >> >>>>> One suggestion is file-as-directory in some form. >> >>>Which is broken, as it forbids hard links to files. >> >>No, it forbids hard links to the directory aspect of the file-directory >> >>duality. >> >How do you distinguish a "hard link to the directory personality" from >> >"hard link to the file personality"? >> Put in (undoubtedly overly) simple terms, if you can do it to a file you >> can do it to the file personality, but if you currently can only do it >> to a directory and get an error from attempting it to a file then in the >> new scheme doing it to the hard link only gives the same error. > Let me sort this out: If it can't be done POSIXly to a directory, it can't > be done in Reiser4 to a file (which really is a directory too). So there > can be exactly _one_ hard link to a file. Way borken. But you can make a hard link to a file in reiser4, and you can access the metadata in both. I did this test: echo "m" > test ln test moo chmod +x moo test echo "0700" > test/metas/rwx dir moo test -rwx-- 2 root root 10 Sep 7 23:17 moo* -rwx-- 2 root root 10 Sep 7 23:17 test* echo "0777" > test/metas/rwx dir moo test -rwxrwxrwx 2 root root 10 Sep 7 23:17 moo* -rwxrwxrwx 2 root root 10 Sep 7 23:17 test* rm moo test mkdir test ln test moo ln: `test': hard link not allowed for directory If this is the intended behaviour I do not know, but it shows that hard links works as normal. You can also do linking to meta-data: echo moo > moo ln -s moo/metas/rwx test dir -rwxrwxrwx 1 rootroot 0 Sep 7 23:24 moo* lrwxrwxrwx 1 rootroot 13 Sep 7 23:25 test -> moo/metas/rwx echo 0700 > test dir moo -rwx-- 1 rootroot 0 Sep 7 23:24 moo* >> Or, we can ask Alexander to help us use his deadlock detection algorithm >> and try to do things right > Good luck with that one. I'd suspect if it can be made to work, it will > have _huge_ overhead, so much that it is useless. I'd love to be proven > wrong, but I won't hold my breath.
Repacker cause fs corruption:
I tried the resier4 repacker yesterday on a 2GB device that I use for /usr/portage in Gentoo. This morning I found these errors in the kernel log: ## Repacker: I am alive, pid = 32717 reiser4[k_reiser4_repac(32717)]: renew_sibling_link (fs/reiser4/tree_walk.c:441)[nikita-3532]: WARNING: Sibling nodes on the different levels: 1 != 1 reiser4 repacker: 659 formatted node(s) processed, 170984 unformatted node(s) processed, ret = -5 ## I did a fsck. This is the output from this: ## * fsck.reiser4 started at Tue Sep 14 10:30:09 2004 Reiser4 journal (journal40) on usr_portage.loop: 0 transactions replayed of the total 0 blocks. Reiser4 fs was detected on usr_portage.loop. Master super block (16): magic: ReIsEr4 blksize:4096 format: 0x0 (format40) uuid: fb81301f-8994-49db-a70a-02d1dcb1ad4c label: usr_portage Format super block (17): plugin: format40 description:Disk-format for reiser4, ver. 1.0.1 magic: ReIsEr40FoRmAt flushes:0 mkfs id:0x6624937f blocks: 524288 free blocks:359376 root block: 152472 tail policy:0x2 (smart) next oid: 0x3041e file count: 96887 tree height:4 key policy: LARGE CHECKING STORAGE TREE FSCK: Node (89186): The last key [15a4b:4(FB):6c657373746966:15a4f:45d] in the node is greater then the right delimiting key [15a4b:4(FB):4368616e67654c:15a50:5aa]. FSCK: Node (152380), item (74), unit (0): Points to the node [89186] with wrong delimiting keys. The whole subtree is skipped. Read nodes 23702 Nodes left in the tree 23701 Leaves of them 23321, Twigs of them 373 Nodes with wrong delimiting keys 1 Leaves of them 1, Twigs of them 0 Time interval: Tue Sep 14 10:30:11 2004 - Tue Sep 14 10:33:23 2004 CHECKING EXTENT REGIONS. Read twigs 373 Time interval: Tue Sep 14 10:33:24 2004 - Tue Sep 14 10:33:31 2004 Warn : Fatal corruptions were found. Semantic pass is skipped. * fsck.reiser4 finished at Tue Sep 14 10:33:31 2004 Closing fs...done 1 fatal corruptions were detected in FileSystem. Run with --build-fs option to fix them. ## Here follows the output from --build-fs: ## * fsck.reiser4 started at Tue Sep 14 10:35:37 2004 Reiser4 journal (journal40) on usr_portage.loop: 0 transactions replayed of the total 0 blocks. Reiser4 fs was detected on usr_portage.loop. Master super block (16): magic: ReIsEr4 blksize:4096 format: 0x0 (format40) uuid: fb81301f-8994-49db-a70a-02d1dcb1ad4c label: usr_portage Format super block (17): plugin: format40 description:Disk-format for reiser4, ver. 1.0.1 magic: ReIsEr40FoRmAt flushes:0 mkfs id:0x6624937f blocks: 524288 free blocks:359376 root block: 152472 tail policy:0x2 (smart) next oid: 0x3041e file count: 96887 tree height:4 key policy: LARGE CHECKING STORAGE TREE FSCK: Node (89186): The last key [15a4b:4(FB):6c657373746966:15a4f:45d] in the node is greater then the right delimiting key [15a4b:4(FB):4368616e67654c:15a50:5aa]. FSCK: Node (152380), item (74), unit (0): Points to the node [89186] with wrong delimiting keys. Removed, content will be inserted later item-by-item. Read nodes 23702 Nodes left in the tree 23701 Leaves of them 23321, Twigs of them 373 Nodes with wrong delimiting keys 1 Leaves of them 1, Twigs of them 0 Time interval: Tue Sep 14 10:35:39 2004 - Tue Sep 14 10:41:01 2004 CHECKING EXTENT REGIONS. Read twigs 373 Time interval: Tue Sep 14 10:41:15 2004 - Tue Sep 14 10:41:25 2004 LOOKING FOR UNCONNECTED NODES Read nodes 0 Good nodes 0 Leaves of them 0, Twigs of them 0 Time interval: Tue Sep 14 10:41:32 2004 - Tue Sep 14 10:41:32 2004 CHECKING EXTENT REGIONS. Read twigs 0 Time interval: Tue Sep 14 10:41:32 2004 - Tue Sep 14 10:41:32 2004 INSERTING UNCONNECTED NODES 1. Twigs: done 2. Twigs by item: done 3. Leaves: done 4. Leaves by item: done Twigs: read 0, inserted 0, by item 0, empty 0 Leaves: read 0, inserted 0, by item 0 Time interval: Tue Sep 14 10:41:38 2004 - Tue Sep 14 10:41:38 2004 CHECKING SEMANTIC TREE FSCK: No 'lost+found' entry found. Building a new object with the key 2a:0:. FSCK: Failed to recognize the plugin for the directory [2a:0:]. FSCK: Trying to recover the directory [2a:0:] with the default plugin--dir40. FSCK: The file [2a:0:] does not have a StatData item. Creating a new one. Plugin dir40. FSCK: Directory [2a:0:]: The e
Re: Repacker cause fs corruption:
> On Tue, Sep 14, 2004 at 10:59:08AM +0200, Spam wrote: >> >> I tried the resier4 repacker yesterday on a 2GB device that I use >> for /usr/portage in Gentoo. This morning I found these errors in the >> kernel log: > Thanks for the report. > How did you start the repacker? which direction did you run it in? The direction is "0". I started by issuing: echo 1 >/sys/fs/reiser4/loop0/repacker/start ~S >> >> ## >> >> Repacker: I am alive, pid = 32717 >> reiser4[k_reiser4_repac(32717)]: renew_sibling_link >> (fs/reiser4/tree_walk.c:441)[nikita-3532]: >> WARNING: Sibling nodes on the different levels: 1 != 1 >> reiser4 repacker: 659 formatted node(s) processed, 170984 >> unformatted node(s) processed, ret = -5 > We have similar bug report, but no information that the breakage was connected > to the repacker. >> >> ## >> >> I did a fsck. This is the output from this: >> >> ## >> >> * fsck.reiser4 started at Tue Sep 14 10:30:09 2004 >> Reiser4 journal (journal40) on usr_portage.loop: 0 transactions replayed of the >> total 0 blocks. >> Reiser4 fs was detected on usr_portage.loop. >> Master super block (16): >> magic: ReIsEr4 >> blksize:4096 >> format: 0x0 (format40) >> uuid: fb81301f-8994-49db-a70a-02d1dcb1ad4c >> label: usr_portage >> >> Format super block (17): >> plugin: format40 >> description:Disk-format for reiser4, ver. 1.0.1 >> magic: ReIsEr40FoRmAt >> flushes:0 >> mkfs id:0x6624937f >> blocks: 524288 >> free blocks:359376 >> root block: 152472 >> tail policy:0x2 (smart) >> next oid: 0x3041e >> file count: 96887 >> tree height:4 >> key policy: LARGE >> >> >> CHECKING STORAGE TREE >> FSCK: Node (89186): The last key >> [15a4b:4(FB):6c657373746966:15a4f:45d] in the node is greater then >> the right delimiting key [15a4b:4(FB):4368616e67654c:15a50:5aa]. >> FSCK: Node (152380), item (74), unit (0): Points to the node >> [89186] with wrong delimiting keys. The whole subtree is skipped. >> >> Read nodes 23702 >> Nodes left in the tree 23701 >> Leaves of them 23321, Twigs of them 373 >> Nodes with wrong delimiting keys 1 >> Leaves of them 1, Twigs of them 0 >> Time interval: Tue Sep 14 10:30:11 2004 - Tue Sep 14 10:33:23 2004 >> CHECKING EXTENT REGIONS. >> Read twigs 373 >> Time interval: Tue Sep 14 10:33:24 2004 - Tue Sep 14 10:33:31 2004 >> Warn : Fatal corruptions were found. Semantic pass is skipped. >> * fsck.reiser4 finished at Tue Sep 14 10:33:31 2004 >> Closing fs...done >> >> 1 fatal corruptions were detected in FileSystem. Run with --build-fs option to fix >> them. >> >> ## >> >> Here follows the output from --build-fs: >> >> ## >> >> * fsck.reiser4 started at Tue Sep 14 10:35:37 2004 >> Reiser4 journal (journal40) on usr_portage.loop: 0 transactions replayed of the >> total 0 blocks. >> Reiser4 fs was detected on usr_portage.loop. >> Master super block (16): >> magic: ReIsEr4 >> blksize:4096 >> format: 0x0 (format40) >> uuid: fb81301f-8994-49db-a70a-02d1dcb1ad4c >> label: usr_portage >> >> Format super block (17): >> plugin: format40 >> description:Disk-format for reiser4, ver. 1.0.1 >> magic: ReIsEr40FoRmAt >> flushes:0 >> mkfs id:0x6624937f >> blocks: 524288 >> free blocks:359376 >> root block: 152472 >> tail policy:0x2 (smart) >> next oid: 0x3041e >> file count: 96887 >> tree height:4 >> key policy: LARGE >> >> CHECKING STORAGE TREE >> FSCK: Node (89186): The last key >> [15a4b:4(FB):6c657373746966:15a4f:45d] in the node is greater then >> the right delimiting key [15a4b:4(FB):4368616e67654c:15a50:5aa]. >> FSCK: Node (152380), item (74), unit (0): Points to the node >> [89186] with wrong delimiting keys. Removed, content will be >> inserted later item-by-item. >>
Re: Repacker cause fs corruption:
> On Tue, Sep 14, 2004 at 12:08:31PM +0200, Spam wrote: >> >> > On Tue, Sep 14, 2004 at 10:59:08AM +0200, Spam wrote: >> >> >> >> I tried the resier4 repacker yesterday on a 2GB device that I use >> >> for /usr/portage in Gentoo. This morning I found these errors in the >> >> kernel log: >> >> > Thanks for the report. >> >> > How did you start the repacker? which direction did you run it in? >> >> The direction is "0". I started by issuing: >> echo 1 >/sys/fs/reiser4/loop0/repacker/start > It looks like the repacker triggers a bug in reiser4. the forward run does not > modify reiser4 nodes by itself. Anything I can do for testing? The data I have on this device isn't critical so I can test more if needed. ~S >> ~S >> >> >> >> >> ## >> >> >> >> Repacker: I am alive, pid = 32717 >> >> reiser4[k_reiser4_repac(32717)]: renew_sibling_link >> >> (fs/reiser4/tree_walk.c:441)[nikita-3532]: >> >> WARNING: Sibling nodes on the different levels: 1 != 1 >> >> reiser4 repacker: 659 formatted node(s) processed, 170984 >> >> unformatted node(s) processed, ret = -5 >>
Repacker output
I have a suggestion about the repacker output. Currently the repacker shows something like this: Repacker: I am alive, pid = 7716 reiser4 repacker: 4395 formatted node(s) processed, 16858 unformatted node(s) processed, ret = 0 What I'd like to see is that both lines contained which device that was processed. The second line should contain the pid too. This would make it easier to read and follow what is happening and where. ~S
Re: [patch] Repacker stats
Hello, I was interested in testing the repacker statistics tool that Piotr Neuman wrote (and others later added to). But from what I can see the repacker is disabled in 2.6.9-rc4-mm1, because of this patch: http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.9-rc4/2.6.9-rc4-mm1/broken-out/reiser4-disable-repacker.patch Is there something wrong with the repacker that is has to be disabled? ~S > Hi > This is my second attempt at repacker statistics, this time based on comments > from Alex Zarochentsev (thanks ;-). > This patch allows reading of repacker stats while it is working. It creates > two files in repacker sysfs hierarchy: formatted_nodes and unformatted_nodes, > which correspond to the stats that repacker prints after it has finished. > Download: > http://kem.p.lodz.pl/~peter/reiser4_repacker_stats-2.patch > Regards > Piotr Neuman ´ --
Re: [patch] Repacker stats
>> Christian Mayrhuber wrote:
>> On Friday 22 October 2004 19:38, Spam wrote:
>>
>>
>>> Hello,
>>>
>>> I was interested in testing the repacker statistics tool that Piotr
>>> Neuman wrote (and others later added to). But from what I can see
>>> the repacker is disabled in 2.6.9-rc4-mm1, because of this patch:
>>>
>>>
>>>
>>>
>>>
>>> http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.9-rc4/2.6.9-rc4-mm1/broken-out/reiser4-disable-repacker.patch
>>>
>>>
>>> Is there something wrong with the repacker that is has to be
>>> disabled?
>>>
>>> ~S
>>>
>>>
>>>
>>
>>Currently the repacker is known to corrupt the filesystem, that's
>>why it's disabled, I guess.
>>
>>
>>
> There are two reasons:
> 1) functionality that is not stable should be turned off until it is
> stable ("" and repacker both).
Turned off, or perhaps better to be marked as unstable or dangerous.
If it isn't available, no one will test and report problems.
> 2) we aren't making payroll and need something nonessential to bundle
> with support if we want people to buy support so that I am to be able to
> quit my day job and pay people. There is nothing less essential yet
> still salable than a repacker/resizer. ext3 had a proprietary resizer,
> so it is not an innovation by us.
This I do understand very well. It brings my thoughts to Diskepper
from Executive Software (www.executive.com). Microsoft included a
basic version of their defrag software into Windows 2000. Users see
the functionality, but they cannot do much with it other than manual
defrag. All advanced features usually needed for corporate world is
only available if you pay to "upgrade" to full version of Diskeeper.
My suggestion is to leave a basic version of the repacker for normal
users, and to offer a more advanced and manageable version for
corporate network and server use? More users will see the
availability of defrag with Reiser4 and choose it above another fs.
In the end, probably, more people/companies would buy the extended
features?
> reiser4 development is going at greatly reduced speed due to our needing
> to take contracts involving other kernel work rather than working on
> reiser4. I am basically unable to spend time reviewing reiser4 code
> because I have this day job fixing things not related to reiser4 in the
> kernel. I am glad to have the day job, but getting money from our work
> on reiser4 would be better.
> I have also asked both the EU and the US for money, wish me luck. I'd
> like to start working on the enhanced semantics for ReiserFS, but I need
> money to do that. We are losing momentum vs. Apple and MS. The resizer
> might bring in the money for enhanced semantics work.
I wish you luck. It would be a shame if you would have to quit the
work on your filesystems.
~S
--
Re: when will it end?
> | current repacker code state is 'unsupported', it is even removed from the > | latest -mm kernels. Namesys plans are to make the repacker proprietary. > Really? > I sort of thought the priority was to get into the mainstream kernel, so > this seems an odd move. Why do something that decreases your benchmark? > ~ And how much would people be willing to pay you when most Unix > filesystems are rarely (if ever) repacked and Windows (even DOS, I > think) comes with a free repacker of its own? Windows 2000 and later comes with a lite version of Diskeeper from Executive Software. You can buy a much more advanced version of Diskeeper to use in your corporate network, on your servers, or your desktop. It offers things like scheduling, continuos background mode and other things. http://www.executive.com/defrag/defrag.asp I think a free, limited/basic version of the repacker may be beneficial to Namesys. It would increase awareness of the feature and thus increase demand for a more advanced version that can work in corporate environments? Many/Most Linux users believe that repacker/defrag is never needed and this is the reason why there are no such tools available. > The current state of the repacker suggests that the tradeoff is not > worth it. If it takes longer than 2 days for an initial repack of a > filesystem like mine, the total amount of time I have to spend waiting > for the system while it's dog-slow and repacking is likely more than the > time I'd save by having it be slightly faster the rest of the time, or > even the time I'd have to work at minimum wage to buy more storage to > cover space the repacker could save me. 2 days? I ran the repacker within hours on a 80GB disk. Perhaps your DMA and/or 32bit I/O is disabled? hdparm -c3 -d1 -A1 will enabled those and disk read-ahead. hdparm -W1 will enable disk write cache. Not recommended if you encounter power loss often as journals and data may be out of sync as the contents of the write cache is lost when a power failure occurs. It will speed up things a little though. ~S > Compare this to Norton SpeedDisk, which takes about half an hour to > defrag a 20-40 gig FAT32/NTFS drive -- the first time. After that, if > run daily, it takes 10-15 mins. > But also, is chunk_size measured in megabytes? (mine was 512) Does it > mean what I think it means? > If the repacker is still free next time I go on vacation, I'll try again > - -- see if a week or two will straighten things out. > -BEGIN PGP SIGNATURE- > Version: GnuPG v1.2.6 (GNU/Linux) > Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org > iQIVAwUBQX88e3gHNmZLgCUhAQIXiQ//fnR727WKFmhkiP/9w02YhANuImptkokC > KjLVJ8LkaOtz7OHecVqrIWNP2V13FDpp/Tn5Nf+h9iDeem6wF9rw4kXM6QX2NNFk > AD5G3dYKLEMAPDkSAXYVqMv6ykI9CH61ELNHBDatfj3gBjz0uLB0kIStI6lM1nNq > Pem2FmqWcM+3SVYEvmFuN4/oBJjwWyNav7CAWt58tvBqY703BNQmsLwEQBrcMAo8 > ifM07H0JRXT9qnKYCNKAbmUhInOAr8IUIN36OUTd/Wp35K3/1C6LicGyG6wDy51b > aT0y4eZd0hTRZVkG5PCJ3Naz1GA56BFIts8Kvi9JtBJeza4/y0xZAtycZjeVr7Hu > ufMjZVWgZTswiOFvNXAVFtJ6RnVDVELnKJtHI6EkFRuyJdgQKcz1oII2f//VORU0 > IST7icAw1ZSSfF08dSnhpNCFVf4ZuSAz/V6VrgFup5+cerg23DGo3bZ+K5I49//3 > hOVNC2wedl8hziNt2kKd7yDNU7WpaPR3PLOusrPsJi25YRAmIhXjDanwOt3XFA3c > jhlTNTApj25FqphP1oCV9PNlUR0xYJXVHCnz8Aiu9ED0Yir8ECyY8Wm9Zgs1haec > dPelzN5KioLnE5+0dLvDaYcMvewA7Vy8NwJ5z/KfcnSpD7dvH90wDE1KNbpkjZxt > nH5vzVbtgEU= > =oGZA > -END PGP SIGNATURE- ´ --
Re: when will it end?
> Spam wrote: >> >> Windows 2000 and later comes with a lite version of Diskeeper from >> Executive Software. >> > MS charges for the OS, so that business model works for them. Nobody > will buy a heavy resizer from us if there is a lite one. That would be > as likely to happen as their buying a support contract. The lite > software is reiser4 without a resizer. Yes, you may be right. I was just voicing some options. Perhaps enough marketing is enough to make users learn about the repacker and its benefits. ~S > Such is my feel of the market. > Hans ´ --
Re: Reiser4 kernel modules
> --- Vladimir Saveliev <[EMAIL PROTECTED]> wrote: >> reiser4 needs some changes in kernel code. >> ftp://ftp.namesys.com/pub/reiser4-for-2.6.9 contains all things which >> are necessary to run reiser4 on 2.6.9 > Right now I go there and find only two files: > ncftp /pub/reiser4-for-2.6.9 > ls -al > -rw-r--r--1 00 0 Nov 2 18:40 > reiser4-for-2.6.9-TEMPORARY_NOT_AVAILABLE > -rw-r--r--1 00 454 Nov 2 15:24 > reiser4-on-2.6.9-HOWTO > I also tried downloading the (huge) 2.6.9-mm patch from ftp.kernel.org. > After I had applied it, to a vanilla 2.6.9 source tree, I couldn't > find Reiser4 anywhere under filesystems when doing a make config. You need to DISABLE 8K stacks for the option of Reiser4 to be visible. > I > built it anyway with ReiserFS enabled, in case you didn't make it a > separate option but just merged it into the existing option, but I > cannot mount a Reiser4 partition. So where should I expect to find > Reiser4 in the config menus? ~S
Re: Reiser4 kernel modules
>> --- Vladimir Saveliev <[EMAIL PROTECTED]> wrote: >>> reiser4 needs some changes in kernel code. >>> ftp://ftp.namesys.com/pub/reiser4-for-2.6.9 contains all things which >>> are necessary to run reiser4 on 2.6.9 >> Right now I go there and find only two files: >> ncftp /pub/reiser4-for-2.6.9 > ls -al >> -rw-r--r--1 00 0 Nov 2 18:40 >> reiser4-for-2.6.9-TEMPORARY_NOT_AVAILABLE >> -rw-r--r--1 00 454 Nov 2 15:24 >> reiser4-on-2.6.9-HOWTO >> I also tried downloading the (huge) 2.6.9-mm patch from ftp.kernel.org. >> After I had applied it, to a vanilla 2.6.9 source tree, I couldn't >> find Reiser4 anywhere under filesystems when doing a make config. > You need to DISABLE 8K stacks for the option of Reiser4 to be > visible. Ehum. I am a little tired. I meant disable 4K stacks :) >> I >> built it anyway with ReiserFS enabled, in case you didn't make it a >> separate option but just merged it into the existing option, but I >> cannot mount a Reiser4 partition. So where should I expect to find >> Reiser4 in the config menus? > ~S ´ --
Re: reiser4 and reiserfs co-existence?
> Funny thing is I did what you suggested, enabling all the debugging, > but it still freezes. Not immediately, but it still hangs. First I > get a segfault on any command I type into the console. After 10 secs > or so of this, It will freeze up. How in the world did you save that > stack trace. I've seen the same thing on my machine, but can't figure > out how to save it, because my filesystem doesn't respond. Remote shell? Conosle logging? Network logging? All three possible with recent kernels. ¨S > On Tue, 9 Nov 2004 12:47:07 -0500, Michael Barry <[EMAIL PROTECTED]> wrote: >> This is the exact same problem I am having. I just bumped the thread >> actually. If you enable debig mode instead of your system freezing you >> will get a stack trace. That trace is available in my thread labeled >> possible bug: clog 0. >> >> Mike >> >> >> >> >> On Tue, 9 Nov 2004 10:41:25 -0500, Jeremy West <[EMAIL PROTECTED]> wrote: >> > Yes, I have a source code tarball that can't be built correctly. You >> > can get this file from other places, but it might be best to test >> > exactly what I'm using. I'm building php, and applying this patch. >> > >> > www.jeremysplace.org/php-4.3.9.tar.bz2 >> > http://www.jeremysplace.org/php-4.3.6-includepath.diff >> > >> > To answer Markus's question. I'm not having any bus errors yet. >> > At least I hope I don't get any. I like to deal with one problem at a >> > time. I will reboot my machine into a LiveCD, and debug some reiser4 >> > for you. How's best to do this anyway. The debug script that you >> > referenced won't find anything unless something odd happens, right? >> > What does the script do? I guess this is a stupid question, huh? >> > >> > >> > >> > On Tue, 09 Nov 2004 13:55:54 +0300, Vladimir Saveliev <[EMAIL PROTECTED]> >> > wrote: >> > > Hello >> > > >> > > On Tue, 2004-11-09 at 10:16, Jeremy West wrote: >> > > > Ok. I think I may have found a glitch in reiser4. It seems that my >> > > > freezing coincidentally happened every time I tried installing php. >> > > > Each time my system froze, the php build would crap out at the same >> > > > point. Basically the bulid would say that it was running the >> > > > pear-installer program, then give me a segmentation fault. >> > > >> > > Do i understand correctly that you have a source code tar ball which can >> > > not be built in reiser4 filesystem? Where can i find it? >> > > >> > > >> > > >> > > > >> > > > So ... it seems to me that reiser4 is having some issues handling >> > > > segfault events. I'm not sure how to tell what caused the segfault. >> > > > I don't think it's memory, because it happens in the same spot of the >> > > > build every time. I've done this 4 times now. That is not random. >> > > > >> > > > >> > > > On Mon, 08 Nov 2004 18:09:15 +0100, Redeeman <[EMAIL PROTECTED]> wrote: >> > > > > On Mon, 2004-11-08 at 11:55 -0500, Jeremy West wrote: >> > > > > >> > > > > >> > > > > > HI, >> > > > > > >> > > > > > I'm new to the list. I've taken the plunge and tasted the speed of >> > > > > > reiser4. Excellent work. I think I've stumbled upon a bug >> > > > > > though. I >> > > > > > have another partition formatted as reiserfs 3.6. When I mount >> > > > > > this >> > > > > > partition and listen to music from it, my reiser4 / partition stops >> > > > > > responding. I mean every command I issue to the kernel results in >> > > > > > a >> > > > > > segmentation fault. Has anyone else had this problem? >> > > > > > >> > > > > > I'm using redeeman-sources, based on mm-sources patch set. >> > > > > how old would this happen to be? i mean.. give a uname -a >> > > > > -- >> > > > > Redeeman <[EMAIL PROTECTED]> >> > > > > >> > > > > >> > > > >> > > >> > > >> > >> > -- >> > Jeremy West >> > //--- >> > "I had a life once... now I have a computer and DSL" >> > >> ´ --
Re: reiser4 for windows
> On Mon, 2004-12-06 at 22:07 -0800, Jiri Klouda wrote: >> > Also, is it a given that reiser4 for windows would work without help >> > from MS? I've just never seen a third-party filesystem driver for >> > windows. You'd think that at least one other filesystem, one of the >> > Linux/BSD/etc ones, would have done this, if it was feasable. Aren't there? Isn't there various compression systems, not to mention PGP disk? Maybe it is not the same thing... >> >> You might be onto something here. I am pretty sure my company would >> pay quite a lot for a working, fast and elegant implementation of >> symbolic links on windows. Either as a new filesystem or as an addition >> onto ntfs. We currently use one such addition and not very usable and >> uses a catalog of symlinks that can easily get corrupted. Yes, NTFS only has hard links, not symlinks =(. I have myself been wanting to use symlinks a lot. >> >> Having reiser4 on Windows or even ext2, would be a huge thing. > a real implementation of ext2 for windows exists.. i had it once in > vmware.. but i cant remember the name >> >> > All this leaves me with the distinct impression that it'd be cheaper to >> > buy some gigabit ethernet (or fibre) and a Linux CIFS/Samba fileserver >> > using reiser4. Steam and others refuse to install on network drives, >> > Windows probably will not boot off a network drive, but I imagine that >> > fixing these would be easier than porting a filesystem. Will disk access over LAN really be anything close to as fast as a real local harddisk, especially if you have to run over Samba? Oh, just noticed you said this in the next paragraph =). Windows can be run from a network drive. Steam, on the other hand prevents network drives for the same reason it has CD checks. It is all about preventing users from running copies. >> >> Unfortunatelly even with very fast ethernet connections and network >> appliance with CIFS access, we still don't get the performance of a >> local filesystem. Plus, you don't want to really give write access >> over network, that slows down anything when you get into hundreds of >> clients. And we really cannot pay for so many network applicances to >> make this scalable proposition. >> >> I wish we could just drop Windows as a platform, but as long as there >> are customers, there will be need to support them as well... :( It is easy to say these things. But really, for lots of applications and uses there is only MacOSX that can compete. Linux has yet a far way to go when it comes to normal desktop and workstation usage. Hardcore and relatively skillful users have no problems with Linux as a desktop environment, but the "gray mass" does. KDE and Gnome are improving fast though =). Anyway, that is another discussion not really for this forum. ~S >> >> -Jiri >> ´ --
Re: Need Warm Fuzzies, ReiserFS (3)
> On Monday 20 December 2004 16:38, Tom Vier wrote: >> On Mon, Dec 20, 2004 at 01:31:25PM +0100, Christian Mayrhuber wrote: >> > The barrier mount option should provide protection against a corrupted >> > journal during power failure for drives with write caching enabled. >> > (Mostly IDE) >> >> There's a mount option for write barriers? It should be enabled by default. >> Otherwise, that's asking for tons of users with corruption. > http://www.kernel.org/pub/linux/kernel/v2.6/ChangeLog-2.6.9 > Before disk barriers you had to disable harddisk write caching to prevent > journal corruption. This is what nearly no one did and blamed the > filesystems instead. > Disk barriers are only in since 2.6.9, so maybe these get turned on per > default after some more testing. That did happen to the data=ordered journal > mode. What happen with the performance when these barriers are active? Is it only during power failure the data in the write cache is lost? Also, does anyone know if the data in the disk write cache is written out if the system crashes/freezes. Power failure is not a very common occurrence, but crashes are much more so. Is it possible to detect if filesystem was unmounted due to power failure or due to a kernel crash? ~S --
Re: Need Warm Fuzzies, ReiserFS (3)
> On Monday 20 December 2004 17:32, Spam wrote: >> What happen with the performance when these barriers are active? > It's faster than with disk writecache off. > I didn't benchmark barriers=flush and writecache on. >> >> Is it only during power failure the data in the write cache is lost? > No. See http://support.microsoft.com/kb/153296/EN-US/ I think the answer to my question was yes. That link is about disk cache not flushed properly before Windows shuts the computer off. >> >> Also, does anyone know if the data in the disk write cache is >> written out if the system crashes/freezes. Power failure is not a >> very common occurrence, but crashes are much more so. > The cache should stay intact as long the harddisk is supplied with power. So, the question remains. Will the drive flush its cache if it still has power or does it need to receive some kind of flush command from the OS? If not, then a OS crash will not harm the integrity of the data in the cache and the disk will flush it out. >> >> Is it possible to detect if filesystem was unmounted due to power >> failure or due to a kernel crash? > Kernel panics don't tend to reset the machine but output a dump to the > console /syslog via network and halt the machine. > If you have a UPS supplier supporting linux you will receive a power failure > notification event to take proper action. > For example: http://www2.apcupsd.com/ No, I meant through software. So that fsck could detect if system lost power and therefore make a more thorough test. If you have UPS then write-cache should never be dangerous? ~S --
Re: Need Warm Fuzzies, ReiserFS (3)
> On Mon, Dec 20, 2004 at 09:31:10PM +0100, Spam wrote: >> If you have UPS then write-cache should never be dangerous? > It's not AS dangerous. You could still lose a psu or someone could trip over > the power cord from the computer to the ups. Indeed. For a server then I would not run with write-cache enabled anyway. But for a home desktop etc I would run with it enabled. Most people with Win2000/XP have write-cache enabled. Yet there are rarely problems due to power failures? ´ --
Re: Congratulations! we have got hash function screwed up
> Esben Stien wrote (ao): >> I really don't like that there is no undelete feature in reiserfs - >> it's not planned for reiserfs-4 either. I see desperate users all the >> time trying to get back what they mistakenly removed. > If you 'see desperate users all the time' you might be amoung the wrong > people ;-) > Maybe you can clue them in the wonderful world of making backups? This is the standard Linux comment... But in reality, it just a way to say you do not know, or that linux lacks an (in this users opinion) important feature. In any case. Undelete has been since ages on many platforms. It IS a useful feature. Accidents CAN happen for many reasons and in some cases you may need to recover data. Besides, a deletion does not fully remove the data, but just unlinks it. In Reiser where there is tailing etc for small files this can be a problem. Either the little file might not be able to be recovered (shouldn't the data still exist, even if it is tailed), or the user need to use a non-tailing policy? > Next time it is not their mistake, but instead a broken harddisk. > Undelete wont save them then. Indeed. Backups are important. But the backup does not recover the last minute changes. They may be a day or a week old at best - even at larger sites! > Or they just edited it instead of rm. ´ well, overwritten data is not so easy to get back. But from what I understand in Linux, is that many applications actually write another file and then unlinks the old file? If that is the case then it may even be possible to get back some overwritten files! ~S --
Re: Congratulations! we have got hash function screwed up
> Spam <[EMAIL PROTECTED]> writes: >> In any case. Undelete has been since ages on many platforms. It IS a >> useful feature. Accidents CAN happen for many reasons and in some >> cases you may need to recover data. >> >> Besides, a deletion does not fully remove the data, but just unlinks >> it. In Reiser where there is tailing etc for small files this can be >> a problem. Either the little file might not be able to be recovered >> (shouldn't the data still exist, even if it is tailed), or the user >> need to use a non-tailing policy? > A working undelete can either hog disk space or die the moment some > large write comes in. And if you're at that point, make it a versioning > file system - but then don't complain about space efficiency. Yes, When data is overwritten then it is overwritten. The longer the user waits to try to recover data, the more risk of this happening. Undelete has existed a long time and people know it is not a foolproof thing. I do not think anyone asked for automatic backup features, but just tools that can try to recover accidental deletions. >> well, overwritten data is not so easy to get back. But from what I >> understand in Linux, is that many applications actually write >> another file and then unlinks the old file? If that is the case then >> it may even be possible to get back some overwritten files! > I see enough applications to just overwrite an output file. Yes, This was an example only. In any case. If there were tools that could scan and recover, even partly, deleted files then I would welcome them. I am sure lots of other people do too. It is very easy to say you need backups of your data, that you need versioning filesystems etc. But not all of this is possible for everyone. Just take a laptop as example. Making backups is not so easy to do frequently - especially not when traveling. Sure, if you run in a corporate environment you can do shadow copying or use other versioning systems and mount that over the network. But for the normal "home" or "small business" users this is not really what you can expect... > This whole discussion doesn't belong here until someone talks about > implementing a whole versioning system for reiser4. I think someone said they wanted undelete recovery features in reiser4 - which was what started this discussion? ´ --
nvram to protect fs during power loss?
There have been some discussions about recovery abilities during power loss if write cache is enabled. Some recovery tool I saw once used NVRAM to store progress info so that if you had a power loss it would be able to resume. Perhaps it would be possible for Reiser4 to store some info, like time indexes etc in NVRAMwhen it send sync commands to the disk. This way it might be possible to avoid corruptions by simply verifying (fsck) the data stored after that time index etc? I have no idea how often NVRAM can be written to before it goes bad. Is there a limit? ~S --
Re: nvram to protect fs during power loss?
Yes, I know that good hardware already do use battery backups and similar. But I was talking about normal consumer hardware; desktops, laptops, etc. I do not know how the internals of reiser4 works, but one of the problems said to be that it cannot be certain about the order and integrity of the data if a power failure occur. But if reiser4 stored some kind of index number, key etc, on regular intervals then that could certainly help in recovering the filesystem to a working state. As it is now you can end up with a severely broken filesystem that doesn't even mount if you have a power failure. IMO a filesystem should do its best to protect the data on it. I thought that perhaps using NVRAM could aid in the recovery after the powerloss, instead of tossing a user "bad error, use --build-sb to fix" during next boot. ~S > On Fri, Dec 31, 2004 at 12:40:08AM +0100, Spam wrote: >> >> There have been some discussions about recovery abilities during >> power loss if write cache is enabled. Some recovery tool I saw once >> used NVRAM to store progress info so that if you had a power loss it >> would be able to resume. > Typically specialized hardware, typically used for NFS (which is stateless) > doing > "caching" of the writes to enhance performance. >> Perhaps it would be possible for Reiser4 to store some info, like >> time indexes etc in NVRAMwhen it send sync commands to the disk. >> This way it might be possible to avoid corruptions by simply >> verifying (fsck) the data stored after that time index etc? > The way the journals/logs does?? ´ --
Re: nvram to protect fs during power loss?
> Why not just purchase a UPS and place it on your system? Seems like > it be a lot cheaper. That way the system can safely shut down in case > of a power outage? How can buying additional hardware be cheaper? Do you expect most users, including Laptop users, to use UPS? > On Fri, 31 Dec 2004 18:46:20 +0200, Hendrik Visage > <[EMAIL PROTECTED]> wrote: >> On Fri, Dec 31, 2004 at 01:45:32PM +0100, Spam wrote: >> > >> > Yes, I know that good hardware already do use battery backups and >> > similar. But I was talking about normal consumer hardware; desktops, >> > laptops, etc. >> >> Okay, if you want to use the NVRAM on a desktop, it's wayy too little >> to even consider. You'll need NVRAM (battery backed up etc.) in the >> order of Megabytes to make it of any use Spam. >> I thought NVRAM already was battery backed up or was as flash? In any case. It was said on this mailing list that because of the write cache we do not know if the journal was written before the actual data. With a simple pointer we can know the latest journal entry that was surely saved before the power loss (you can update the NVRAM when you tell the disk to sync etc). When you reboot the the fsck tools can scan all the journal entries that occurred after to verify that the data is on disk or not. This must be much faster than doing a full scan of the entire filesystem? How can disk require several megabytes? I am not talking about storing the journals in the NVRAM. ~S --
Re: Congratulations! we have got hash function screwed up
generic bug in handling hash collisions? >>>Tea hash is designed to be more resistant. >>> >>> >> >>As the example posted shows, tea doesn't look better, it generates >>nicely-looking collisions, too. >> >> > You mean, in practice you hit them, or with an artificially generated > set of filenames intended to cause collisions you get those collisions? Excuse me, but do you mean that there are undocumented limits on what files can be named to, and how many files with similar or random names in a ReiferFS volume? This sounds bad...
Re: Congratulations! we have got hash function screwed up
> On Thu, Jan 06, 2005 at 05:13:23PM +0100, Spam wrote: >> >>>>generic bug in handling hash collisions? >> >>>> >> >>>> >> >>>Tea hash is designed to be more resistant. >> >>> >> >>> >> >> >> >>As the example posted shows, tea doesn't look better, it generates >> >>nicely-looking collisions, too. >> >> >> >> >> > You mean, in practice you hit them, or with an artificially generated >> > set of filenames intended to cause collisions you get those collisions? >> >> Excuse me, but do you mean that there are undocumented limits on what >> files can be named to, and how many files with similar or random >> names in a ReiferFS volume? > No, I'd say it's pretty well documented that reiserfs fails under > certain hash collision conditions instead of continueing to work > (albeit more slowly). > The nature of the hash collisions must be pretty obvious if a shell > script can be written to demonstrate the problem. >> >> This sounds bad... > It's a risk assessment. What are the odds of your normal data sets > hitting the bug or of someone with malicious intent introducing > a demonstration program vs the performance hit of a filesystem > without the problem. How can I assess the risk, if I do not know how to produce the bugs? You say certain conditions. But from what I read earlier in the thread, a directory with a fonts in them.? > All filesystems will fail or suffer degraded performance under > certain conditions, you need to determine what conditions are acceptable > for your data. Slow can be acceptable. But failing? No, a filesystem should not fail. --
Re: rebuild-tree working for more then 3 days- what to do?
> Hello > On Fri, 2005-01-07 at 15:11, Shoshannah Forbes wrote: >> On 07/01/2005, at 12:20, Vladimir Saveliev wrote: >> >> > I would move the problem hard disk to mashine which has (or can >> > download) latest reiserfsck >> > (http://thebsh.namesys.com/pub/reiserfsprogs/reiserfsprogs >> > -3.6.19.tar.gz), >> > make sure that it performs I/O of that disk decently (hdparm -t >> > /dev/problem-disk) and run reiserfsck --rebuild-tree for broken >> > filesystem there. >> >> Thanks for the fast reply. >> The only other machine I have here is a Mac running OSX. Would it even >> be possible to do this from a Mac? >> If not, can anything be done from a LiveCD? > When you boot off liveCD - do you have a network and place where > reiserfsprogs can be put and compiled besides that 60gb device with > broken reiserfs? > Then boot liveCD, download reiserfsprogs and fsck. There are liveCD's with reiser4 and reiser4progs. For example: http://lxnay.dnsalias.org/livecd-iso/index.php ~S
Re: shrink reiserfs
> On Sun, Jan 23, 2005 at 10:49:56AM -0700, Jake Hawkes wrote: >> Alex Zarochentsev said: >> > Hello, >> > >> > On Sun, Jan 23, 2005 at 08:48:14AM -0700, Jake Hawkes wrote: >> >> Hi. >> >> >> >> resize_reiserfs says that there are already too many allocated blocks. >> > >> > which resize_reiserfs command produces the error message? >> >> resize_reiserfs -v -s -1G /dev/hda4 >> >> and when that failed, I tried >> >> resize_reiserfs -v -s -1M /dev/hda4 > and it failed too? > which reiserfsprogs version are you using? It is possible > that this bug is fixed already in the latest release of > reiserfsprogs (3.6.19). Just a side-note. After upgrading to latest reiserfsprogs, then do a fsck before attempting to resize the filesystem. ~S >> >> >> [EMAIL PROTECTED] jake $ df -h >> >> FilesystemSize Used Avail Use% Mounted on >> >> /dev/hda4 38G 7.1G 31G 20% / >> >> /dev/hda2 16G 15G 1.6G 91% /mnt/xp >> >> none 506M 0 506M 0% /dev/shm >> >> >> >> >> >> so how can this be possible if it is only 20% used? >> > >> > the shrinker can't make the fs less than 7.1G in size. >> >> >> I had to read that af ew times before I realised you were talking about my >> installation and not a general reiserfs issue :D >> >> given the command above then, what can I do?
Where to get Reiser 4 Source -Class assignment
Trying to get a look at the Reiser 4 sourcecode for a class assignment. We are interested in the Tree Structure, Tree Balancing, and Reiser's use of compression. However, the how-to, namesys.com for d/ling the code using BitKeeper is outdated according to the BitKeeper people. Can someone tell me how to get the Reiser4 source?
Re: Quicker alternative to "find /"?
> Am Sonntag, den 15.08.2004, 23:16 +0200 schrieb Felix E. Klee: >> I'd like to store the directory structure of a partition formatted as >> ReiserFS into a file. Currently, I use >> >> find / >file >> >> This process takes approximately 5 minutes (the result is 26MB of >> data). Are there any alternative *quicker* ways to do this? > The main problem is that find only uses one thread. This thread only > reads one directory at once and as a result of that you'll get a lot of > seeks. I am confused about this in general with most filesystems. I thought that all filenames/foldernames etc were stored in one place and not spread out over the intire filesystem. It seem to me very strange that things like find/ls -R etc takes so so long just read/list files like this on any modern filesystem. > This can usually be improved *a lot* by doing a massively multi-threaded > search with a lot of threads trying to read a lot of directories at > once. The disk io scheduler will then linearize all the outstanding read > requests. > I've done something similar to speed up a diff -r using a shell script > (not for find but for reading the file content that should be compared).
