Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Hi! Most users not only cannot patch a kernel, they don't know what a patch is. It most certainly does. obviously you can provide complete kernels, including precompiled ones. Most distros have a yum or apt or similar tool to suck down packages, it's trivial for users to add a site to that, so you could provide packages if you want and make it easy for them. What's more, many distros patch their kernels extensively. They listen to their users, too. So if there are a lot of users wanting this to be in the kernel, let them complain -- loudly -- to their distro to patch for Reiser4. This is not true any more. SUSE (and RedHat) really try not to add patches to their kernels unless patch is obvious bugfix or merged to some later mainline. Yeah, we do exceptions, but getting exception is *way* harder than it used to be. Pavel -- Thanks for all the (sleeping) penguins.
ext3 vs reiserfs speed (was Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion))
Hi! Using guilt as an argument in a technical discussion is a flashing red sign that says I have no technical rebuttal Wow, that is really nervy. Let's recap this all: * reiser4 has a 2x performance advantage over the next fastest FS (ext3), and when compression ships in a month that will double again as well as save space. See http://www.namesys.com/benchmarks.html, and Does that mean that ext3 is faster than reiser3? Wow, that would be good reason to switch default filesystem to ext3 (or reiser4?) in next suse release. Pavel -- Thanks for all the (sleeping) penguins.
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Horst H. von Brand wrote: Vladimir V. Saveliev [EMAIL PROTECTED] wrote: On Tue, 2006-08-01 at 17:32 +0200, Åukasz Mierzwa wrote: What fancy (beside cryptocompress) does reiser4 do now? it is supposed to provide an ability to easy modify filesystem behaviour in various aspects without breaking compatibility. If it just modifies /behaviour/ it can't really do much. And what can be done here is more the job of the scheduler, not of the filesystem. Keep your hands off it! Say wha? There's a lot you can do with the _representation_ of the on-disk format without changing the _physical_ on-disk format. As a very simple example, a plugin could add a sysfs-like folder with information about that particular filesystem. Yes, I know there are better ways to do things, but there are things you can change about behavior without (I think) touching the scheduler. Or am I wrong about the scope of the scheduler? If it somehow modifies /on disk format/, it (by *definition*) isn't compatible. Ditto. Cryptocompress is compatible with kernels that have a working cryptocompress plugin. Other kernels will notice that they are meant to be read by cryptocompress, and (I hope) refuse to read files they won't be able to. Same would be true of any plugin that changes the disk format. But, the above comments about behavior still hold. There's a lot you can do with plugins without changing the on-disk format. If you want a working example, look to your own favorite filesystems that support quotas, xattrs, and acls -- is an on-disk FS format with those enabled compatible with a kernel that doesn't support them (has them turned off)? How about ext3, with its journaling -- is the journaling all in the scheduler? But isn't the ext3 disk format compatible with ext2? quota support xattrs and acls Without those, it is next to useless anyway. What is? The FS? I use neither on desktop machines, though I'd appreciate xattrs for Beagle. Or are you talking about the plugins? See above, then.
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Vladimir V. Saveliev [EMAIL PROTECTED] wrote: On Tue, 2006-08-01 at 17:32 +0200, Åukasz Mierzwa wrote: Dnia Fri, 28 Jul 2006 18:33:56 +0200, Linus Torvalds [EMAIL PROTECTED] napisaÅ: In other words, if a filesystem wants to do something fancy, it needs to do so WITH THE VFS LAYER, not as some plugin architecture of its own. We already have exactly the plugin interface we need, and it literally _is_ the VFS interfaces - you can plug in your own filesystems with register_filesystem(), which in turn indirectly allows you to plug in your per-file and per-directory operations for things like lookup etc. What fancy (beside cryptocompress) does reiser4 do now? it is supposed to provide an ability to easy modify filesystem behaviour in various aspects without breaking compatibility. If it just modifies /behaviour/ it can't really do much. And what can be done here is more the job of the scheduler, not of the filesystem. Keep your hands off it! If it somehow modifies /on disk format/, it (by *definition*) isn't compatible. Ditto. Can someone point me to a list of things that are required by kernel mainteiners to merge reiser4 into vanilla? list of features reiser4 does not have now: O_DIRECT support - we are working on it now various block size support Is this required? quota support xattrs and acls Without those, it is next to useless anyway. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, ChileFax: +56 32 797513
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Dnia Wed, 02 Aug 2006 20:45:07 +0200, Horst H. von Brand [EMAIL PROTECTED] napisał: Vladimir V. Saveliev [EMAIL PROTECTED] wrote: On Tue, 2006-08-01 at 17:32 +0200, Å�ukasz Mierzwa wrote: Dnia Fri, 28 Jul 2006 18:33:56 +0200, Linus Torvalds [EMAIL PROTECTED] napisaÅ‚: In other words, if a filesystem wants to do something fancy, it needs to do so WITH THE VFS LAYER, not as some plugin architecture of its own. We already have exactly the plugin interface we need, and it literally _is_ the VFS interfaces - you can plug in your own filesystems with register_filesystem(), which in turn indirectly allows you to plug in your per-file and per-directory operations for things like lookup etc. What fancy (beside cryptocompress) does reiser4 do now? it is supposed to provide an ability to easy modify filesystem behaviour in various aspects without breaking compatibility. If it just modifies /behaviour/ it can't really do much. And what can be done here is more the job of the scheduler, not of the filesystem. Keep your hands off it! You modify the way the fs stores files or let You access them, since when it is a job for a scheduler? If it somehow modifies /on disk format/, it (by *definition*) isn't compatible. Ditto. Can someone point me to a list of things that are required by kernel mainteiners to merge reiser4 into vanilla? list of features reiser4 does not have now: O_DIRECT support - we are working on it now various block size support Is this required? quota support xattrs and acls Without those, it is next to useless anyway. I don't use any of this and I live quite happly.
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
On Mon, Jul 31, 2006 at 10:57:35AM -0500, David Masover wrote: Wil Reichert wrote: Any idea how the fragmentation resulting from re-syncing the tree affects performance over time? Yes, it does affect it a lot. I have no idea how much, and I've never benchmarked it, but purely subjectively, my portage has gotten slower over time. Delayed allocation still performs a lot better here than the v3 immediate allocation. In addition, tree balancing operations are performed on flush as well, so what you get on disk is basically an almost-optimal tree. Of course, this will change a bit over time, but with v4 it takes a lot longer for that to happen than with v3 afaict. There _has_ been some worthwile development in the meantime : ) Kind regards, Chris pgpYfJOp8uyxN.pgp Description: PGP signature
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
On Mon, Jul 31, 2006 at 06:05:01PM +0200, Łukasz Mierzwa wrote: I gues that extens are much harder to reuse then normal inodes so when You have something as big as portage tree filled with nano files wich are being modified all the time then You just can't keep performance all the time. You can always tar, rm -fr /usr/portage, untar and You will probably speed things up a lot. I submitted a script to this list which takes care of everything required to recreate your fs. It even converts between different filesystems, for migration purposes or comparitive tests, and currently supports ext2|3, reiser3|4 and xfs. The thing is undergoing some surgery atm. to reduce forced disk flushes. I already replaced the call to sync() after every operation by one fsync() call on the archive file before the formatting takes place. What is still missing is functionality to retrieve things like fs label and UUID from the existing fs and reuse them during mkfs. Testing is also pending, so you might not want to hold your breath waiting for the funky version, the idea of which is to leave everything as it was found, except for better disk layout and possibly changed fs type. It is a completely different approach from convertfs, which tries to do the conversion in-place by moving the fs's contents into a new fs created within a sparse file on the same device and relocating the sparse file's blocks afterwards. My guess is that a failure of any kind in the latter process will destroy your data (this was the case last time I checked), while I do (at least try) everything to ensure that the tarball is written to platters before mkfs occurs. The new version will be posted to wiki.namesys.com asap, no timeframe attached though as Thursday yields an exam, so maybe on Friday, but who knows. The version already posted to the list works well, I used it at least a hundred times, even on stuff like /home and /usr (the latter works only from a live cd or custom initramfs). Kind regards, Chris pgpJCvRI7J8nt.pgp Description: PGP signature
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Dnia Fri, 28 Jul 2006 18:33:56 +0200, Linus Torvalds [EMAIL PROTECTED] napisał: In other words, if a filesystem wants to do something fancy, it needs to do so WITH THE VFS LAYER, not as some plugin architecture of its own. We already have exactly the plugin interface we need, and it literally _is_ the VFS interfaces - you can plug in your own filesystems with register_filesystem(), which in turn indirectly allows you to plug in your per-file and per-directory operations for things like lookup etc. What fancy (beside cryptocompress) does reiser4 do now? Can someone point me to a list of things that are required by kernel mainteiners to merge reiser4 into vanilla? I feel like I'm getting lost with current reiser4 status and things that are need to be done. Łukasz Mierzwa
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Hello On Tue, 2006-08-01 at 17:32 +0200, Łukasz Mierzwa wrote: Dnia Fri, 28 Jul 2006 18:33:56 +0200, Linus Torvalds [EMAIL PROTECTED] napisał: In other words, if a filesystem wants to do something fancy, it needs to do so WITH THE VFS LAYER, not as some plugin architecture of its own. We already have exactly the plugin interface we need, and it literally _is_ the VFS interfaces - you can plug in your own filesystems with register_filesystem(), which in turn indirectly allows you to plug in your per-file and per-directory operations for things like lookup etc. What fancy (beside cryptocompress) does reiser4 do now? it is supposed to provide an ability to easy modify filesystem behaviour in various aspects without breaking compatibility. Can someone point me to a list of things that are required by kernel mainteiners to merge reiser4 into vanilla? list of features reiser4 does not have now: O_DIRECT support - we are working on it now various block size support quota support xattrs and acls list of warnings about reiser4 code: I think that last big list of useful comments (from Christoph Hellwig [EMAIL PROTECTED]) is addressed. Well, except for one minor (I believe) place in file release. Currently, Andrew is trying to find some time to review reiser4 code. I feel like I'm getting lost with current reiser4 status and things that are need to be done. Łukasz Mierzwa
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Christian Trefzer wrote: On Mon, Jul 31, 2006 at 10:57:35AM -0500, David Masover wrote: Wil Reichert wrote: Any idea how the fragmentation resulting from re-syncing the tree affects performance over time? Yes, it does affect it a lot. I have no idea how much, and I've never benchmarked it, but purely subjectively, my portage has gotten slower over time. Delayed allocation still performs a lot better here than the v3 immediate allocation. In addition, tree balancing operations are performed on flush as well, so what you get on disk is basically an almost-optimal tree. Of course, this will change a bit over time, but with v4 it takes a lot longer for that to happen than with v3 afaict. There _has_ been some worthwile development in the meantime : ) Hmm. The thing is, I don't remember v3 slowing down much at all, whereas v4 slowed down pretty dramatically after the first few weeks. It does seem pretty stable now, though, and it doesn't seem to be getting any slower. I've had this particular FS since... hmm... Is there an FS tool to check mkfs time? I think it's a year now, but I'd like to be sure. If not, I'll just find the oldest file, but the clock on this machine isn't reliable (have to set it with NTP every boot)...
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Hans Reiser writes: Nikita Danilov wrote: [...] As you see, ext2 code already has multiple file plugins, with persistent plugin id (stored in i_mode field of on-disk struct ext2_inode). Nikita. So the job is already done. Good. Reiser4 can be included then.:) Indeed, namesys already implemented requirements pointed to by Christoph Hellwig, as far as meta-data plugins are concerned, so this issue is not an obstacle. Hans The Easily Agreeable Reiser PS: please, don't trim CC lists. Nikita.
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Hans Reiser [EMAIL PROTECTED] wrote: David Masover wrote: If indeed it can be changed easily at all. I think the burden is on you to prove that you can change it to be more generic, rather than saying Well, we could do it later, if people want us to... None of the filesystems other than reiser4 have any interest in using plugins, Then they are probably nonsense... and this whole argument over how it should be in VFS is nonsensical because nobody but us has any interest in using the functionality. The burden is on the generic code authors to prove that they will ever ever do anything at all besides complain. Frankly, I don't think they will. I think they will never produce one line of code. It is /your/ burden to show that they are worthwhile. And if they are, thay aren't worthwhile only in your pet filesystem... Please cite one ext3 developer who is signed up to implement ext3 using plugins if they are supported by VFS. Have you asked? . It also prevents users from getting advances they could be getting today, for no reason. It prevents users from doing nothing. Most users not only cannot patch a kernel, they don't know what a patch is. It most certainly does. Simple answer: Fork the kernel with all your grandiose plans, and see which one wins. Open source, remember? It is not as if your only chance is to get it into the official kernel. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, ChileFax: +56 32 797513
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
=) That was sorta the plan. Any idea how the fragmentation resulting from re-syncing the tree affects performance over time? Wil On 7/30/06, Christian Trefzer [EMAIL PROTECTED] wrote: On Sun, Jul 30, 2006 at 11:39:42PM +0200, Christian Trefzer wrote: In order to avoid having to pull the whole tree via rsync again, you might want to grab my script from the list and adapt it to your needs. Of course, you can tar it up manually instead. Silly me, but after approx. 9h of studying, little wonder ; ) Kind regards, Chris
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Wil Reichert wrote: =) That was sorta the plan. Any idea how the fragmentation resulting from re-syncing the tree affects performance over time? Try to post replies at the bottom, or below the context. Yes, it does affect it a lot. I have no idea how much, and I've never benchmarked it, but purely subjectively, my portage has gotten slower over time.
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Dnia Mon, 31 Jul 2006 17:57:35 +0200, David Masover [EMAIL PROTECTED] napisał: Wil Reichert wrote: =) That was sorta the plan. Any idea how the fragmentation resulting from re-syncing the tree affects performance over time? Try to post replies at the bottom, or below the context. Yes, it does affect it a lot. I have no idea how much, and I've never benchmarked it, but purely subjectively, my portage has gotten slower over time. I gues that extens are much harder to reuse then normal inodes so when You have something as big as portage tree filled with nano files wich are being modified all the time then You just can't keep performance all the time. You can always tar, rm -fr /usr/portage, untar and You will probably speed things up a lot.
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
On Monday 31 July 2006 21:35, Łukasz Mierzwa wrote: I gues that extens are much harder to reuse then normal inodes so when You have something as big as portage tree filled with nano files wich are being modified all the time then You just can't keep performance all the time. You can always tar, rm -fr /usr/portage, untar and You will probably speed things up a lot. I agree here. On my home system this gave me a visible improvement in the sync process. I blew up my portage tree and I rebuilt it, and it gave me around ~10% improvement - this was around 6 months ago. I am not all that greedy, so i didn't do this do this than once :-) Sarath
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
David Masover wrote: Nikita Danilov wrote: As you see, ext2 code already has multiple file plugins, with persistent plugin id (stored in i_mode field of on-disk struct ext2_inode). Aha! So here's another question: Is it fair to ask Reiser4 to make its plugins generic, or should we be asking ext2/3 first? Sh., ext* already made their plugins generic, job is done:)
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Maciej Sołtysiak wrote: Hmm, what about linspire / freespire ? Linsire is a proud reiser4 debugging sponsor as the website (http://www.namesys.com) says. Wouldn't they want to include reiser4 in their distro first? Not if the mainstream kernel is not going to add it. Hans
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Dnia Sat, 29 Jul 2006 20:31:59 +0200, David Masover [EMAIL PROTECTED] napisał: Nikita Danilov wrote: As you see, ext2 code already has multiple file plugins, with persistent plugin id (stored in i_mode field of on-disk struct ext2_inode). Aha! So here's another question: Is it fair to ask Reiser4 to make its plugins generic, or should we be asking ext2/3 first? Doesn't iptables have plugins? Maybe we should make them generic so other packet filters can use them ;)
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Dnia Sun, 30 Jul 2006 03:08:55 +0200, Hans Reiser [EMAIL PROTECTED] napisał: Maciej Sołtysiak wrote: Hmm, what about linspire / freespire ? Linsire is a proud reiser4 debugging sponsor as the website (http://www.namesys.com) says. Wouldn't they want to include reiser4 in their distro first? Not if the mainstream kernel is not going to add it. If I got it all rigth then the VFS issue is there from the moment when Hans made a request to merge reiser4 into mainstream, nobody nows what will happen tomorrow, it will go in without moving things to vfs, it can go in with moving some code to vfs or it can stay forever outside. If it's not sure then no distro will use it as a main fs (or at all) becouse if tommarow Hans will say Ok, will move things to VFS then for the next year or so reiser4 will be simply dead, it will take a lot of time to recode it and this change will provide a lot of bugs that will need to be debuged, it won't a be a single patch anymore as You will need to patch that adds reiser4 and a patch that alters VFS. What distro would toy with VFS? None until it's all done, fully and trully tested and merged into mainline, it won't happen quickly and it is not said that there won't be any more complains when this is done. In my opinion there is no best option for reiser4 only less evil ways, it's just too different from other fs, too independent. Again I'm no expert, just enduser.
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
On Sat, Jul 29, 2006 at 03:30:15PM -0500, David Masover wrote: Is /usr/portage still faster on Reiser4? I know it was when I switched, but that was years ago... It is for sure. v4 is even better with fantastillions of small files than v3. uziel pgpxeVIQPVXTn.pgp Description: PGP signature
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Łukasz Mierzwa wrote: Dnia Sat, 29 Jul 2006 20:31:59 +0200, David Masover [EMAIL PROTECTED] napisał: Nikita Danilov wrote: As you see, ext2 code already has multiple file plugins, with persistent plugin id (stored in i_mode field of on-disk struct ext2_inode). Aha! So here's another question: Is it fair to ask Reiser4 to make its plugins generic, or should we be asking ext2/3 first? Doesn't iptables have plugins? Maybe we should make them generic so other packet filters can use them ;) Hey, yeah! I mean, not everyone wants to run the ipchains emulation on top of iptables! Some people really want to run ipchains with iptables plugins! /sarcasm It is REALLY time for this discussion to get technical again, and to go way, way over my head. And it's time for me to go build my MythTV box, and see if I can shake out some Reiser4 bugs.
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Hans Reiser [EMAIL PROTECTED] wrote: Let me put it from my perspective and stop pretending to be unbiased, so others can see where I am coming from. OK, but /that/ was pretty clear from day one... No one was interested in our plugins. Should tell you something... We put the design on a website, spoke at conferences, no one but users were interested. Again, should tell you something... Look, a cool new gadget nobody ever imagined before sure attracts lots of people. Intriguing. Play around a bit. Go for next last novel gadget. Rinse, repeat. No one would have conceived of having plugins if not for us. Perhaps because nobody else sees any sense in them? Our plugins affect no one else. Wrong. Others will want plugins if they are any use, at the very least. And then there is the possibility of /massive/ maintainance problems (So, you /are/ using Reiser 4, the broken file has attached plugin A version 5.3, and is reached through a directory with plugin D 3.4rc5, and ...). /I/ wouldn't want to be at the receiving end of bug reports which after a dozen rounds boil down to this. Our self-contained code should not be delayed Not your call to make. because other people delayed getting interested in our ideas and now they don't want us to have an advantage from leading. Your problem, not Linux'. If they want to some distant day implement generic plugins, for which they have written not one line of code to date, fine, we'll use it when it exists, but right now those who haven't coded should get out of the way of people with working code. But they are! Just branch the kernel, and be done with it. It is not fair or just to do otherwise. /You/ are asking the kernel developers for a /huge/ favor. Even totally go out of their ways, and acting contrary to their set ways and beliefs. Saying no to that can't be called unfair... It also prevents users from getting advances they could be getting today, for no reason. OK, so you have /never/ seen any reasons given here for not placing Reiser 4 into the kernel? Strange... Our code will not be harder to change once it is in the kernel, it will be easier, because there will be more staff funded to work on it. Right. Just like Reiser 3 right now. As for this we are all too grand to be bothered with money to feed our families business, building a system in which those who contribute can find a way to be rewarded is what managers do. Free software programmers may be willing to live on less than others, but they cannot live on nothing, and code that does not ever ship means living on nothing. Then go do something else... If reiser4 is delayed enough, for reasons that have nothing to do with its needs, and without it having encumbered anyone else, it won't be ahead of the other filesystems when it ships. How is that important in any way for the Linux kernel? This is not (and has not been for quite some time now) an experimental operating system. And it has /never/ been a dumpling ground for the next grand idea, it has always been about sound engineering. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, ChileFax: +56 32 797513
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Hmm, looks like I have a partition to re-format now. Wil On 7/30/06, Christian Trefzer [EMAIL PROTECTED] wrote: On Sat, Jul 29, 2006 at 03:30:15PM -0500, David Masover wrote: Is /usr/portage still faster on Reiser4? I know it was when I switched, but that was years ago... It is for sure. v4 is even better with fantastillions of small files than v3. uziel
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
If reiser4 is delayed enough, for reasons that have nothing to do with its needs, and without it having encumbered anyone else, it won't be ahead of the other filesystems when it ships. How is that important in any way for the Linux kernel? This is not (and has not been for quite some time now) an experimental operating system. And it has /never/ been a dumpling ground for the next grand idea, it has always been about sound engineering. One of the groundbreaking things about Linux' (modular) development model was, I thought, that it can succeed at both... not necessarily at the same time, but for different users? --T
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Hi Wil, On Sun, Jul 30, 2006 at 02:10:04PM -0700, Wil Reichert wrote: Hmm, looks like I have a partition to re-format now. In order to avoid having to pull the whole tree via rsync again, you might want to grab my script from the list and adapt it to your needs. Kind regards, Chris pgpNAbzw9ZKq9.pgp Description: PGP signature
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
On Sun, Jul 30, 2006 at 11:39:42PM +0200, Christian Trefzer wrote: In order to avoid having to pull the whole tree via rsync again, you might want to grab my script from the list and adapt it to your needs. Of course, you can tar it up manually instead. Silly me, but after approx. 9h of studying, little wonder ; ) Kind regards, Chris pgpj1TrQGhCRH.pgp Description: PGP signature
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Christian Trefzer wrote: On Sun, Jul 30, 2006 at 11:39:42PM +0200, Christian Trefzer wrote: In order to avoid having to pull the whole tree via rsync again, you might want to grab my script from the list and adapt it to your needs. Of course, you can tar it up manually instead. Silly me, but after approx. 9h of studying, little wonder ; ) In fact, the official install guide tells you to download a snapshot tarball first, then start syncing.
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
David Masover wrote: If indeed it can be changed easily at all. I think the burden is on you to prove that you can change it to be more generic, rather than saying Well, we could do it later, if people want us to... None of the filesystems other than reiser4 have any interest in using plugins, and this whole argument over how it should be in VFS is nonsensical because nobody but us has any interest in using the functionality. The burden is on the generic code authors to prove that they will ever ever do anything at all besides complain. Frankly, I don't think they will. I think they will never produce one line of code. Please cite one ext3 developer who is signed up to implement ext3 using plugins if they are supported by VFS. . It also prevents users from getting advances they could be getting today, for no reason. It prevents users from doing nothing. Most users not only cannot patch a kernel, they don't know what a patch is. It most certainly does. Hans
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Most users not only cannot patch a kernel, they don't know what a patch is. It most certainly does. obviously you can provide complete kernels, including precompiled ones. Most distros have a yum or apt or similar tool to suck down packages, it's trivial for users to add a site to that, so you could provide packages if you want and make it easy for them. -- if you want to mail me at work (you don't), use arjan (at) linux.intel.com
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Jeff Garzik wrote: Using guilt as an argument in a technical discussion is a flashing red sign that says I have no technical rebuttal Wow, that is really nervy. Let's recap this all: * reiser4 has a 2x performance advantage over the next fastest FS (ext3), and when compression ships in a month that will double again as well as save space. See http://www.namesys.com/benchmarks.html, and then ask the reiserfs-list@namesys.com whether those benchmarks are fair representations of their experiences. This is in a field where a 25% advantage is a hard won big deal. * we described our plugin architecture in 2001. No other FS developers were interested, only the users were, and it was presented quite a lot. * So we implemented plugins for ourselves, because no other FS developers would possibly have supported us touching their code. (I do not say that they erred in this.) * No one has actually made a serious case for it being genericizable when you get to the details, it is all just handwaving. I'd be surprised if 10% of it was FS independent, and unsurprised if making that 10% FS independent made the code ossified and hard to maintain. I do not in anyway claim that those who choose to implement Reiser4 plugins are not deeply affected by Reiser4 design choices. Most of the value of writing Reiser4 plugins comes from being able to reuse Reiser4 code as you choose to in the process, and if Reiser4 is not to your taste as a whole, then nobody should impose our plugins upon you. VFS is a bad enough straight jacket for FS developers, we don't need even more mandated design decisions for the FS developers to come who will be brighter than us. Actually, I would like to see Nate Diller implement a competing VFS layer, I think he would do a very good job of that. * Here we are today, and Reiser4 plugins work. Now some say that because we did it for Reiser4 and not for every other FS, that we should be excluded from the kernel. So we are supposed to re-implement it as generic code, which will involve years of time, and then finally something will be coded and nobody but us will use it, and then they will tell us that because nobody but us wants to use it it cannot go in. If you disagree, find one ext3 developer who wants to rewrite ext3 to use plugins and change its disk format to do it. And you have the nerve to say that this ever was a technical discussion? Our code measurably works the best. If folks want to imitate it, go ahead, but don't blame us for making our code work without first making those other folks's code work. The technical rebuttal you ask for is http://www.namesys.com/benchmarks.html. The only time this argument gets technical is when akpm is involved. He was right about what should technically be done about batch write, which, by the way, was greeted upon completion with an if only reiser4 uses it then it should not go in response. We are being penalized for thinking too differently, and this whole ping-pong between no we don't want to do it your way and you did it your way for only you, redo it for us even though we won't ever use it and oh, you redid it for us but none of us want to use it, so no it is an imposition and cannot go in is the Kafka-esque manifestation of that. If only reiser4 wants to use something, then just let us do it in our little corner without bothering anybody else. (Though any advice from akpm that he has time for giving us is always welcome.) David, we aren't asking to be in the band, we are asking to be in the jukebox. I think enough users want to go 2x as fast that the users would benefit from our being in the jukebox. Hans
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Mike and Lukasz, please post your email to not just reiserfs-list, where only the reiserfs team will read it, but also to lkml if you could, please? Thanks for your support, user opinions count for a lot on lkml.
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Hans Reiser writes: David Masover wrote: If indeed it can be changed easily at all. I think the burden is on you to prove that you can change it to be more generic, rather than saying Well, we could do it later, if people want us to... None of the filesystems other than reiser4 have any interest in using plugins, and this whole argument over how it should be in VFS is nonsensical because nobody but us has any interest in using the functionality. The burden is on the generic code authors to prove that they will ever ever do anything at all besides complain. Frankly, I don't think they will. I think they will never produce one line of code. Please cite one ext3 developer who is signed up to implement ext3 using plugins if they are supported by VFS. In fact, they all do: struct inode_operations ext2_file_inode_operations; struct inode_operations ext2_dir_inode_operations; struct inode_operations ext2_special_inode_operations; struct inode_operations ext2_symlink_inode_operations; struct inode_operations ext2_fast_symlink_inode_operations; As you see, ext2 code already has multiple file plugins, with persistent plugin id (stored in i_mode field of on-disk struct ext2_inode). Hans Nikita.
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Arjan van de Ven wrote: Most users not only cannot patch a kernel, they don't know what a patch is. It most certainly does. obviously you can provide complete kernels, including precompiled ones. Most distros have a yum or apt or similar tool to suck down packages, it's trivial for users to add a site to that, so you could provide packages if you want and make it easy for them. What's more, many distros patch their kernels extensively. They listen to their users, too. So if there are a lot of users wanting this to be in the kernel, let them complain -- loudly -- to their distro to patch for Reiser4. It could be made even easier than that -- if Reiser4 is really so self-contained, it could be a whole separate set of modules, distributed on its own. Most gamers have to be content with doing something similar with the nvidia drivers -- for different reasons (licensing) but with the same results. I know Gentoo handles this automatically (emerge nvidia-kernel). Hmm, maybe it makes it a pain to have it as a root filesystem, so that really needs distro support. And yet, we have a whole system designed specifically for being able to load modules and tweak settings before the root FS is available. It's called initrd, or more recently, initramfs. I use an old-style initrd on this box, because my root FS is on an nvidia RAID, so I have to run a program called dmraid before I mount my root FS -- it's really trivial for me to have Reiser4 as a module, and I do, despite it being a root FS. I suspect that, all technical, political, and mine is bigger arguments aside, being available as a root FS of a distro, especially a default FS, would go a long way towards inclusion in the kernel. So all you have to do is find a reasonably popular and friendly distro, with people who are (for the moment) easier to deal with than kernel people. Most people, if they even know what a filesystem or a kernel is, still won't bother compiling their own kernel, you're right. But that means they are more likely to be using a distro-patched kernel than a stock, vanilla one. Is this enough to be in the jukebox, Hans? Of course, it's odd that I mention Gentoo, the Gentoo people (as a rule) hate ReiserFS, but there are far more distros than there are popular kernel forks. I'm sure someone will be interested. That's assuming that making further changes (putting stuff in the VFS) is out of the question (for now). signature.asc Description: OpenPGP digital signature
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Hans Reiser wrote: David Masover wrote: If indeed it can be changed easily at all. I think the burden is on you to prove that you can change it to be more generic, rather than saying Well, we could do it later, if people want us to... None of the filesystems other than reiser4 have any interest in using plugins, and this whole argument over how it should be in VFS is nonsensical because nobody but us has any interest in using the functionality. The burden is on the generic code authors to prove that they will ever ever do anything at all besides complain. Frankly, I don't think they will. I think they will never produce one line of code. I think it's fair to say that 5-10 years from now, with different ext3 maintainers, when the Reiser4 concept has proven itself, people will want plugins for ext3, and the ext3 developers will like the idea. ext* is one of those things that just refuses to die. I use ext3 for my /boot fs, so that I don't have to patch Grub for Reiser4, and so that at least I can mess with the bootloader from any rescue CD if something goes wrong. It's for kind of the same reason that Gentoo builds a 32-bit Grub, even though I'm booting a 64-bit OS -- just in case. I also use ext2 for my initrd. There are other monstrosities that will likely never die, also. ISO9660, UDF, and VFAT probably all have worse storage characteristics than Reiser4, in that as I understand it, they won't pack multiple files into a block. So Reiser4 might even make a good boot cd FS, storing things more efficiently -- but even if I'm right, those three filesystems will last forever, because they are currently well supported on every major OS, and I think one of ISO/UDF is required for DVDs. So for whatever reason someone's using another filesystem, even if all they need is the on-disk format (my reason for ext3 /boot and vfat on USB thumbdrives), I think it's reasonable to expect that they may one day want plugin functionality. People who like Reiser filesystems will do just fine running Reiser4 with a (udf|iso|vfat) storage driver, but people who don't will just want the higher level stuff. You're probably right and this is years of work for something that may not be worth anything, but I think this is what is going through people's heads as they look at this plugin system. So see my comments about distro inclusion. signature.asc Description: OpenPGP digital signature
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Nikita Danilov wrote: As you see, ext2 code already has multiple file plugins, with persistent plugin id (stored in i_mode field of on-disk struct ext2_inode). Aha! So here's another question: Is it fair to ask Reiser4 to make its plugins generic, or should we be asking ext2/3 first? signature.asc Description: OpenPGP digital signature
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
David Masover writes:
Nikita Danilov wrote:
As you see, ext2 code already has multiple file plugins, with
persistent plugin id (stored in i_mode field of on-disk struct
ext2_inode).
Aha! So here's another question: Is it fair to ask Reiser4 to make its
plugins generic, or should we be asking ext2/3 first?
ext2/3 plugins are generic: in Linux every file system can implement
per-object behavior by specifying
{file,inode,dentry,address_space}_operations. This mechanism is provided
by VFS (and, in fact, is the only way that VFS interacts with file
system) and is completely generic.
Nikita.
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
On Saturday 29 July 2006 23:41, David Masover wrote: I know Gentoo handles this automatically (emerge nvidia-kernel). I hate to say this again, but its not automatically. It requires more knowledge of ther user, and is more than asking users to patch the kernel. (I am in the support industry and I know of solaris admins, who have used that os for more than ten years ask me what the command patch does and what its impact is to production systems :-) ) I use an old-style initrd on this box, because my root FS is on an nvidia RAID, so I have to run a program called dmraid before I mount my root FS -- it's really trivial for me to have Reiser4 as a module, and I do, despite it being a root FS. As a fact I have it as a root FS (and that too reiser4, and as a module). Surprisingly, this is a wiki article too in http://gentoo-wiki.com, by someone who's more patient enought to write one :-) I suspect that, all technical, political, and mine is bigger arguments aside, being available as a root FS of a distro, especially a default FS, would go a long way towards inclusion in the kernel. So all you have to do is find a reasonably popular and friendly distro, with people who are (for the moment) easier to deal with than kernel people. Its actually a matter of a hastle for the end user. That's where I would agree with Hans' comments quite earlier. Most people, if they even know what a filesystem or a kernel is, still won't bother compiling their own kernel, you're right. But that means they are more likely to be using a distro-patched kernel than a stock, vanilla one. Well, that's different, and that's the main problem in the linux empowerment that we see around ourselves. It finally revolves around the user, and as harsh as it may seem, it ultimately is the user who decides which fs is better (Give or take, they don't know the difference between a kernel or user-space. or for that matter far more basic things.) Of course, it's odd that I mention Gentoo, the Gentoo people (as a rule) hate ReiserFS, but there are far more distros than there are popular kernel forks. I'm sure someone will be interested. I do, and that's partly due to the speed of /usr/portage on reiser4, and the easiness of blowing everything and starting from scratch : -) Cheers, Sarath -- The gentlemen looked one another over with microscopic carelessness.
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Sarath Menon wrote: On Saturday 29 July 2006 23:41, David Masover wrote: I know Gentoo handles this automatically (emerge nvidia-kernel). I hate to say this again, but its not automatically. It requires more My point is, there's a fairly large group of users who would be willing to do that, as they're willing to do that to get their video drivers working. Also, assuming a distro did choose to support it, the only reason nvidia-kernel isn't just distributed with a pre-built kernel (on pre-built OSes, anyway) is licensing. This isn't a problem for Reiser4, which is GPL'd. I suspect that, all technical, political, and mine is bigger arguments aside, being available as a root FS of a distro, especially a default FS, would go a long way towards inclusion in the kernel. So all you have to do is find a reasonably popular and friendly distro, with people who are (for the moment) easier to deal with than kernel people. Its actually a matter of a hastle for the end user. That's where I would agree with Hans' comments quite earlier. Putting it in the kernel doesn't make it any more or less of a hassle for the end-user than getting distro support. I remember downloading a different set of Debian floppies which supported XFS, before XFS was mainstream. In that sense, it's somewhat done already -- there is a Gentoo livecd that is kept patched for Reiser4. The problem with Gentoo, of course, is that if you're going to use Gentoo, you're going to be compiling your own kernel. So when it comes down to getting vanilla-sources or gentoo-sources, it wouldn't take much -- just a reiser4-sources, or a separate reiser4-module package. Most people, if they even know what a filesystem or a kernel is, still won't bother compiling their own kernel, you're right. But that means they are more likely to be using a distro-patched kernel than a stock, vanilla one. Well, that's different, and that's the main problem in the linux empowerment that we see around ourselves. It finally revolves around the user, and as harsh as it may seem, it ultimately is the user who decides which fs is better (Give or take, they don't know the difference between a kernel or user-space. or for that matter far more basic things.) If I remember right, SuSe had ReiserFS as the default at one point. If even one moderately popular Linux had Reiser4 as the default FS, it would get a LOT more exposure than it would simply being included (as EXPERIMENTAL, at that) in the vanilla kernel. Of course, it's odd that I mention Gentoo, the Gentoo people (as a rule) hate ReiserFS, but there are far more distros than there are popular kernel forks. I'm sure someone will be interested. I do, and that's partly due to the speed of /usr/portage on reiser4, and the easiness of blowing everything and starting from scratch : -) Yes, I love /var/lib/portage/world also. Is /usr/portage still faster on Reiser4? I know it was when I switched, but that was years ago...
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Hello David, Saturday, July 29, 2006, 8:11:11 PM, you wrote: What's more, many distros patch their kernels extensively. They listen to their users, too. So if there are a lot of users wanting this to be in the kernel, let them complain -- loudly -- to their distro to patch for Reiser4. Hmm, what about linspire / freespire ? Linsire is a proud reiser4 debugging sponsor as the website (http://www.namesys.com) says. Wouldn't they want to include reiser4 in their distro first? I removed the busy people from CC to save their time reading this. -- Best regards, Maciej
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Nikita Danilov wrote: Hans Reiser writes: David Masover wrote: If indeed it can be changed easily at all. I think the burden is on you to prove that you can change it to be more generic, rather than saying Well, we could do it later, if people want us to... None of the filesystems other than reiser4 have any interest in using plugins, and this whole argument over how it should be in VFS is nonsensical because nobody but us has any interest in using the functionality. The burden is on the generic code authors to prove that they will ever ever do anything at all besides complain. Frankly, I don't think they will. I think they will never produce one line of code. Please cite one ext3 developer who is signed up to implement ext3 using plugins if they are supported by VFS. In fact, they all do: struct inode_operations ext2_file_inode_operations; struct inode_operations ext2_dir_inode_operations; struct inode_operations ext2_special_inode_operations; struct inode_operations ext2_symlink_inode_operations; struct inode_operations ext2_fast_symlink_inode_operations; As you see, ext2 code already has multiple file plugins, with persistent plugin id (stored in i_mode field of on-disk struct ext2_inode). Hans Nikita. So the job is already done. Good. Reiser4 can be included then.:) Hans The Easily Agreeable Reiser
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Jeff Garzik [EMAIL PROTECTED] wrote: [...] It is then simple to follow that train of logic: why not make it easy to replace the directory algorithm [and associated metadata]? or the file data space management algorithms? or even the inode handling? why not allow customers to replace a stock algorithm with an exotic, site-specific one? IMVHO, such experiments should/must be done in userspace. And AFAICS, they can today. Go wild, but leave the kernel alone. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, ChileFax: +56 32 797513
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Horst H. von Brand wrote: Jeff Garzik [EMAIL PROTECTED] wrote: [...] It is then simple to follow that train of logic: why not make it easy to replace the directory algorithm [and associated metadata]? or the file data space management algorithms? or even the inode handling? why not allow customers to replace a stock algorithm with an exotic, site-specific one? IMVHO, such experiments should/must be done in userspace. And AFAICS, they can today. inode handling? Really? But what's wrong with people doing such experiments outside the kernel? AFAICS, exotic, site-specific one is not something that would be considered for inclusion.
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
On Fri, 28 Jul 2006, David Masover wrote: But what's wrong with people doing such experiments outside the kernel? AFAICS, exotic, site-specific one is not something that would be considered for inclusion. Here's a few ground rules at least from my viewpoint: - as long you call them plugins and treat them as such, I (and I suspect a lot of other people) are totally uninterested, and in fact, a lot of people will suspect that the primary aim is to either subvert the kernel copyright rules, or at best to create a mess of incompatible semantics with no sane overlying rules for locking etc. - the kernel does not, and _will_ not, support hooks that aren't actually used (and make sense) by normal kernel uses, for largely the same reasons. Interfaces need to be architected, make sense, and have real and valid GPL'd uses. In other words, if a filesystem wants to do something fancy, it needs to do so WITH THE VFS LAYER, not as some plugin architecture of its own. We already have exactly the plugin interface we need, and it literally _is_ the VFS interfaces - you can plug in your own filesystems with register_filesystem(), which in turn indirectly allows you to plug in your per-file and per-directory operations for things like lookup etc. If that isn't enough, then the filesystem shouldn't make its own internal plug-in architecture that bypasses the VFS layer and exposes functionality that isn't necessarily sane. For example, reiser4 used to have (perhaps still does) these cool files that can be both directories and links, and I don't mind that at all, but I _do_ mind the fact that when Al Viro (long long ago) pointed out serious locking issues with them, those seemed to be totally brushed away. So as far as I'm concerned, the problem with reiser4 is that it hasn't tried to work with the VFS people. Now, I realize that the main VFS people aren't always easy to work with (Al and Christoph, take a bow), but that doesn't really change the basic facts. Al in particular is _always_ right. I don't think I've ever had the cojones to argue with Al.. Linus
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Linus Torvalds wrote: In other words, if a filesystem wants to do something fancy, it needs to do so WITH THE VFS LAYER, not as some plugin architecture of its own. Where does VFS store the plugin ids that specify per file variations? /etc/fstab? Also, is (current) VFS the interface for specifying where the hash directory plugin goes (specifies what order directory entries are within the directory)? What about the node layout plugin? The disk format plugin? Etc.? Our approach is different, but it has reasons. We eliminated the layer of indirection that Hellwig objected to, in which VFS called the plugin which called its method. (Let us try to avoid arguments over whether if you extend VFS it is still called VFS or is called reiser4's plugin layer, agreed?) Regarding copyright, these plugins are compiled in. I have resisted dynamically loaded plugins for now, for reasons I will not go into here. You can either portray reiser4 as duplicating VFS, or you can portray it as taking it to the next level, in which files (objects with classes and methods) vary rather than solely filesystems. I would prefer the latter.;-) If you agree with taking it to the next level, then it is only to be expected that there are things that aren't well suited as they are, like parsing /etc/fstab when you have a trillion files. It is not very feasible to do it for all of the filesystems all at once given finite resources, it needs a prototype. We already have exactly the plugin interface we need, and it literally _is_ the VFS interfaces - you can plug in your own filesystems with register_filesystem(), which in turn indirectly allows you to plug in your per-file and per-directory operations for things like lookup etc. If that isn't enough, then the filesystem shouldn't make its own internal plug-in architecture that bypasses the VFS layer and exposes functionality that isn't necessarily sane. For example, reiser4 used to have (perhaps still does) these cool files that can be both directories and links, and I don't mind that at all, but I _do_ mind the fact that when Al Viro (long long ago) pointed out serious locking issues with them, those seemed to be totally brushed away. We disabled them, and we won't enable them until them until the bug is fixed. It is fixable, but not within this year's programmer resources to fix it. I thank him for pointing out the bug, and it is not trivial to fix it. I don't think I've ever had the cojones to argue with Al.. Linux needs all kinds of people, not just the kind that can audit locking and copy Plan 9 well (which was very valuable to do), but now that Linux is large in the market it also needs those who can take it where Plan 9 has not already been. Why should we remain technology trailers instead of moving into the role of leaders? We have finite resources. We can give you a working filesystem with roughly twice the IO performance of the next fastest you have that does not disturb other filesystems,. (4x once the compression plugin is fully debugged). It also fixes various V3 bugs without disturbing that code with deep fixes. We cannot take every advantage reiser4 has and port it to every other filesystem in the form of genericized code as a prerequisite for going in, we just don't have the finances. Without plugins our per file compression plugins and encryption plugins cannot work. We can however let other filesystems use our code, and cooperate as they extend it and genericize it for their needs. Imposing code on other development teams is not how one best leads in open source, one sets an example and sees if others copy it. That is what I propose to do with our plugins. If no one copies, then we have harmed no one. Reasonable? Hans
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Hans Reiser wrote: Linus Torvalds wrote: In other words, if a filesystem wants to do something fancy, it needs to do so WITH THE VFS LAYER, not as some plugin architecture of its own. (Let us try to avoid arguments over whether if you extend VFS it is still called VFS or is called reiser4's plugin layer, agreed?) Ok, assuming you actually extend the VFS. The point is that if we want plugins, we don't have to implement them in ext3, but we have to put the plugin interface somewhere standard that is obviously not part of one filesystem (the VFS is the place) so that ext3 can implement a plugin system without having to read or touch a line of reiser4 code, and without compiling reiser4 into the kernel. It may ultimately not be any different, technically. This seems more like an organizational and political thing. But that doesn't make it less important or valid. Regarding copyright, these plugins are compiled in. I have resisted dynamically loaded plugins for now, for reasons I will not go into here. Good point, there's no GPL issue here. Plugins will either not be distributed (used internally) or distributed as GPL. If you agree with taking it to the next level, then it is only to be expected that there are things that aren't well suited as they are, like parsing /etc/fstab when you have a trillion files. It is not very feasible to do it for all of the filesystems all at once given finite resources, it needs a prototype. Doesn't have to be in fstab, I hope, but think of it this way: ext3 uses JBD for its journaling. As I understand it, any other filesystem can also use JBD, and ext3 is mostly ext2 + JDB. So, make the plugin interface generic enough that it compliments the VFS, doesn't duplicate it, and doesn't exist as part of Reiser4 (and requires Reiser4 to be present). This may be just a bunch of renaming or a lot of work, I don't know, but I suspect it would make a lot of people a lot happier. We have finite resources. We can give you a working filesystem with roughly twice the IO performance of the next fastest you have that does not disturb other filesystems,. (4x once the compression plugin is fully debugged). It also fixes various V3 bugs without disturbing that code with deep fixes. We cannot take every advantage reiser4 has and port it to every other filesystem in the form of genericized code as a prerequisite for going in, we just don't have the finances. This is a very compelling argument to me, but that's preaching to the choir, I've been running Reiser4 since before it was released, and before it looked like it was going to be stable anytime soon. It may be bold of me to speak for the LKML, but I think the general consensus is: The speed of a nonworking program is irrelevant -- no one cares how fast it is if it breaks things, either now or in the future. Currently, the concern is that it breaks things in the future, like adding plugin support to other filesystems. And no one else cares what your finances are. Not out of compassion, but out of practicality. For instance, it would be a huge financial benefit to me if the kernel displayed, in big bold letters while booting, that DAVID MASOVER WROTE THIS! (I'm sure Linus knows what I'm talking about.) It would also be untrue in my case, and pointless for everyone else in the kernel, so I have to find another way to make money. This is because one way Linux stays ahead of the competition (technologically) is by having quality be a much greater motivation than money. Without plugins our per file compression plugins and encryption plugins cannot work. We can however let other filesystems use our code, and cooperate as they extend it and genericize it for their needs. Imposing code on other development teams is not how one best leads in open source, one sets an example and sees if others copy it. That is what I propose to do with our plugins. If no one copies, then we have harmed no one. Reasonable? Someone still has to maintain the FS. Anyway, like I said, this is a very compelling argument for me, but code speaks louder than words. Maybe, if you insist it's not in the VFS, maybe use some insanely simple FS like RomFS to demonstrate another FS using plugins? Do that, and put it in the VFS. Maybe implement something like cramfs as a romfs plugin (another demo). Maybe even per-file -- implement zisofs as isofs + compression plugin. I think that would effectively kill any argument that plugins are bad because they are only in Reiser4. Beyond that is all marketing, I guess. The word plugin is not helping here, too many people remember Plugins like Macromedia Flash...
RE: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Doesn't have to be in fstab, I hope, but think of it this way: ext3 uses JBD for its journaling. As I understand it, any other filesystem can also use JBD, and ext3 is mostly ext2 + JDB. The fact that no other major journaling filesystems use JBD except EXT3 might make this idea less appealing.. You can also say other filesystems CAN use plugin. But who actually want to use it now? Even if there are people that want to use it, they should help doing the work. I remember someone said something along the lines of Linux is evolution, not revolution. To me it seems unreasonable to put all the revolutionary VFS burden upon reiserfs team. It's not practical. Hua
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Let me put it from my perspective and stop pretending to be unbiased, so others can see where I am coming from. No one was interested in our plugins. We put the design on a website, spoke at conferences, no one but users were interested. No one would have conceived of having plugins if not for us. Our plugins affect no one else. Our self-contained code should not be delayed because other people delayed getting interested in our ideas and now they don't want us to have an advantage from leading. If they want to some distant day implement generic plugins, for which they have written not one line of code to date, fine, we'll use it when it exists, but right now those who haven't coded should get out of the way of people with working code. It is not fair or just to do otherwise. It also prevents users from getting advances they could be getting today, for no reason. Our code will not be harder to change once it is in the kernel, it will be easier, because there will be more staff funded to work on it. As for this we are all too grand to be bothered with money to feed our families business, building a system in which those who contribute can find a way to be rewarded is what managers do. Free software programmers may be willing to live on less than others, but they cannot live on nothing, and code that does not ever ship means living on nothing. If reiser4 is delayed enough, for reasons that have nothing to do with its needs, and without it having encumbered anyone else, it won't be ahead of the other filesystems when it ships.
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Hua Zhong wrote: I remember someone said something along the lines of Linux is evolution, not revolution. To me it seems unreasonable to put all the revolutionary VFS burden upon reiserfs team. It's not practical. Thanks for saying that Hua. We have a guy named Nate Diller, who probably could fix VFS up pretty nicely if Namesys did not have him earning the consulting income the rest of the team lives on (he is doing io scheduler work at the moment). That said, he would need a lot of time, and stopping reiser4 inclusion to await his work merely ensures his work will never happen.
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Dnia Fri, 28 Jul 2006 15:34:36 +0200, Hans Reiser [EMAIL PROTECTED] napisał: Let me put it from my perspective and stop pretending to be unbiased, so others can see where I am coming from. No one was interested in our plugins. We put the design on a website, spoke at conferences, no one but users were interested. No one would have conceived of having plugins if not for us. Our plugins affect no one else. Our self-contained code should not be delayed because other people delayed getting interested in our ideas and now they don't want us to have an advantage from leading. If they want to some distant day implement generic plugins, for which they have written not one line of code to date, fine, we'll use it when it exists, but right now those who haven't coded should get out of the way of people with working code. It is not fair or just to do otherwise. It also prevents users from getting advances they could be getting today, for no reason. Our code will not be harder to change once it is in the kernel, it will be easier, because there will be more staff funded to work on it. As for this we are all too grand to be bothered with money to feed our families business, building a system in which those who contribute can find a way to be rewarded is what managers do. Free software programmers may be willing to live on less than others, but they cannot live on nothing, and code that does not ever ship means living on nothing. If reiser4 is delayed enough, for reasons that have nothing to do with its needs, and without it having encumbered anyone else, it won't be ahead of the other filesystems when it ships. It just hited me that 90% of mails (those I've read and remember) in which You guys are talking why r4 should or should not be merged did not contain a patch or not even a line of code as a reference, most of complains feels so abstract and ahead of time, there is nothing wrong with planing ahead but does it still makes sense when nobody else actually said that he would want to use it in future? Can't it be pusshed up to vfs later if it proves itself and there is demand for it? My question is what does it break now? It's been in mm for some time now, what troubles does it couse? Did anyone complained that it breaks mm and should be dropped? I'm just a end user and have no idea of linux internals, but all the fuzz about r4 seems so political and it's not just me who things so.
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
On Fri, 2006-07-28 at 22:58 +0200, Łukasz Mierzwa wrote: It just hited me that 90% of mails (those I've read and remember) in which You guys are talking why r4 should or should not be merged did not contain a patch or not even a line of code as a reference, most of complains feels so abstract and ahead of time, there is nothing wrong with planing ahead but does it still makes sense when nobody else actually said that he would want to use it in future? Can't it be pusshed up to vfs later if it proves itself and there is demand for it? My question is what does it break now? It's been in mm for some time now, what troubles does it couse? Did anyone complained that it breaks mm and should be dropped? I'm just a end user and have no idea of linux internals, but all the fuzz about r4 seems so political and it's not just me who things so. Agreed. As an outsider looking in trying to understand this all, it sounds like Hans can't win either way. People complain if the plugin code is contained entirely within Reiser4 and people complain if he tries to add functionality to the VFS that only Reiser4 will likely use for the foreseeable future. Or maybe ever use, we don't know yet? Ignoring pseudo files which Hans has pulled for now anyways, what difference is their between Reiser4 plugins and what EXT3 does with mount options like dir_index? 3rd party plugins can't be dynamically loaded, and some of them need mount options to enable them even after they are compiled in. So people can't mistakenly load some plugin that corrupts their data, or changes the on-disk format making it impossible to downgrade kernel versions. Is it the fact that Reiser4 plugins can be enabled/disable on a per file basis? If you don't use the word plugin, how is adding dir_index code to EXT3 any different then adding encryption/compression code to Reiser4? -- Mike Benoit [EMAIL PROTECTED] signature.asc Description: This is a digitally signed message part
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Hans Reiser wrote: plugins if not for us. Our plugins affect no one else. Our self-contained code should not be delayed because other people delayed And at the moment, I can still use Reiser4. If I ever make a distro, I will include Reiser4 support, probably as the default FS. That will help with getting into the kernel. So, why is it that it's urgent to get into the kernel? It will have to be bootstrapped one way or another -- either get it into the kernel so distros are more likely to include it, or get it into distros so the kernel is more likely to include it. But this is exactly the kind of thing that has happened before. With XFS, with Nvidia even -- clean it up, do it the way the kernel people want you to, because they're the ones who will have to maintain it for 20 years, and make sure it doesn't stop working or break anything else. advantage from leading. If they want to some distant day implement generic plugins, for which they have written not one line of code to date, fine, we'll use it when it exists, but right now those who haven't coded should get out of the way of people with working code. It is not fair or just to do otherwise. It also prevents users from getting advances they could be getting today, for no reason. It prevents users from doing nothing. Our code will not be harder to change once it is in the kernel, it will be easier, because there will be more staff funded to work on it. If indeed it can be changed easily at all. I think the burden is on you to prove that you can change it to be more generic, rather than saying Well, we could do it later, if people want us to... As for this we are all too grand to be bothered with money to feed our families business, building a system in which those who contribute can find a way to be rewarded is what managers do. Free software programmers may be willing to live on less than others, but they cannot live on nothing, and code that does not ever ship means living on nothing. Let me put this in perspective the best way I know how, with an inane analogy: Suppose there's a band. A good band, full of impossible superstars, led by a benevolent dictator -- for the sake of argument, let's call him Elvis. (the King -- dictator...) The band's doing really well, and Elvis crew are getting paid fairly well just to share their music. (Ok, maybe Elvis didn't write anything, but bear with me...) Now, along comes a young Jimi Hendrix. He wants to be in the band, and Elvis says Sure, just come up with a song we like and we'll play it, and you can even play it with us! Sounds like a pretty good deal, so Jimi goes and tells all his friends, a couple of girls... Now, Jimi finishes his song, Elvis listens to it, and if you know anything about the music Elvis did and the music Hendrix did, you can imagine what happens next. Elvis says This song just isn't us. But if you change it here, and here, and maybe here, we'll play it. Jimi is devastated. He'd been counting on playing it with them that night, and if he doesn't, he won't have any groupies, all his friends will laugh at him, and his life will kind of suck. But, does anyone really think Elvis has any business singing Voodoo Child? Or Purple Haze? Is it really fair to ask Elvis to completely change his act and embarrass himself to help Jimi out? The answer is, Jimi shouldn't have staked so much on something that was never a guarantee. And what's more, the real-life Jimi Hendrix never played with Elvis, but had a very successful band of his own. And if Elvis was still alive, seeing Jimi play might make him change his mind, maybe -- but at least with his own band, Jimi's success isn't pinned on playing with Elvis. This analogy is flawed in many ways, aside from just being plain chronologically impossible, but while I'm sure Linus feels bad for you, I don't think it's his obligation to compromise his kernel to help you out with your financial situation. So it would help a lot if you wouldn't keep bringing it up in what should be a technical discussion. So, if you can't make it work with VFS, then I guess you can't, and you're stuck either creating another interface which is not tied to any one filesystem and isn't tied to the VFS either, or coming up with a better (more specific) idea of how to make the Reiser4 plugin system acceptable to kernel maintainers without having to eat Ramen for a few years. Understand that I'm putting on my devil's advocate hat right now. I'd love to see Reiser4 merged tomorrow, or a week from now, exactly as it's written today, but I just don't see it happening. I'd also love to get more technical, but I just don't know the Reiser4 internals well enough to understand the feasibility (or not) of any of my vague ideas.
Re: metadata plugins (was Re: the 'official' point of view expressed by kernelnewbies.org regarding reiser4 inclusion)
Hans Reiser wrote: As for this we are all too grand to be bothered with money to feed our families business, building a system in which those who contribute can find a way to be rewarded is what managers do. Free software programmers may be willing to live on less than others, but they cannot live on nothing, and code that does not ever ship means living on nothing. Using guilt as an argument in a technical discussion is a flashing red sign that says I have no technical rebuttal If reiser4 is delayed enough, for reasons that have nothing to do with its needs, and without it having encumbered anyone else, it won't be ahead of the other filesystems when it ships. What delay? This is open source. There is nothing stopping you from worldwide distribution of the coolest Linux filesystem in the world. Jeff
