Re: Thinking outside the box on file systems
* Date: Sun, 19 Aug 2007 03:57:58 +0100 >> don't forget the ACPI interpreter. > > YAProof that bogons follow Boze statistics... or bugons, then. Why big minds didn't do rdev-like binary patching of the kernel image with binary ACPI data? Getting such data in (any) userspace would be the only thing in the install process of a modern distro, otherwise it's just a boring decompressing with copying. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
On 20 Aug 2007, Helge Hafting stated: > Shooting down bad ideas saves tremendous amounts of work, > killing an idea at the discussion stage means the idea never > got to the much more labor-intensive implementation stage. > > This don't mean that all new ideas are killed, only the bad ones. Even the good ones pretty much never start out perfect[1]. It always takes some criticism and fixing before the code is right, let alone the design. (Of course part of the problem with this idea is that it was so vague that the only problems there *could* be with it were huge gaping ones: you can't have subtle problems with an idea with no subtle elements. Unfortunately it had a lot of those, and fixing them without junking the whole idea would be hard. No, I haven't thought about how to do it: Marc might like to, though, what with it being his idea and all.) [1] except mingo's. mingo has the Perfect Design Mojo. How this came to pass, mere mortals may not speculate. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [OT] Re: Thinking outside the box on file systems
On Mon, 2007-08-20 at 09:21 -0700, Randy Dunlap wrote: > so enlighten us. What editor do you use/suggest/push? Corbet ! - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[OT] Re: Thinking outside the box on file systems
On Mon, 20 Aug 2007 04:18:21 -0700 (PDT) Marc Perkel wrote: > My rant on VI is to make a point. That point being > that when you use an editor that totally sucks then > it's going to cause you to write code that sucks. It > going to lower your standards. It's going to create a > culture where poorly done work is considered > acceptable. When you use an editor as poor as vi then > the idea that rm * doesn't work becomes acceptable and > justifiable, as demonstrated here by people who > ACTUALLY DEFENDED IT. so enlighten us. What editor do you use/suggest/push? --- ~Randy *** Remember to use Documentation/SubmitChecklist when testing your code *** - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
Marc Perkel wrote: What's the point? People are openly hostile to new ideas here. I started out nice and laid out my ideas and you have a bunch of morons who attack anything new. People here are not hostile to any new idea. They are generally hostile to anyone who suggest some "improvement" that he wants others to actually implement. Because that is acting like a boss, and you're not the boss here. Also - what do you expect when you bring an idea and say "well, don't look at what might go wrong, that's just limitations of the existing parts of linux?" Sure, everybody can overlook how your ideas don't fit into current linux - but then they have nothing more to say. What people here do is to look at new ideas/patches, and point out problems with them. When there are no more problems left, the patch is applied and linux improves. If we're not to point out problems with your ideas, then they can't be used in linux. That is not hostility, but how linux development works. Mostly people try to save you from doing unnecessary work - so you don't go and implement a flawed filesystem that will be rejected immediately when you show up with the patch. Shooting down bad ideas saves tremendous amounts of work, killing an idea at the discussion stage means the idea never got to the much more labor-intensive implementation stage. This don't mean that all new ideas are killed, only the bad ones. If you want your ideas accepted, you'll have to come up with something that isn't flawed, that is well planned, not just a bunch of "well - we could do *that* perhaps" but then it turns out that *that* idea was flawed as well. Helge Hafting Helge Hafting - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
On Mon, Aug 20, 2007 at 04:18:21AM -0700, Marc Perkel wrote: > Look at the reality of the situation. Linux is free > and yey it can't compete with operating systems that > are paid for. Maybe the reason is that when someone > point out the something is broken all yopu get is > justification and excuses and insults. And when someone points out windows is broken, Microsoft doesn't give a damn and you can't do a thing about it. > Read the thread from the beginning and you'll see that > I started out that way. > > If you attack people who are pointing out flaws and > making suggestions then people will stop pointing out > flaws and making suggestions. Just because you think they are flaws doesn't mean they are. Your complaint of rm * certainly fits that. You just didn't know what you were doing or you would have realized rm * was the wrong thing to use. > Think about it. Why did it take 20 years for Linux to > fix the RM problem? If you type RM * you expect the > files to be gone, not some stupid error that I'm > trying to delete too many files. There is no problem with rm *. The wildcard expansion is done by the shell, and different shells have different maximum command line lenghts (it would be pretty inefficient to allow infinite command lines). Windows makes the stupid choice that every single program has to invent and support wildcard expansion on its own. This means most commands on windows don't support wildcards at all. At least doing it in the shell makes unix consistent and much more flexible for the user. In a few cases you may have a wildcard that expands to too much. Well that is why we have find and xargs and such. It was a known limitation, and an efficient solution was invented to deal with the unusual case of too many files matching a wildcard. Not a problem, nothing to fix. > So who's fault is that? I say it's a problem with > Linux culture. If something is broken you have to > justify it instead of fixing it. The user was at fault for doing something stupid and not using the right tools for the job. > If developers take that kind of attitude then progress > stops. You prefer the developer telling you that you have to not use wildcards and type all filesnames manually (as windows would for most commands), or that instead they make the system allocate all system ram for your command line just in case you decide you should delete 2 billion files at once with rm *? > You guys are trying to may the RM problem MY FAULT > because I didn't say it nicely. We'll it doesn't have > to be said nicely. If something is broken then it > needs fixed regardless of who and how it is pointed > out. > > A BUG is a BUG is a BUG. You fix bugs, not make > excuses and try to explain it away. If you went up to > any computer user and asked them if when they type "rm > *" that they expect the files to be deleted they will > say "yes". Yet in the Linux work the command doesn't > work. And it's not like it breaks after MILLIONS of > files. It breaks on just a few thousand files, if > that. Not a bug. A design decision. A sensible one at that, and one to which a workaround exists (which happens to be faster than rm * too). -- Len Sorensen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
On Sat, Aug 18, 2007 at 09:07:05PM -0700, Marc Perkel wrote: > No Al, there isn't any shortage of arrogance here. Yes you provide plenty yourself. > Let me try to repeat what I'm talking about as simply > as I can. > > First - I'm describing a kind of functionality and > suggesting Linux should have it. I know a lot of it > can be done because much of what I'm suggesting is > already working in Windows and Netware. And how many limitations does windows and netware have that linux doesn't have, exactly due to that special functionality? part of the usefulness of permissions on unix is that they are so simple to understand and manage. They are really hard to screw up. The permission systems on windows and netware are much more complex and much hardware for people to understand, and hence much easier to get wrong causing potential security problems on your system without you realizing it. > I'm not the one who's going to code it. I'm just > saying that it would be nice if Linux had the > functionality of other operating systems - and - take > it to the next level - match it and do even better. Many of the bugs of other operating systems would be nice to avoid. In some cases what you may see as a feature is a major cause of problems especially performance problems. > As to thinking outside the box, what I'm proposing is > outside the box relative to Linux. It's not as > original as compared to Windows or Netware which is > even better. How about dropping that stupid phrase. > The idea is that Linux is lacking features that other > OSs have. What I'm suggesting is that Linux not only > match it but to create an even more powerful rights > layer that is more powerful than the rest and I'm > outlining a concept in the hopes that people would get > excited about the concept and want to build on the > idea. Well so far your system has been shown to be very inefficient (with no solution to that proposed by anyone). Many of the ideas have been shown to have been tried in the past and then abandoned due to not working very well or having severe scalability problems. There are lots of people (very often people currently in university) who suffer from the problem of believing anything they think of that isn't in current use must be a new unique idea that must be used because no one else had thought of it. In almost every case it is not a new idea, and the reason it is not in use is because it didn't work the last time someone thought of it and tried it. Or in some cases it is in use but just not in a field the new genious has any knowledge of. > I'm just telling you what I'd like to see. I'm not > going to code it. So I'm only going to talk about what > is possible. How it's done will be up to any > programmers who might be inspired by the idea. If no > one is inspired the Linux will continue to be in last > place when it comes to file system features relating > to fine grain permissions. How do you know what is possible? Could it be that many of the problems on netware and windows are caused by their choice of filesystem design? > In Linux, for example, users are allowed to delete > files that they are prohibited from reading or > writing. In Netware if a user can't read or write to > the file they won't even be able to see that the file > exists, let alone delete it. In some cases it makes sense. After all you are not deleting the file, you are delting one hardlink to it and the hardlink is part of the directory. If other directories have hardlinks to the same file, those get to stay around. Hence the permissions make sense. > In Netware I can move a directory tree into another > tree and the objects that have rights in the other > tree will have rights to all the new files without > having to run utilities on the command line to > recursively change the permission afterwards. Does netware have hardlinks? Does it have chroots? bind mounts? How about the ability to delete open files without causing problems for programs that have the files open? > The point - Linux isn't going to move forward and > catch up unless there is a fundamental change in the > thinking behind Linux permissions. There is a > cultural lack of innovation here. I discussed this > with Andrew Morton and he made some suggestions but > there's real hostility towards new concepts here. > Something I don't understand. At some point Linux > needs to grow beyond just being an evolved Unix clone > and that's not going to happen if you don't think > differently. Linux seems to be gaining ground in usage. Netware on the other hand seems just about dead. What does that say about things? > I still believe that the VI editor causes brain > damage. :) You should see what word and notepad do to people. -- Len Sorensen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ a
Re: Thinking outside the box on file systems
Marc Perkel wrote: No Al, there isn't any shortage of arrogance here. Let me try to repeat what I'm talking about as simply as I can. First - I'm describing a kind of functionality and suggesting Linux should have it. I know a lot of it can be done because much of what I'm suggesting is already working in Windows and Netware. Let's not get confused here... ._I_ am suggesting a feature that windows and netware have done already, that being inheritable acls. YOU suggested that permissions be configurable based on pattern matching on the name of the file. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
Kyle Moffett wrote: One last comment: 50ms to update in-memory dentries would be FRIGGING TERRIBLE!!! Using Perl, an interpreted language, the following script takes 3.39s to run on one of my lower-end systems: for (0 .. 1) { mkdir "a-$_"; mkdir "b-$_"; rename "a-$_", "b-$_"; } It's not even deleting things afterwards so it's populating a directory with ten thousand entries. We can easily calculate 10,000/3.39 = 2,949 entries per second, or 0.339 milliseconds per entry. When I change it to rmdir things instead, the runtime goes down to 2.89s == 3460 entries/sec == 0.289 milliseconds per entry. If such a scheme even increases the overhead of a directory rename by a hundredth of a millisecond on that box it would easily be a 2-3% performance hit. Given that people tend to kill for 1% performance boosts, that's not likely to be a good idea. The question is how many dentries are cached at the time? And it looks like you are just renaming, not moving, so there would be no need to recompute the acls at all. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
On Mon, 20 Aug 2007, Marc Perkel wrote: > [Snipped...] > > What's the point? People are openly hostile to new > ideas here. I started out nice and laid out my ideas > and you have a bunch of morons who attack anything > new. [Snopped...] > > Marc Perkel > Junk Email Filter dot com > http://www.junkemailfilter.com > No. You started out by saying you have this great idea that you want somebody else to develop. This met with strong resistance of course, because linux-kernel developers don't work for you. If you were to create some new "thinking out-of-the-box" file system, you could include it into Linux just as all the other file-systems have been included (it started with minix). If it was truly useful, it might even get included into the main-line distributions if you maintained it. People are not openly hostile to new ideas. They are openly hostile to being abused. Cheers, Dick Johnson Penguin : Linux version 2.6.22.1 on an i686 machine (5588.29 BogoMips). My book : http://www.AbominableFirebug.com/ _ The information transmitted in this message is confidential and may be privileged. Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited. If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to [EMAIL PROTECTED] - and destroy all copies of this information, including any attachments, without reading or disclosing them. Thank you. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
Hi Marc What's the point? People are openly hostile to new ideas here. I started out nice and laid out my ideas and you have a bunch of morons who attack anything new. If you think using subjects like "Thinking out of the box" (implicitely calling everybody else narrow-minded) and "vi causes brain damage" is starting out nice, you also got a serious communication problem. Look at the reality of the situation. Linux is free and yey it can't compete with operating systems that are paid for. Maybe the reason is that when someone point out the something is broken all yopu get is justification and excuses and insults. Funny, even Microsoft acknoledges that Linux can very well compete, as does your beloved Novell that just recently bought Suse. Maybe you haven't noticed yet that you're the only one left that thinks Linux can't compete. Think about it. Why did it take 20 years for Linux to fix the RM problem? If you type RM * you expect the files to be gone, not some stupid error that I'm trying to delete too many files. Well, it's not a stupid error, this is called a limit. Other people have already explained to you how the UNIX shell works, so I'm not going to repeat it again. That said, I even would admit that I have been bitten by this limit before (deleting a few thousand bounced mail in a spool directory), So who's fault is that? I say it's a problem with Linux culture. If something is broken you have to justify it instead of fixing it. I use Linux since the mid 90's and remember thousands and thousands of bugs fixed and limits removed. But you must be here longer and have the better view of how "the Linux culture" really works. You guys are trying to may the RM problem MY FAULT because I didn't say it nicely. We'll it doesn't have to be said nicely. If something is broken then it needs fixed regardless of who and how it is pointed out. Nobody denied the limit, you were just pointed out that you don't have a fucking clue what the behavior actually means and where the limit lies. And calling other people brain-damaged at the same time... > So wat does it tell you when something like this is left broken for so long? What it tells me is that the development process is broken. Well, it tells _me_: - It is a limit and not a bug - The limit is not severe, not many people constantly have to delete millions of files in the directory without deleting the directory itself - the limit can be worked around by "find . |xargs \rm" But, as you proved again and again, you're the expert. My rant on VI is to make a point. That point being that when you use an editor that totally sucks then it's going to cause you to write code that sucks. It going to lower your standards. It's going to create a culture where poorly done work is considered acceptable. When you use an editor as poor as vi then the idea that rm * doesn't work becomes acceptable and justifiable, as demonstrated here by people who ACTUALLY DEFENDED IT. You might have wanted to make this point. But all you really showed is that you're an arrogant, ignorant loudmouth, takling about things you have no clue about. I bet you haven't written a single line of decent code in your life. Kind regards Tim - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
--- Brennan Ashton <[EMAIL PROTECTED]> wrote: > While i highly support innovation, until i see a > well layed out > structure of what exactly you are looking for i have > a hard time > expressing any view that are meaningful, could you > create some kind of > wiki or summery email (if this is really this > important to you) most > of us are lazy and have better things to do, make it > easy for us. > A) Create a list of the current problems or just > inefficiencies in > the current system. > B) Create a list of all the points that make up > your view of a good > file system. > C) Cross the two lists showing how your idea would > fix the current problems. > > I am not saying that the current way is the right or > wrong way, just > that i think you have organised your ideas as if you > are thinking out > loud by email (which is ok by me, just stop the > direct attacks if you > are). > I agree that every company and program gets caught > in a rut, that does > not conform to changing markets and technology, > especially if it was > at one time a success, IBM, Microsoft, Apple, Sun, > American Auto > Industry to name a few. These are also example > companies that have > gotten the idea that what they do might not be right > way and have made > some attempt to step back (some more successfully > than others) or face > loss in market share. > Just remember a key point when rethinking something > as key as a file > system, while your new battery may be much more > efficient than my old > AA, i want it to work with my old flashlights too > (and no aftermarket > refit kit). > Should the surroundings be modified for the target, > or the target > modified for the surroundings? > Your little rants about VI and rm are not helpful, > if these programs > were so bad then why have they survived. Linux is on > hell of a project > to be put together, sorry but innovation did come > from people using VI > and Emacs. btw i highly recommend the command man, > you should try it. > -- > Brennan Ashton > Bellingham, Washington > > "The box said, 'Requires Windows 98 or better'. So I > installed Linux" > What's the point? People are openly hostile to new ideas here. I started out nice and laid out my ideas and you have a bunch of morons who attack anything new. At least finally someone fixed the RM problem. Look at the reality of the situation. Linux is free and yey it can't compete with operating systems that are paid for. Maybe the reason is that when someone point out the something is broken all yopu get is justification and excuses and insults. Read the thread from the beginning and you'll see that I started out that way. If you attack people who are pointing out flaws and making suggestions then people will stop pointing out flaws and making suggestions. Think about it. Why did it take 20 years for Linux to fix the RM problem? If you type RM * you expect the files to be gone, not some stupid error that I'm trying to delete too many files. So who's fault is that? I say it's a problem with Linux culture. If something is broken you have to justify it instead of fixing it. If developers take that kind of attitude then progress stops. You guys are trying to may the RM problem MY FAULT because I didn't say it nicely. We'll it doesn't have to be said nicely. If something is broken then it needs fixed regardless of who and how it is pointed out. A BUG is a BUG is a BUG. You fix bugs, not make excuses and try to explain it away. If you went up to any computer user and asked them if when they type "rm *" that they expect the files to be deleted they will say "yes". Yet in the Linux work the command doesn't work. And it's not like it breaks after MILLIONS of files. It breaks on just a few thousand files, if that. So wat does it tell you when something like this is left broken for so long? What it tells me is that the development process is broken. My rant on VI is to make a point. That point being that when you use an editor that totally sucks then it's going to cause you to write code that sucks. It going to lower your standards. It's going to create a culture where poorly done work is considered acceptable. When you use an editor as poor as vi then the idea that rm * doesn't work becomes acceptable and justifiable, as demonstrated here by people who ACTUALLY DEFENDED IT. Marc Perkel Junk Email Filter dot com http://www.junkemailfilter.com Shape Yahoo! in your own image. Join our Network Research Panel today! http://surveylink.yahoo.com/gmrs/yahoo_panel_invite.asp?a=7 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
While i highly support innovation, until i see a well layed out structure of what exactly you are looking for i have a hard time expressing any view that are meaningful, could you create some kind of wiki or summery email (if this is really this important to you) most of us are lazy and have better things to do, make it easy for us. A) Create a list of the current problems or just inefficiencies in the current system. B) Create a list of all the points that make up your view of a good file system. C) Cross the two lists showing how your idea would fix the current problems. I am not saying that the current way is the right or wrong way, just that i think you have organised your ideas as if you are thinking out loud by email (which is ok by me, just stop the direct attacks if you are). I agree that every company and program gets caught in a rut, that does not conform to changing markets and technology, especially if it was at one time a success, IBM, Microsoft, Apple, Sun, American Auto Industry to name a few. These are also example companies that have gotten the idea that what they do might not be right way and have made some attempt to step back (some more successfully than others) or face loss in market share. Just remember a key point when rethinking something as key as a file system, while your new battery may be much more efficient than my old AA, i want it to work with my old flashlights too (and no aftermarket refit kit). Should the surroundings be modified for the target, or the target modified for the surroundings? Your little rants about VI and rm are not helpful, if these programs were so bad then why have they survived. Linux is on hell of a project to be put together, sorry but innovation did come from people using VI and Emacs. btw i highly recommend the command man, you should try it. -- Brennan Ashton Bellingham, Washington "The box said, 'Requires Windows 98 or better'. So I installed Linux" - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
[Lurker delurking here: I've seen Marc pull this sort of trick on multiple mailing lists now and I've had enough. Coming up with wild ideas, thinking they must be ideas nobody else has ever considered, and never thinking them through for five minutes is sort of Marc's hallmark, I'm afraid.] On 19 Aug 2007, Marc Perkel uttered the following: > No Al, there isn't any shortage of arrogance here. Actually Al's being unusually nice to you. A regular would have had a few lines of `you must be kidding' but he's trying to get you to see *why* you are wrong. (It's not working.) > Let me try to repeat what I'm talking about as simply > as I can. Precision is better. > First - I'm describing a kind of functionality and > suggesting Linux should have it. I know a lot of it > can be done because much of what I'm suggesting is > already working in Windows and Netware. Windows and Netware don't have multiple namespaces to cope with (e.g. chroot()); nor do they have the ability to detach parts of their filesystem tree completely from other parts (so they're not even connected at the root). Their VFSes are much less capable than Linux's. ACLs provide some of the inheritability you want anyway. In regards to the general idea of doing permission checking in userspace: that works to some extent --- see FUSE --- but you really *cannot* allow the entity who specifies permissions check semantics on the filesystem to have less privileges than the entity who mounted the filesystem in the first place. It's safe for that entity to specify such things because it can read the device the filesystem is mounted from anyway, thus overriding all permissions if it wants to: but any entity of lesser privilege could use this as a really cheap privilege escalation attack. (This is fundamental and unavoidable and no amount of thinking outside any sort of box will fix it.) > I'm not the one who's going to code it. I'm just Might I suggest then that coming here and bloviating is not the right approach? In the Linux world there are two ways to proceed: doing the work yourself or paying someone to. You can sometimes get lucky and find that someone else has already done what you were thinking of, but it is vanishingly rare to have an idea so cool that someone else catches fire and does all the work for you. In fact I can't recall ever seeing it happen, although I suppose it might among longtime collaborators some of whom are very overloaded with other things. So you could make this a mount option, I suppose, but nothing more pluggable than that. (I don't see how your userspace permissions layer will avoid calling back into the VFS, either: take care to avoid infloops when you code it.) > saying that it would be nice if Linux had the > functionality of other operating systems - and - take > it to the next level - match it and do even better. `The next level'? What `next level'? > As to thinking outside the box, what I'm proposing is > outside the box relative to Linux. It's not as > original as compared to Windows or Netware which is > even better. Actually neither of those OSes provide regex-based permissions checking that I know of. It's not a fundamentally silly idea but if you'd been paying even the slightest attention to this list or one of its summaries (like LWN's) over the last, oh, three years you'd have noticed the difficulty winning people over that security modules that do depend on name matching are experiencing. And AppArmor has clear semantics, working code and actual users, none of which your idea has got. (And in regard to your idea being so startlingly original in the Linux world, well, you're not quite a decade behind the times.) > The idea is that Linux is lacking features that other > OSs have. What I'm suggesting is that Linux not only > match it but to create an even more powerful rights > layer that is more powerful than the rest and I'm I see no sign of significant extra power in what you've mentioned so far: only extra ways to create security holes and/or infinite regressions. > outlining a concept in the hopes that people would get > excited about the concept and want to build on the > idea. It's not working. > I'm just telling you what I'd like to see. I'm not > going to code it. So I'm only going to talk about what > is possible. Anyone on this list is capable enough to have thought of this one themselves, and generally to have seen the holes as well. You don't even seem to have *looked* for holes, which is strange given that this is pretty much the most important job of anyone who's just thought of a new idea. > one is inspired the Linux will continue to be in last > place when it comes to file system features relating > to fine grain permissions. Yay, very crude moral blackmail (well, it would be moral blackmail if what you were saying was in any way persuasive). Way to inspire people! > In Linux, for example, users are allowed to delete > files that they are prohibited from reading or
Re: Thinking outside the box on file systems
No Al, there isn't any shortage of arrogance here. Let me try to repeat what I'm talking about as simply as I can. First - I'm describing a kind of functionality and suggesting Linux should have it. I know a lot of it can be done because much of what I'm suggesting is already working in Windows and Netware. I'm not the one who's going to code it. I'm just saying that it would be nice if Linux had the functionality of other operating systems - and - take it to the next level - match it and do even better. As to thinking outside the box, what I'm proposing is outside the box relative to Linux. It's not as original as compared to Windows or Netware which is even better. The idea is that Linux is lacking features that other OSs have. What I'm suggesting is that Linux not only match it but to create an even more powerful rights layer that is more powerful than the rest and I'm outlining a concept in the hopes that people would get excited about the concept and want to build on the idea. I'm just telling you what I'd like to see. I'm not going to code it. So I'm only going to talk about what is possible. How it's done will be up to any programmers who might be inspired by the idea. If no one is inspired the Linux will continue to be in last place when it comes to file system features relating to fine grain permissions. In Linux, for example, users are allowed to delete files that they are prohibited from reading or writing. In Netware if a user can't read or write to the file they won't even be able to see that the file exists, let alone delete it. In Netware I can move a directory tree into another tree and the objects that have rights in the other tree will have rights to all the new files without having to run utilities on the command line to recursively change the permission afterwards. The point - Linux isn't going to move forward and catch up unless there is a fundamental change in the thinking behind Linux permissions. There is a cultural lack of innovation here. I discussed this with Andrew Morton and he made some suggestions but there's real hostility towards new concepts here. Something I don't understand. At some point Linux needs to grow beyond just being an evolved Unix clone and that's not going to happen if you don't think differently. I still believe that the VI editor causes brain damage. :) Marc Perkel Junk Email Filter dot com http://www.junkemailfilter.com Park yourself in front of a world of choices in alternative vehicles. Visit the Yahoo! Auto Green Center. http://autos.yahoo.com/green_center/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
On Sat, Aug 18, 2007 at 07:03:06PM -0700, [EMAIL PROTECTED] wrote: > >>I suspect you will find it somewhat hard to convince *anybody* on > >>this list to put either a regex engine or a Perl interpreter into the > >>kernel. I doubt you could even get a simple shell-style pattern > >>matcher in. First of all, both of the former chew up enormous gobs > >>of stack space *AND* they're NP-complete. Eh? regex via NFA is O(expression size * string length) time and O(expression size) space. If you can show that regex matching is NP-complete, you've got a good shot at Nevanlinna Prize... Not that it made regex in kernel a good idea, but fair is fair - unless you can show any mentioning of backrefs upthread...[1] > You just can't do such > >>matching even in polynomial time, let alone something that scales > >>appropriately for an OS kernel like, say, O(log(n)). > > > >Already been done. Take a look at "AppArmor" aka "Immunix". > > don't forget the ACPI interpreter. YAProof that bogons follow Boze statistics... - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
On Sat, 18 Aug 2007, Alan wrote: On Wed, 2007-08-15 at 13:22 -0400, Kyle Moffett wrote: On Aug 15, 2007, at 13:09:31, Marc Perkel wrote: The idea is that people have permissions - not files. By people I mean users, groups, managers, applications etc. One might even specify that there are no permission restrictions at all. Part of the process would be that the kernel load what code it will use for the permission system. It might even be a little perl script you write. Also - you aren't even giving permission to access files. It's permission to access name patterns. One could apply REGEX masks to names to determine permissions. So if you have permission to the name you have permission to the file. Please excuse me, I'm going to go stand over in the corner for a minute. *hahahahahaa hahahahahaaa hahaa hoo hee snicker sniff* *wanders back into the conversation* Sorry about that, pardon me. I suspect you will find it somewhat hard to convince *anybody* on this list to put either a regex engine or a Perl interpreter into the kernel. I doubt you could even get a simple shell-style pattern matcher in. First of all, both of the former chew up enormous gobs of stack space *AND* they're NP-complete. You just can't do such matching even in polynomial time, let alone something that scales appropriately for an OS kernel like, say, O(log(n)). Already been done. Take a look at "AppArmor" aka "Immunix". don't forget the ACPI interpreter. David Lang - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
On Wed, 2007-08-15 at 10:34 -0700, Marc Perkel wrote: > Keep in mind that this is about thinking outside the > box. Don't let new ideas scare you. My cat thinks outside the box all the time. Cleaning it up is a real pain. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
On Wed, 2007-08-15 at 13:22 -0400, Kyle Moffett wrote: > On Aug 15, 2007, at 13:09:31, Marc Perkel wrote: > > The idea is that people have permissions - not files. By people I > > mean users, groups, managers, applications > > etc. One might even specify that there are no permission > > restrictions at all. Part of the process would be that the kernel > > load what code it will use for the permission system. It might even > > be a little perl script you write. > > > > Also - you aren't even giving permission to access files. It's > > permission to access name patterns. One could apply REGEX masks to > > names to determine permissions. So if you have permission to the > > name you have permission to the file. > > Please excuse me, I'm going to go stand over in the corner for a minute. > > *hahahahahaa hahahahahaaa hahaa hoo hee snicker sniff* > > *wanders back into the conversation* > > Sorry about that, pardon me. > > I suspect you will find it somewhat hard to convince *anybody* on > this list to put either a regex engine or a Perl interpreter into the > kernel. I doubt you could even get a simple shell-style pattern > matcher in. First of all, both of the former chew up enormous gobs > of stack space *AND* they're NP-complete. You just can't do such > matching even in polynomial time, let alone something that scales > appropriately for an OS kernel like, say, O(log(n)). Already been done. Take a look at "AppArmor" aka "Immunix". - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
On Sat, Aug 18, 2007 at 09:45:54AM -0700, Marc Perkel wrote: > Linux isn't going to make progress when people try to > figure out how to make something NOT work rather than > to make something work. So if you are going to put > effort into this then why not try to figure out how to > get around the issues you are raising rather than to > attack the idea as unsolvable. It's your idea; _you_ get to defend it against the problems found by reviewers. And whining about negativity is the wrong way to do that. Look at it that way: there is science and there is feel-good woo. The former depends on peer review. The latter depends on not having it and vague handwaving is the classical way of avoiding it. So are the claims of being a "visionary" and accusing critics of being uncooperative reactionaries conspiring against the progress. So far you are doing very poorly; if you want somebody else to join you in experimenting with these ideas, you are acting in a very inefficient way (and if you don't want anybody else, you'll obviously have to deal with details yourself anyway). Asserting that critics should patch the holes in your handwaving is unlikely to impress anybody; arrogance is not in short supply around here and yours is not even original. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
--- Kyle Moffett <[EMAIL PROTECTED]> wrote: > On Aug 17, 2007, at 15:01:48, Phillip Susi wrote: > > [EMAIL PROTECTED] wrote: > >> It will become even *more* of a "not that common" > if the lock will > >> block moves and ACL changes *across the > filesystem* for > >> potentially *minutes* at a time. > > > > It will not take anywhere NEAR minutes at a time > to update the in > > memory dentries, more like 50ms. > > One last comment: > > 50ms to update in-memory dentries would be FRIGGING > TERRIBLE!!! > Using Perl, an interpreted language, the following > script takes 3.39s > to run on one of my lower-end systems: > > for (0 .. 1) { > mkdir "a-$_"; > mkdir "b-$_"; > rename "a-$_", "b-$_"; > } > > It's not even deleting things afterwards so it's > populating a > directory with ten thousand entries. We can easily > calculate > 10,000/3.39 = 2,949 entries per second, or 0.339 > milliseconds per entry. > > When I change it to rmdir things instead, the > runtime goes down to > 2.89s == 3460 entries/sec == 0.289 milliseconds per > entry. > > If such a scheme even increases the overhead of a > directory rename by > a hundredth of a millisecond on that box it would > easily be a 2-3% > performance hit. Given that people tend to kill for > 1% performance > boosts, that's not likely to be a good idea. > > Cheers, > Kyle Moffett > > What I suggested was a concept of a new way to look at a file system. What you are arguing here is why it wouldn't work based on your theories as to how such a file system would be implemented. In attacking how slow you think it might be you are making assumptions that wouldn't apply to how this would be implemented. You are assuming that it would be implemented in ways that you are familiar with. That is a wrong assumption. Linux isn't going to make progress when people try to figure out how to make something NOT work rather than to make something work. So if you are going to put effort into this then why not try to figure out how to get around the issues you are raising rather than to attack the idea as unsolvable. When I originally suggested that the names would be a "hash" I didn't mean that it is going to be only a hash. You have successfully argued that just a hash would have problems. Which means that a real solution is going to be more complex. I suggest that it would be easier to figure out how to make moves of large directory structure fast and effecient with automatic inheritance of rights. I know it can be done because Microsoft is doing it and Novell Netware was doing it 20 years ago. So the fact that it is done by others disproves your arguments that it can't be done. Marc Perkel Junk Email Filter dot com http://www.junkemailfilter.com Moody friends. Drama queens. Your life? Nope! - their life, your story. Play Sims Stories at Yahoo! Games. http://sims.yahoo.com/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
On Aug 17, 2007, at 15:01:48, Phillip Susi wrote: [EMAIL PROTECTED] wrote: It will become even *more* of a "not that common" if the lock will block moves and ACL changes *across the filesystem* for potentially *minutes* at a time. It will not take anywhere NEAR minutes at a time to update the in memory dentries, more like 50ms. One last comment: 50ms to update in-memory dentries would be FRIGGING TERRIBLE!!! Using Perl, an interpreted language, the following script takes 3.39s to run on one of my lower-end systems: for (0 .. 1) { mkdir "a-$_"; mkdir "b-$_"; rename "a-$_", "b-$_"; } It's not even deleting things afterwards so it's populating a directory with ten thousand entries. We can easily calculate 10,000/3.39 = 2,949 entries per second, or 0.339 milliseconds per entry. When I change it to rmdir things instead, the runtime goes down to 2.89s == 3460 entries/sec == 0.289 milliseconds per entry. If such a scheme even increases the overhead of a directory rename by a hundredth of a millisecond on that box it would easily be a 2-3% performance hit. Given that people tend to kill for 1% performance boosts, that's not likely to be a good idea. Cheers, Kyle Moffett - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
[EMAIL PROTECTED] wrote: I suspect Kyle is not quite correct - it's probably the case that you don't have to consider just the in-memory dentries, but *all* the descendent objects in the entire file system. If you have a clever proof that on-disk can't *possibly* be affected, feel free to present it. Why would you have to consider the descendent entries on disk when you are only changing an entry in the parent? The effects of that change are only computed in memory when the dentry for a child is created, so you don't have to do a bunch of disk churning to change permissions on the whole tree. In fact, all of the children may very well have NO acl of their own stored on disk, which also saves space. The whole idea here is that there is ONE acl that applies to the whole tree, rather than have every object in the tree have its own acl. That's why every object in the tree on the disk is not effected by a change. It will become even *more* of a "not that common" if the lock will block moves and ACL changes *across the filesystem* for potentially *minutes* at a time. It will not take anywhere NEAR minutes at a time to update the in memory dentries, more like 50ms. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
On Fri, 17 Aug 2007 11:19:21 EDT, Phillip Susi said: > Kyle Moffett wrote: >> Problem 1: "updating cached acls of descendent objects": How do you >> find out what a 'descendent object' is? Answer: You can't without >> recursing through the entire in-memory dentry tree. I suspect Kyle is not quite correct - it's probably the case that you don't have to consider just the in-memory dentries, but *all* the descendent objects in the entire file system. If you have a clever proof that on-disk can't *possibly* be affected, feel free to present it. (Does anybody know offhand what means 'chacl -r' uses to avoid race conditions with directories being moved in/out from under it, or does it just say "we'll make a best stab at it"?) > Yes, it would take some cpu time, and yes, it would have to use a lock > to protect the acl which would also lock out moves. Is that such a high > cost? Changing acls and moving whole directory trees around is not THAT > common of an operation... if it takes a wee bit more cpu time, I doubt > anyone will complain. It will become even *more* of a "not that common" if the lock will block moves and ACL changes *across the filesystem* for potentially *minutes* at a time. pgpA4iSi6Zhsd.pgp Description: PGP signature
Re: Thinking outside the box on file systems
Kyle Moffett wrote: Problem 1: "updating cached acls of descendent objects": How do you find out what a 'descendent object' is? Answer: You can't without recursing through the entire in-memory dentry tree. Such recursion is lock-intensive and has poor performance. Furthermore, you have to do the entire recursion as an atomic operation; other cross-directory renames or ACL changes would invalidate your results halfway through and cause race conditions. Yes, it would take some cpu time, and yes, it would have to use a lock to protect the acl which would also lock out moves. Is that such a high cost? Changing acls and moving whole directory trees around is not THAT common of an operation... if it takes a wee bit more cpu time, I doubt anyone will complain. Oh, and by the way, the kernel has no real way to go from a dentry to a (process, fd) pair. That data simply is not maintained because it is unnecessary and inefficent to do so. Without that data you *can't* What would you need (process,fd) for? You just need to walk the tree of dentries and update them. determine what is "dependent". Furthermore, even if you could it still wouldn't work because you can't even tell which path the file was originally opened via. Say you run: mount --bind /mnt/cdrom /cdrom umount /mnt/cdrom Now any process which had a cwd or open directory handle in "/cdrom" is STILL USING THE ACLs from when it was mounted as "/mnt/cdrom". If you have the same volume bind-mounted in two places you can't easily distinguish between them. Caching permission data at the vfsmount won't even help you because you can move around vfsmounts as long as they are in subdirectories: mkdir -p /a/b/foo mount -t tmpfs tmpfs /a/b/foo mv /a/b /quux umount /quux/foo At this point you would also have to look at vfsmounts during your recursive traversal and update their cached ACLs too. Each bind mount point would need its own dentry so it could have its own acl and parent pointer. Problem 2: "Some other directory that you have a handle to": When you are given this relative path and this cwd ACL, how do you determine the total ACL of the parent directory: path: ../foo/bar cached cwd total-ACL: root rwx (inheritable) bob rwx (inheritable) somegroup rwx (inheritable) jane rwx ".." partial-ACL root +rwx (inheritable) somegroup +rx (inheritable) Answer: you can't. For example, if "/" had the permission 'root +rwx (inheritable)', and nothing else had subtractive permissions, then the "root +rwx (inheritable)" in the parent dir would be a no-op, but you can't tell that without storing a complete parent directory history. What? The total acl of the parent directory is in its dentry. Now assume that I "mkdir /foo && set-some-inheritable-acl-on /foo && mv /home /foo/home". Say I'm running all sorts of X apps and GIT and a number of other programs and have some conservative 5k FDs open on /home. This is actually something I've done before (without the ACLs), albeit accidentally. With your proposal, the kernel would first have to identify all of the thousands of FDs with cached ACL data across a very large cache-hot /home directory. For each FD, it would have to store an updated copy of the partial-ACL states down its entire path. Oh, and you can't do any other ACL or rename operations in the entire subtree while this is going on, because that would lead to the first update reporting incorrect results and racing with the second. You are also extremely slow, deadlock-prone, and memory hungry, since you have to take an enormous pile of dentry locks while doing the recursion. Nobody can even open files with relative paths while this is going on because the cached ACLs are in an intermediate and inconsistent state: they're updated but the directory isn't in its new position yet. Again, such a move would take some cpu time but the locks would not block file opens, just other moves and permission changes. I don't think this burden is terribly high for what is a relatively rare operation. This is completely opposite the way that permissions currently operate in Linux. When I am chrooted, I don't care about the permissions of *anything* outside of the chroot, because it simply doesn't exist. Yes, it is different... if it wasn't we wouldn't be talking about it. Furthermore you still don't answer the "computing ACL of parent directory requires lots of space" problem. What problem? No, you aren't getting it: YOUR CWD DOES NOT GO AWAY WHEN YOU MOVE IT OR UMOUNT -L IT. NEITHER DO OPEN DIRECTORY HANDLES. Sorry for yelling It effectively does if you chmod 000 it. Same idea. You yank permissions to cwd, then you can't access cwd anymore. but this is the crux of the point I am trying to make. Any permissions system which cannot handle a *completely* discontiguous filesystem space cannot work on Linux; end of story. The primary reason behind t
Re: Thinking outside the box on file systems
On Thu, 16 Aug 2007 21:24:22 PDT, Marc Perkel said: > And then everything that's stored as /1/2/3/4 is still > the same but the sections resolve to different names. At that point, you need to go re-think your full-pathname permission scheme, because that severely broke it. > I'm sure there are errors in my logic but I'm trying > to show that if you are persistent in trying to come > up with ideas on how to do something you will > eventually make it work. Not true - there are *large* fields where *no* amount of persistence will make it work - squaring a circle or trisecting an angle by geometric means, perpetual motion machines, and many broken-as-designed computer programs. > But if you are looking for > ways to make it not work then you probably won't solve > it. Right. It's not *our* job to solve your problem. It's *our* job to find ways to not make it work - those are called *bugs*. We try very hard to keep buggy code and ideas out of the kernel. So we make it difficult for bugs. > So the correct response to this message isn't to prove > that my method won't work, but to come up with a You're the one proposing something. *You* get to come up with something that works. It's *our* job to review and comment and point out problems. You want to get something to happen, you take the comments, and use them to improve the result. We point out every possible issue we can find (and maybe suggest ways to fix them). That's what code review is about. And if you think people like me and Kyle are rough on you, I suggest you go back in the archives and read what people like Al Viro and Chris Hellwig had to say about ReiserFS4. And *that* was not just a hand-wave like you're doing, that was an actual *working* filesystem. > method that will work. You have to look for a solution > rather than attack other people's solutions. No, *you* have to look for a solution that is good enough to withstand the combined scrutiny of everybody on the linux-kernel list. That's why Linux works, and why we're able to have several *million* lines of changes between 2.6.2N and 2.6.2(N+1), and only have several dozen things on the "known regressions" list. > That's what thinking outside the box means. > > Impossible = Challenge Good. Let us know when you come up with something that actually works. pgpc1ZQoTtFWc.pgp Description: PGP signature
Re: Thinking outside the box on file systems
Several people have asked about how to mass move a tree under my idea for a new kind of file system. I have an idea. Suppose you have the file name as follies. /one/two/three/four/file1 Except the are a million files in /four/ named file1 to file100. We want to move thes files to /seven/six/five. How do you do that fast? Here's an idea. Suppose you have a hash not only of files but of the directory sections. Every new section is added and givem a number. For simplicity the table might look like this: one 1 two 2 three 3 four 4 five 5 six 6 seven 7 So what the path is reduced to is /1/2/3/4 which point to those names. So if you want to move it then you change the names in the database. seven 1 six 2 five 3 And then everything that's stored as /1/2/3/4 is still the same but the sections resolve to different names. I'm sure there are errors in my logic but I'm trying to show that if you are persistent in trying to come up with ideas on how to do something you will eventually make it work. But if you are looking for ways to make it not work then you probably won't solve it. So the correct response to this message isn't to prove that my method won't work, but to come up with a method that will work. You have to look for a solution rather than attack other people's solutions. That's what thinking outside the box means. Impossible = Challenge Marc Perkel Junk Email Filter dot com http://www.junkemailfilter.com Park yourself in front of a world of choices in alternative vehicles. Visit the Yahoo! Auto Green Center. http://autos.yahoo.com/green_center/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
On Aug 16, 2007, at 11:09:16, Phillip Susi wrote: Kyle Moffett wrote: Let me repeat myself here: Algorithmically you fundamentally CANNOT implement inheritance-based ACLs without one of the following (although if you have some other algorithm in mind, I'm listening): (A) Some kind of recursive operation *every* time you change an inheritable permission (B) A unified "starting point" from which you begin *every* access-control lookup (or one "starting point" per useful semantic grouping, like a namespace). The "(A)" is presently done in userspace and that's what you want to avoid. As to (B), I will attempt to prove below that you cannot implement "(B)" without breaking existing assumptions and restricting a very nice VFS model. No recursion is needed because only one acl exists, so that is the only one you need to update. At least on disk. Any cached acls in memory of descendant objects would need updated, but the number of those should be relatively small. The starting point would be the directory you start the lookup from. That may be the root, or it may be some other directory that you have a handle to, and thus, already has its effective acl computed. Problem 1: "updating cached acls of descendent objects": How do you find out what a 'descendent object' is? Answer: You can't without recursing through the entire in-memory dentry tree. Such recursion is lock-intensive and has poor performance. Furthermore, you have to do the entire recursion as an atomic operation; other cross-directory renames or ACL changes would invalidate your results halfway through and cause race conditions. Oh, and by the way, the kernel has no real way to go from a dentry to a (process, fd) pair. That data simply is not maintained because it is unnecessary and inefficent to do so. Without that data you *can't* determine what is "dependent". Furthermore, even if you could it still wouldn't work because you can't even tell which path the file was originally opened via. Say you run: mount --bind /mnt/cdrom /cdrom umount /mnt/cdrom Now any process which had a cwd or open directory handle in "/cdrom" is STILL USING THE ACLs from when it was mounted as "/mnt/cdrom". If you have the same volume bind-mounted in two places you can't easily distinguish between them. Caching permission data at the vfsmount won't even help you because you can move around vfsmounts as long as they are in subdirectories: mkdir -p /a/b/foo mount -t tmpfs tmpfs /a/b/foo mv /a/b /quux umount /quux/foo At this point you would also have to look at vfsmounts during your recursive traversal and update their cached ACLs too. Problem 2: "Some other directory that you have a handle to": When you are given this relative path and this cwd ACL, how do you determine the total ACL of the parent directory: path: ../foo/bar cached cwd total-ACL: root rwx (inheritable) bob rwx (inheritable) somegroup rwx (inheritable) jane rwx ".." partial-ACL root +rwx (inheritable) somegroup +rx (inheritable) Answer: you can't. For example, if "/" had the permission 'root +rwx (inheritable)', and nothing else had subtractive permissions, then the "root +rwx (inheritable)" in the parent dir would be a no- op, but you can't tell that without storing a complete parent directory history. Now assume that I "mkdir /foo && set-some-inheritable-acl-on /foo && mv /home /foo/home". Say I'm running all sorts of X apps and GIT and a number of other programs and have some conservative 5k FDs open on / home. This is actually something I've done before (without the ACLs), albeit accidentally. With your proposal, the kernel would first have to identify all of the thousands of FDs with cached ACL data across a very large cache-hot /home directory. For each FD, it would have to store an updated copy of the partial-ACL states down its entire path. Oh, and you can't do any other ACL or rename operations in the entire subtree while this is going on, because that would lead to the first update reporting incorrect results and racing with the second. You are also extremely slow, deadlock-prone, and memory hungry, since you have to take an enormous pile of dentry locks while doing the recursion. Nobody can even open files with relative paths while this is going on because the cached ACLs are in an intermediate and inconsistent state: they're updated but the directory isn't in its new position yet. Unsolvable problems with each option: (1.a.I) You just broke all sorts of chrooted daemons. When I start bind in its chroot jail, it does the following: chdir("/private/bind9"); chroot("."); setgid(...); setuid(...); The "/private" directory is readable only by root, since root is the only one who will be navigating you into these chroots for any reason. You only switch UID/GID after the chroot() call, at which point you are i
Re: Thinking outside the box on file systems
Marc Perkel wrote: > Yep - way outside the box - and thus the title of the > thread. > > The idea is that people have permissions - not files. > By people I mean users, groups, managers, applications > etc. One might even specify that there are no > permission restrictions at all. Part of the process > would be that the kernel load what code it will use > for the permission system. It might even be a little > perl script you write. This isn't anything new. It is, in fact, described in many places. Permissions can, most generally, be described as a matrix of objects and security domains. This matrix is large and, generally, highly regular. If we slice the matrix up and associate each column with an object, we call it an "access control list". If we slice the matrix up and associate each row with a security domain, we call it a "capability." These can be, and often are, daisy-chained, so that an access control list can contain "all possessors of capability X", for example. Groups in Unix are, in fact, a form of capabilities. -hpa - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
[EMAIL PROTECTED] wrote: That's how it works *now*. That's *not* what happens with Marc's "Out of Box filesystem", because you need to deal with the fact that the pathnames to everything under it just changed. I think you cross jumped subthreads. I have gone off on a different topic now really and have been talking about acls being dynamically inherited by children, not regular expression permission matching. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
On Thu, 16 Aug 2007 13:28:26 EDT, Phillip Susi said: > [EMAIL PROTECTED] wrote: > > What happens if I do a 'mv /home /home1'? Looks like more than a > > "relatively > > small" number. A cold-cache 'find' takes a few minutes to wade through it > > all, > > so any solutions you come up with should beware of locking issues... > > Then the directory is moved one dentry on disk needs changed to > reflect the new name. That's how it works *now*. That's *not* what happens with Marc's "Out of Box filesystem", because you need to deal with the fact that the pathnames to everything under it just changed. pgpxtd8A2SQGM.pgp Description: PGP signature
Re: Thinking outside the box on file systems
[EMAIL PROTECTED] wrote: What happens if I do a 'mv /home /home1'? Looks like more than a "relatively small" number. A cold-cache 'find' takes a few minutes to wade through it all, so any solutions you come up with should beware of locking issues... Then the directory is moved one dentry on disk needs changed to reflect the new name. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
On Thu, 16 Aug 2007 11:09:16 EDT, Phillip Susi said: > No recursion is needed because only one acl exists, so that is the only > one you need to update. At least on disk. Any cached acls in memory of > descendant objects would need updated, but the number of those should be > relatively small. On my laptop (this is a *laptop*, mind you): % df -i /home FilesystemInodes IUsed IFree IUse% Mounted on /dev/mapper/VolGroup00-home 655360 532361 122999 82% /home What happens if I do a 'mv /home /home1'? Looks like more than a "relatively small" number. A cold-cache 'find' takes a few minutes to wade through it all, so any solutions you come up with should beware of locking issues... pgppZkwwTO2kI.pgp Description: PGP signature
Re: Thinking outside the box on file systems
Kyle Moffett wrote: Let me repeat myself here: Algorithmically you fundamentally CANNOT implement inheritance-based ACLs without one of the following (although if you have some other algorithm in mind, I'm listening): (A) Some kind of recursive operation *every* time you change an inheritable permission (B) A unified "starting point" from which you begin *every* access-control lookup (or one "starting point" per useful semantic grouping, like a namespace). The "(A)" is presently done in userspace and that's what you want to avoid. As to (B), I will attempt to prove below that you cannot implement "(B)" without breaking existing assumptions and restricting a very nice VFS model. No recursion is needed because only one acl exists, so that is the only one you need to update. At least on disk. Any cached acls in memory of descendant objects would need updated, but the number of those should be relatively small. The starting point would be the directory you start the lookup from. That may be the root, or it may be some other directory that you have a handle to, and thus, already has its effective acl computed. What ACL would "task->cwd" use? Options: (1.a) Use the one calculated during the original chdir() call. (1.b) Navigate "up" task->cwd building an ACL backwards. (1.c) $CAN_YOU_THINK_OF_SOMETHING_ELSE_HERE 1.a Unsolvable problems with each option: (1.a.I) You just broke all sorts of chrooted daemons. When I start bind in its chroot jail, it does the following: chdir("/private/bind9"); chroot("."); setgid(...); setuid(...); The "/private" directory is readable only by root, since root is the only one who will be navigating you into these chroots for any reason. You only switch UID/GID after the chroot() call, at which point you are inside of a sub-context and your cwd is fully accessible. If you stick an inheritable ACL on "/private", then the "cwd" ACL will not allow access by anybody but root and my bind won't be able to read any config files. If you want the directory to be root accessible but the files inside to have wider access then you set the acl on the directory to have one ace granting root access to the directory, and one ace that is inheritable granting access to bind. This latter ace does not apply to the directory itself, only to its children. You also break relative paths and directory-moving. Say a process does chdir("/foo/bar"). Now the ACL data in "cwd" is appropriate for /foo/bar. If you later chdir("../quux"), how do you unapply the changes made when you switched into that directory? For inheritable ACLs, you can't "unapply" such an ACL state change unless you save state for all the parent directories, except... What happens when you are in "/foo/bar" and another process does "mv /foo/bar /foobar/quux"? Suddenly any "cwd" ACL data you have is completely invalid and you have to rebuild your ACLs from scratch. Moreover, if the directory you are in was moved to a portion of the filesystem not accessible from your current namespace then how do you deal with it? Yes, if /foo/quux is not already cached in memory, you would have to walk the tree to build its acl. /foo should already be cached in memory so this work is minimal. Is this so horrible of a problem? As for moving, it is handled the same way as any other event that makes cwd go away, such as deleting it or revoking your access; cwd is now invalid. For example: NS1 has the / root dir of /dev/sdb1 mounted on /mnt NS2 has the /bar subdir of /dev/sdb1 mounted on /mnt Your process is in NS2 and does chdir("/mnt/quux"). A user in NS1 does: "mv /mnt/bar/quux /mnt/quux". Now your "cwd" is in a directory on a filesystem you have mounted, but it does not correspond *AT ALL* to any path available from your namespace. Which would be no different than if they just deleted the entire thing. Your cwd no longer exists. Another example: Your process has done dirfd=open("/media/cdrom/somestuff") when the admin does "umount -l /media/cdrom". You still have the CD-ROM open and accessible but IT HAS NO PATH. It isn't even mounted in *any* namespace, it's just kind of dangling waiting for its last users to go away. You can still do fchdir(dirfd), openat(dirfd, "foo/bar", ...), open("./foo"), etc. What's this got to do with acls? If you are asking what effect the umount thas on the acls of the cdrom, the answer is none. The acls are on the disc and nothing on the disc has changed. No, this is correct because in the root directory "/", the ".." entry is just another link to the root directory. So the absolute path "/../../../../../.." is just a fancy name for the root directory. The above jail-escape-as-root exploit is possible because it is impossible to determine whether a directory is or is not a subentry of another directory without an exhaustive search. So when your "cwd" points to a path outside of the chroot, the one sp
Re: Thinking outside the box on file systems
On Thu, 16 Aug 2007, Helge Hafting wrote: > Marc Perkel wrote: >> Kyle, What I'm suggesting is scrapping all existing >> concepts and replacing them with something entirely >> new. Posix, Unix, SELinux go away except for an >> emulation layer for backwards compatibility. What I'm >> suggesting is to start over and do it right. >> > If you want to get any support for "starting over", > then you need to: > 1. Point out some serious problem with the existing stuff, > otherwise why _bother_ start over > 2. Come up with a truly better idea (demonstrably better) >that isn't full of so obvious flaws that a seasoned kernel >developer can shoot it down in 5 minutes. > > Trying to be a visionary with a "great idea" that you are > prepared to let others implement just don't work on > this list. If you want to go that route, you > start a company, hire programmers, tell them to > implement your vision. If your idea is good then > your company succeeds. > > If you want to be an open-source visionary, you have to > do the initial work yourself until you attract other interested people. > >> One of the problems with the Unix/Linux world is that >> your minds are locked into this one model. In order to >> do it right it requires the mental discipline to break >> out of that. >> > Or perhaps unix have the best model already? ;-) > If you want a big break with the existing unix models, then > perhaps a entirely new project is in order, rather than > trying to change linux. Linux is after all, in use by millions > who are satisfied with the linux filesystem model already. > > Now, linux is open-source, so you can of course use it as a > starting point for your different system. Then you can compete > with "standard linux" - see who attracts most developers and > most users in the long run. > > Helge Hafting > - I think Marc Perkel's basic idea was implimented under CP/M as a "user level." Also, the concept of a virtual directory structure has been around about as long as "container files," such as those on the old Intel MDS200 development station. This is hardly "thinking out-of-the-box." It's just a rehash of some abandoned stuff. That said, a major speed limitation of directory lookups is the manipulation and comparison of variable-length strings. Maybe Mark might be able to improve this by truly thinking out-of-the-box. Cheers, Dick Johnson Penguin : Linux version 2.6.22.1 on an i686 machine (5588.29 BogoMips). My book : http://www.AbominableFirebug.com/ _ The information transmitted in this message is confidential and may be privileged. Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited. If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to [EMAIL PROTECTED] - and destroy all copies of this information, including any attachments, without reading or disclosing them. Thank you. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
Marc Perkel wrote: Kyle, What I'm suggesting is scrapping all existing concepts and replacing them with something entirely new. Posix, Unix, SELinux go away except for an emulation layer for backwards compatibility. What I'm suggesting is to start over and do it right. If you want to get any support for "starting over", then you need to: 1. Point out some serious problem with the existing stuff, otherwise why _bother_ start over 2. Come up with a truly better idea (demonstrably better) that isn't full of so obvious flaws that a seasoned kernel developer can shoot it down in 5 minutes. Trying to be a visionary with a "great idea" that you are prepared to let others implement just don't work on this list. If you want to go that route, you start a company, hire programmers, tell them to implement your vision. If your idea is good then your company succeeds. If you want to be an open-source visionary, you have to do the initial work yourself until you attract other interested people. One of the problems with the Unix/Linux world is that your minds are locked into this one model. In order to do it right it requires the mental discipline to break out of that. Or perhaps unix have the best model already? ;-) If you want a big break with the existing unix models, then perhaps a entirely new project is in order, rather than trying to change linux. Linux is after all, in use by millions who are satisfied with the linux filesystem model already. Now, linux is open-source, so you can of course use it as a starting point for your different system. Then you can compete with "standard linux" - see who attracts most developers and most users in the long run. Helge Hafting - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
Marc Perkel wrote: I am describing a kind of functionality and not tied to the method that implements that functionality. Perhaps a straight hash of the name isn't the best way to implement it. Just because someone tried to do something like what I'm suggesting years ago and it didn't work doesn't mean that it can't be done. You just have to come up with a better method. Actually, you are the one who have to come up with a better method. :-) You implement it, you deploy on your server, you work out the quirks, you win the discussion and impress everybody. Lets take this example. We are moving a million files from one branch if a tree to another. Do we wait for a million renames and hashes to occur? Of course not. So what to we do? We continue to be innovative. One must first adopt the attitude that anything can be done - you just have to be persistent until you figure out how. Feel free to be persistent and do the work & testing needed. People have done so before, which is why there are so many linux filesystems to choose from. What you can't do though, is to start with an attitude that "anything can be done" and then expect to change the direction of kernel work before everything is figured out. Experimental stuff stays in your private tree till it is proven - for a reason. In this case we could have a name translation layer so if you want to do a move you change the translation layer indicating that a move occurred. Thus access to the new files get translated into the old name and accessed until the files are rehashed. That name translation layer has a nasty failure mode. Let's say I want to move all files in /usr/local/* into /usr/old-local/ The name translation layer makes the transition seem instant, even though there are now 10 million files queued for renaming. References to /usr/local now goes to /usr/old-local/ thanks to translation. Now the problem. I did the move in order to fill /usr/local/ with new improved files. But all references to /usr/local now goes to /usr/old-local/, so the new files go there as well. A translation layer _can_ work around this, by noting that /usr/local is currently being translated, the move queue has not finished processing, so further accesses to /usr/local goes to /usr/local-tmp/ instead, to be moved into /usr/local when that other batch of moves finishes. In doing this, you set yourself up for lots of complexity and lots of behind-the-scenes work. People will wonder why the server is so busy when nobody is doing much, and what if there is a power crash or disk error on a busy server. It'd be an "interesting" fsck job on a machine that went down while processing several such deferred moves and translations. :-/ My point - you start with what you want to do and then you figure out how to make it happen. I can't answer all the details of how to make it happen but when I do something I start with the idea that if this were done right it would work this way and then I figure out how. Sure - you don't have to prove that it will work now. Working it out over time and then showing us is ok. Helge Hafting - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
Mmm, slow-as-dirt hotel wireless. What fun... On Aug 15, 2007, at 18:14:44, Phillip Susi wrote: Kyle Moffett wrote: I am well aware of that, I'm simply saying that sucks. Doing a recursive chmod or setfacl on a large directory tree is slow as all hell. Doing it in the kernel won't make it any faster. Right... I'm talking about getting rid of it entirely. Let me repeat myself here: Algorithmically you fundamentally CANNOT implement inheritance-based ACLs without one of the following (although if you have some other algorithm in mind, I'm listening): (A) Some kind of recursive operation *every* time you change an inheritable permission (B) A unified "starting point" from which you begin *every* access- control lookup (or one "starting point" per useful semantic grouping, like a namespace). The "(A)" is presently done in userspace and that's what you want to avoid. As to (B), I will attempt to prove below that you cannot implement "(B)" without breaking existing assumptions and restricting a very nice VFS model. Not necessarily. When I do "vim some-file-in-current-directory", for example, the kernel does *NOT* look up the path of my current directory. It does (in pseudocode): if (starts_with_slash(filename)) { entry = task->cwd; } else { entry = task->root; } while (have_components_left(filename) entry = lookup_next_component(filename); return entry; Right and task->cwd would have the effective acl in memory, ready to be combined with any acl set on the file. What ACL would "task->cwd" use? Options: (1.a) Use the one calculated during the original chdir() call. (1.b) Navigate "up" task->cwd building an ACL backwards. (1.c) $CAN_YOU_THINK_OF_SOMETHING_ELSE_HERE Unsolvable problems with each option: (1.a.I) You just broke all sorts of chrooted daemons. When I start bind in its chroot jail, it does the following: chdir("/private/bind9"); chroot("."); setgid(...); setuid(...); The "/private" directory is readable only by root, since root is the only one who will be navigating you into these chroots for any reason. You only switch UID/GID after the chroot() call, at which point you are inside of a sub-context and your cwd is fully accessible. If you stick an inheritable ACL on "/private", then the "cwd" ACL will not allow access by anybody but root and my bind won't be able to read any config files. You also break relative paths and directory-moving. Say a process does chdir("/foo/bar"). Now the ACL data in "cwd" is appropriate for /foo/bar. If you later chdir("../quux"), how do you unapply the changes made when you switched into that directory? For inheritable ACLs, you can't "unapply" such an ACL state change unless you save state for all the parent directories, except... What happens when you are in "/foo/bar" and another process does "mv /foo/bar /foobar/ quux"? Suddenly any "cwd" ACL data you have is completely invalid and you have to rebuild your ACLs from scratch. Moreover, if the directory you are in was moved to a portion of the filesystem not accessible from your current namespace then how do you deal with it? For example: NS1 has the / root dir of /dev/sdb1 mounted on /mnt NS2 has the /bar subdir of /dev/sdb1 mounted on /mnt Your process is in NS2 and does chdir("/mnt/quux"). A user in NS1 does: "mv /mnt/bar/quux /mnt/quux". Now your "cwd" is in a directory on a filesystem you have mounted, but it does not correspond *AT ALL* to any path available from your namespace. Another example: Your process has done dirfd=open("/media/cdrom/somestuff") when the admin does "umount -l /media/cdrom". You still have the CD-ROM open and accessible but IT HAS NO PATH. It isn't even mounted in *any* namespace, it's just kind of dangling waiting for its last users to go away. You can still do fchdir(dirfd), openat(dirfd, "foo/ bar", ...), open("./foo"), etc. In Linux the ONLY distinction between "relative" and "absolute" paths is that the "absolute" path begins with a magic slash which implies that you start at the hidden "root" fd the kernel manages. More detail on problems with the "building the ACL from scratch" part: That's not even paying attention to functions like "fchdir" or their interactions with "chroot" and namespaces. I can probably have an open directory handle to a volume in a completely different namespace, a volume which isn't even *MOUNTED* in my current fs namespace. Using that file-handle I believe I can "fchdir", "openat", etc, in a completely different namespace. I can do the same thing with a chroot, except there I can even "escape": /* Switch into chroot. Doesn't drop root privs */ chdir("/some/dir/somewhere"); chroot("."); /* Malicious code later on */ chdir("/"); chroot("another_dir"); chdir("../../../../../../../../.."); chroot("."); /* Now I'm back in the real root filesystem */ I don't see what thi
Re: Thinking outside the box on file systems
On Wed, 15 Aug 2007 15:48:15 PDT, Marc Perkel said: > > Consider the rules: > > > > peter '*a*' can create > > peter '*b*' cannot create > > > > Peter tries to create 'foo-ab-bar' - is he allowed > > to or not? > > > > First - I'm proposing a concept, not writing the > implementation of the concept. You are asking what > happens when someone write conflicting rules. That > depends on how you implement it. Conflicting rules can > cause unpredictable results. Good. Go work out what the rules have to be in order for the system to behave sanely. "Hand-waving concept" doesn't get anywhere. Fully fleshed-out concepts sometimes do - once they sprout code to actually implement them. > The point is that one can choose any rule system they > want and the rules applies to the names of the files > and the permissions of the users. No, you *can't* choose any rule system you want - some rule systems are unworkable because they create security exposures (usually of the "ln /etc/passwd /tmp/foo" variety, but sometimes race conditions as well). > > For an exersize, either write a program or do by > > hand: > All you would have to do is create a set of rules that > emulates the current rules and you would have the same > results. Good. Go create it. Let us know when you're done. Remember - not only do you need to have it generate the same results, you need to find a way to implement it so that it's somewhere near the same speed as the current code. If it's 10 times slower, it's not going to fly no matter *how* "cool" it is. pgp7EvsAH2cHY.pgp Description: PGP signature
Re: Thinking outside the box on file systems
--- [EMAIL PROTECTED] wrote: > On Wed, 15 Aug 2007 13:50:17 PDT, Marc Perkel said: > > I don't see it as being any worse that what we > have > > now. To open a file you have to start at the > bottom > > and open each directory and evaluate the > permissions > > on the way to the file. In my system you have to > look > > up the permission of the string at each "/" > separator. > > Seems to me that every system would have these > same > > steps. > > No - you need to look at the *whole* string - that's > the > whole *point* of your system, remember? > > Just a few msgs back, you gave a nice example of > having a file with \ in the name rather than / > because > it came from a Windows user. So you *do* need to > check *every* > pattern against the filename, because it *could* > match. > In a system with several hundred thousand or more > patterns, > that could be painful. > > Also, you need to figure out how to deal with all > the various > silly corner cases that people will end up trying to > do. > > Consider the rules: > > peter '*a*' can create > peter '*b*' cannot create > > Peter tries to create 'foo-ab-bar' - is he allowed > to or not? > First - I'm proposing a concept, not writing the implementation of the concept. You are asking what happens when someone write conflicting rules. That depends on how you implement it. Conflicting rules can cause unpredictable results. It may be as simple as first rule wins. Or it may require all the rules to be true. In the above example I would say it is not allowed because it matches a denial condition. The point is that one can choose any rule system they want and the rules applies to the names of the files and the permissions of the users. > For an exersize, either write a program or do by > hand: > > Create a list of patterns that correctly express the > ownership > and permissions of *every* file on your current > Linux box. > > Then repeat on a large box with multiple users and a > few Oracle > databases or webservers. > > Then write a small tool, that given that list, a > username, > a filename, and the operation (read, write, open, > unlink, etc), > says "Yes or No". > > Then run 'strace /bin/ls' in a large directory, take > all the > filenames listed in the strace output, and see if > your tool can > answer "yes or no" fast enough to make 'ls' > feasible. > > Come back when you get that part done, and we'll > discuss how it > would have to work in the kernel. All you would have to do is create a set of rules that emulates the current rules and you would have the same results. > Marc Perkel Junk Email Filter dot com http://www.junkemailfilter.com Boardwalk for $500? In 2007? Ha! Play Monopoly Here and Now (it's updated for today's economy) at Yahoo! Games. http://get.games.yahoo.com/proddesc?gamekey=monopolyherenow - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
--- Kyle Moffett <[EMAIL PROTECTED]> wrote: > Al Viro added to the CC, since he's one of the > experts on this stuff > and will probably whack me with a LART for > explaining it all wrong, > or something. :-D > Thanks - I appreciate that. Just to catch everyone up on what this thread is about, I'm proposing a new way of looking at file systems where files no longer have permission, owners, or groups, or file attributes. The idea is that people, groups, managers, applications, and other objects have permissions to names that are pointers to files. > On Aug 15, 2007, at 16:38:36, Phillip Susi wrote: > > Kyle Moffett wrote: > >> We've *always* had to do this; that's what "chmod > -R" or "setfacl - > >> R" are for :-D. The major problem is that the > locking and lookup > >> overhead gets really significant if you have to > look at the entire > >> directory tree in order to determine the > permissions for one > >> single object. I definitely agree that we need > better GUIs for > >> managing file permissions, but I don't see how > you could modify > >> the kernel in this case to do what you want. > > > > I am well aware of that, I'm simply saying that > sucks. Doing a > > recursive chmod or setfacl on a large directory > tree is slow as all > > hell. > > Doing it in the kernel won't make it any faster. > > > > As for hard links, your access would depend on > which name you use > > to access the file. The file itself may still > have an acl that > > grants or denies access to people no matter what > name they use, but > > if it allows inheritance, then which name you > access it by will > > modify the effective acl that it gets. > > You can't safely preserve POSIX semantics that way. > For example, > even without *ANY* ability to read /etc/shadow, I > can easily "ln /etc/ > shadow /tmp/shadow", assuming they are on the same > filesystem. If > the /etc/shadow permissions depend on inherited ACLs > to enforce > access then that one little command just made your > shadow file world- > readable/writeable. Oops. > > Think about it this way: > Permissions depend on *what* something is, not > *where* it is. Under > Linux you can leave the digital equivalent of a > $10,000 piece of > jewelry lying around in /var/www and not have to > worry about it being > compromised as long as you set your permissions > properly (not that I > recommend it). Moving the piece of jewelry around > your house does > not change what it *is* (and by extension does not > change the > protection required on it), any more than "ln > /etc/shadow /tmp/ > shadow" (or "mv") changes what *it* is. If your > /house is really > extraordinarily secure then you could leave the > jewelry lying around > as /house/gems.bin with permissions 0777, but if > somebody had a back- > door to /house (an open fd, a careless typo, etc) > then you'd have the > same issues. My proposal is the same somewhat. If one put restricting on a specific name to deny access to users then that denial follows that filename even if it is copied or moved. However if a file has no specific restrictions and is in a restricted directory then the file inherits the restrictions and permissions of the new directory based on where it is. If you don't want your jewlery laying around then don't put a copy of it in a folder where users have access to it. Marc Perkel Junk Email Filter dot com http://www.junkemailfilter.com Yahoo! oneSearch: Finally, mobile search that gives answers, not web links. http://mobile.yahoo.com/mobileweb/onesearch?refer=1ONXIC - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
Kyle Moffett wrote: I am well aware of that, I'm simply saying that sucks. Doing a recursive chmod or setfacl on a large directory tree is slow as all hell. Doing it in the kernel won't make it any faster. Right... I'm talking about getting rid of it entirely. You can't safely preserve POSIX semantics that way. For example, even without *ANY* ability to read /etc/shadow, I can easily "ln /etc/shadow /tmp/shadow", assuming they are on the same filesystem. If the /etc/shadow permissions depend on inherited ACLs to enforce access then that one little command just made your shadow file world-readable/writeable. Oops. That's why /etc/shadow would have an IRM that would block inheriting any permissions. Think about it this way: Permissions depend on *what* something is, not *where* it is. Under No, they don't. At least not always. A lot of times people want permissions based on where it is. Linux you can leave the digital equivalent of a $10,000 piece of jewelry lying around in /var/www and not have to worry about it being compromised as long as you set your permissions properly (not that I recommend it). Moving the piece of jewelry around your house does not change what it *is* (and by extension does not change the protection required on it), any more than "ln /etc/shadow /tmp/shadow" (or "mv") changes what *it* is. If your /house is really extraordinarily secure then you could leave the jewelry lying around as /house/gems.bin with permissions 0777, but if somebody had a back-door to /house (an open fd, a careless typo, etc) then you'd have the same issues. Yes, but if you move it to your front porch, you have changed the protection on it. If you want to be sure of it being protected, keep it locked up in a safe... in other words, use the IRM. Not necessarily. When I do "vim some-file-in-current-directory", for example, the kernel does *NOT* look up the path of my current directory. It does (in pseudocode): if (starts_with_slash(filename)) { entry = task->cwd; } else { entry = task->root; } while (have_components_left(filename) entry = lookup_next_component(filename); return entry; Right and task->cwd would have the effective acl in memory, ready to be combined with any acl set on the file. That's not even paying attention to functions like "fchdir" or their interactions with "chroot" and namespaces. I can probably have an open directory handle to a volume in a completely different namespace, a volume which isn't even *MOUNTED* in my current fs namespace. Using that file-handle I believe I can "fchdir", "openat", etc, in a completely different namespace. I can do the same thing with a chroot, except there I can even "escape": /* Switch into chroot. Doesn't drop root privs */ chdir("/some/dir/somewhere"); chroot("."); /* Malicious code later on */ chdir("/"); chroot("another_dir"); chdir("../../../../../../../../.."); chroot("."); /* Now I'm back in the real root filesystem */ I don't see what this has to do with this discussion, and I also can't believe that is correct... the chdir( "../../../../.." ) should fail because there is no such directory. The locking penalty is because the path-lookup is *not* implied. The above chroot example shows that in detail. If you have to do the lookup in *reverse* on every open operation then you have to either: (A) Store lots of security context with every open directory (cwd included). When a directory you have open is moved, you still have full access to everything inside it since your handle's data hasn't changed. Yes, the effective acl of the open directory is kept in memory, but in the directory itself, not the handle to it, thus when the directory is moved, it's acl is recomputed for the new location and updated immediately. It is like using fcntl to set a file to non blocking... it is the file you set, not the handle to it, so it effects other processes that have inherited or duplicated the file. See above. Linux has a very *very* powerful VFS nowadays. It also has support for shared subtrees across private namespaces, allowing things like polyinstantiation. For example you can configure a Linux box so every user gets their own empty /tmp directory created and mounted on their namespace's /tmp when they log in, but all the rest of the system mounts are all shared. I still can't see how these features cause a problem. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
On Wed, 15 Aug 2007 13:50:17 PDT, Marc Perkel said: > I don't see it as being any worse that what we have > now. To open a file you have to start at the bottom > and open each directory and evaluate the permissions > on the way to the file. In my system you have to look > up the permission of the string at each "/" separator. > Seems to me that every system would have these same > steps. No - you need to look at the *whole* string - that's the whole *point* of your system, remember? Just a few msgs back, you gave a nice example of having a file with \ in the name rather than / because it came from a Windows user. So you *do* need to check *every* pattern against the filename, because it *could* match. In a system with several hundred thousand or more patterns, that could be painful. Also, you need to figure out how to deal with all the various silly corner cases that people will end up trying to do. Consider the rules: peter '*a*' can create peter '*b*' cannot create Peter tries to create 'foo-ab-bar' - is he allowed to or not? For an exersize, either write a program or do by hand: Create a list of patterns that correctly express the ownership and permissions of *every* file on your current Linux box. Then repeat on a large box with multiple users and a few Oracle databases or webservers. Then write a small tool, that given that list, a username, a filename, and the operation (read, write, open, unlink, etc), says "Yes or No". Then run 'strace /bin/ls' in a large directory, take all the filenames listed in the strace output, and see if your tool can answer "yes or no" fast enough to make 'ls' feasible. Come back when you get that part done, and we'll discuss how it would have to work in the kernel. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
Al Viro added to the CC, since he's one of the experts on this stuff and will probably whack me with a LART for explaining it all wrong, or something. :-D On Aug 15, 2007, at 16:38:36, Phillip Susi wrote: Kyle Moffett wrote: We've *always* had to do this; that's what "chmod -R" or "setfacl - R" are for :-D. The major problem is that the locking and lookup overhead gets really significant if you have to look at the entire directory tree in order to determine the permissions for one single object. I definitely agree that we need better GUIs for managing file permissions, but I don't see how you could modify the kernel in this case to do what you want. I am well aware of that, I'm simply saying that sucks. Doing a recursive chmod or setfacl on a large directory tree is slow as all hell. Doing it in the kernel won't make it any faster. As for hard links, your access would depend on which name you use to access the file. The file itself may still have an acl that grants or denies access to people no matter what name they use, but if it allows inheritance, then which name you access it by will modify the effective acl that it gets. You can't safely preserve POSIX semantics that way. For example, even without *ANY* ability to read /etc/shadow, I can easily "ln /etc/ shadow /tmp/shadow", assuming they are on the same filesystem. If the /etc/shadow permissions depend on inherited ACLs to enforce access then that one little command just made your shadow file world- readable/writeable. Oops. Think about it this way: Permissions depend on *what* something is, not *where* it is. Under Linux you can leave the digital equivalent of a $10,000 piece of jewelry lying around in /var/www and not have to worry about it being compromised as long as you set your permissions properly (not that I recommend it). Moving the piece of jewelry around your house does not change what it *is* (and by extension does not change the protection required on it), any more than "ln /etc/shadow /tmp/ shadow" (or "mv") changes what *it* is. If your /house is really extraordinarily secure then you could leave the jewelry lying around as /house/gems.bin with permissions 0777, but if somebody had a back- door to /house (an open fd, a careless typo, etc) then you'd have the same issues. As for performance implications, I hardly think it is worrisome. Each directory in the path has to be looked up anyhow so you already have their acls, so when you finally reach the file, you just have to take the union of the acls encountered on the way. Should only be a few cpu cycles. Not necessarily. When I do "vim some-file-in-current-directory", for example, the kernel does *NOT* look up the path of my current directory. It does (in pseudocode): if (starts_with_slash(filename)) { entry = task->cwd; } else { entry = task->root; } while (have_components_left(filename) entry = lookup_next_component(filename); return entry; That's not even paying attention to functions like "fchdir" or their interactions with "chroot" and namespaces. I can probably have an open directory handle to a volume in a completely different namespace, a volume which isn't even *MOUNTED* in my current fs namespace. Using that file-handle I believe I can "fchdir", "openat", etc, in a completely different namespace. I can do the same thing with a chroot, except there I can even "escape": /* Switch into chroot. Doesn't drop root privs */ chdir("/some/dir/somewhere"); chroot("."); /* Malicious code later on */ chdir("/"); chroot("another_dir"); chdir("../../../../../../../../.."); chroot("."); /* Now I'm back in the real root filesystem */ There's also the question of what to do about namespaces and bind mounts. If I run "mount --bind /foo /home/foo", then do I get different file permissions depending on what path I access the file by? What if I then run chroot("/foo"), do I get different file permissions then? What if I have two namespaces, each with their own root filesystem (say "root1" and "root2"), and I mount the other namespace's root filesystem in a subdir of each: NS1: mount /dev/root2 /otherns NS2: mount /dev/root1 /otherns Now I have the following paths to the same file, do these get different permissions or the same? NS1:/foo == NS2:/otherns/foo NS2:/bar == NS1:/otherns/bar If you answered that they get different permissions, then how do you handle the massive locking performance penalty due to the extra lookups? If you answered "same permissions", then how do you explain the obvious discrepancy to the admin? Good question. I would say the bind mount should have a flag to specify. That way the admin can choose where it should inherit from. I also don't see where this locking penalty is. The locking penalty is because the path-lookup is *not* implied. The above chroot example
Re: Thinking outside the box on file systems
On Wed, Aug 15, 2007 at 01:44:50PM -0700, Marc Perkel wrote: > Yes - that's a good example. Git is far more powerful > and a different paradigm for CVS. Someone had to think > outside the box and come up with a new way of looking > at things. I'm trying to do something like that with > this idea. > > To me it make more sense to get rid of file > permissions and look at people permissions. It reminds > me of a story a friend of mine told about her 4 year > old son. > > The story was that they were driving down the road > when they saw a wheel come off a truck. The son said, > "look mommy, that wheel lost it's truck." > > To me files are like the wheel. Rather than having the > file know all it's owners it makes more sense for the > owners to know it's files. Except for the fact that on most systems each owner may have millions of files, while very rarely does a file have more than a few dozen owners or groups. Having to wade through the permissions of millions of things seems like a lot more work than checking a few dozen things. Also the thing is that I care who can access files, while I do not really care what particular set of files a user has access to. It is the data I am protecting, not the user. -- Len Sorensen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
--- Phillip Susi <[EMAIL PROTECTED]> wrote: > Marc Perkel wrote: > > > > Kyle - you are still missing the point. chmod goes > > away. File permissions goes away. Directories as > you > > know them goes away. > > You are missing the point Marc... open()ing a file > will have to perform > a number of these pattern matches to decide if it is > allowed or not... > this would be a HUGE overhead. > > I don't see it as being any worse that what we have now. To open a file you have to start at the bottom and open each directory and evaluate the permissions on the way to the file. In my system you have to look up the permission of the string at each "/" separator. Seems to me that every system would have these same steps. Marc Perkel Junk Email Filter dot com http://www.junkemailfilter.com Park yourself in front of a world of choices in alternative vehicles. Visit the Yahoo! Auto Green Center. http://autos.yahoo.com/green_center/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
--- Kyle Moffett <[EMAIL PROTECTED]> wrote: > On Aug 15, 2007, at 15:26:07, Lennart Sorensen > wrote: > > On Wed, Aug 15, 2007 at 10:59:12AM -0700, Marc > Perkel wrote: > >> When one thinks outside the box one has to think > about evolving > >> beyond what you are used to. When I moved > >> beyond DOS I have to give up the idea of 8.3 file > names. The idea > >> here is to come up with a model that can emulate > the existing > >> system for backwards compatibility. > > > > But moving beyond 8.3 didn't prevent you from > still using 8.3 names > > if you wanted too. Longer file names are just an > extension of > > shorter ones. > > As another example, take a look at "git", the SCM we > use for the > kernel, as contrasted with the older CVS. You can > import your > complete CVS history into it without data loss, and > then you can even > continue to use it the exact same way you used to > use CVS, with some > slight differences in command-line syntax. Once you > are ready to > move further, though, you can create multiple local > branches to have > your co-workers pull from to test changes. You > discover that merging > branches is much easier in git than in CVS. Your > company starts to > use a more distributed development model, they > implement a policy > telling developers to break up their changes into > smaller pieces and > write better change-logs. Somebody suddenly > discovers the ability to > "sign" a particular release version with a private > key, and you start > doing that as part of your release management to > ensure that the > codebase marked with a client tag is the exact same > one you actually > shipped to that client. > > On a fundamental level, GIT is a completely > different paradigm from > CVS. Its internal operations are entirely > differently organized, it > uses different algorithms and different storage > formats. The end > result of that is that it's literally orders of > magnitude faster on > large codebases. But to the USER it can be used > exactly the same; > you could even write a little CVS-to-GIT wrapper > which imported your > CVS into a git repo and then let you operate on it > using "gcvs" > commands the same way you would have operated on > real CVS repositories. > > Just some food for thought > > Cheers, > Kyle Moffett > Yes - that's a good example. Git is far more powerful and a different paradigm for CVS. Someone had to think outside the box and come up with a new way of looking at things. I'm trying to do something like that with this idea. To me it make more sense to get rid of file permissions and look at people permissions. It reminds me of a story a friend of mine told about her 4 year old son. The story was that they were driving down the road when they saw a wheel come off a truck. The son said, "look mommy, that wheel lost it's truck." To me files are like the wheel. Rather than having the file know all it's owners it makes more sense for the owners to know it's files. Marc Perkel Junk Email Filter dot com http://www.junkemailfilter.com Got a little couch potato? Check out fun summer activities for kids. http://search.yahoo.com/search?fr=oni_on_mail&p=summer+activities+for+kids&cs=bz - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
Marc Perkel wrote: Kyle - you are still missing the point. chmod goes away. File permissions goes away. Directories as you know them goes away. You are missing the point Marc... open()ing a file will have to perform a number of these pattern matches to decide if it is allowed or not... this would be a HUGE overhead. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
Kyle Moffett wrote: We've *always* had to do this; that's what "chmod -R" or "setfacl -R" are for :-D. The major problem is that the locking and lookup overhead gets really significant if you have to look at the entire directory tree in order to determine the permissions for one single object. I definitely agree that we need better GUIs for managing file permissions, but I don't see how you could modify the kernel in this case to do what you want. I am well aware of that, I'm simply saying that sucks. Doing a recursive chmod or setfacl on a large directory tree is slow as all hell. So what would you have happen when you move another directory into that directory? Should it retain its permissions? If they change based on the new directory then do you recurse into each subdirectory? Such recursing to modify permissions also has significant performance implications. What about if the file is hardlinked elsewhere; do those permissions change? Simple... the file retains any acl it already had, AND the acl of the new directory now applies. Most likely the moved file had no acl and was just inheriting its effective acl from its old parent directory. The end result is that people who used to have access to the file by virtue of it being in their directory no longer do, and the people who are supposed to have access to all files in the new directory get access to this one. As for hard links, your access would depend on which name you use to access the file. The file itself may still have an acl that grants or denies access to people no matter what name they use, but if it allows inheritance, then which name you access it by will modify the effective acl that it gets. As for performance implications, I hardly think it is worrisome. Each directory in the path has to be looked up anyhow so you already have their acls, so when you finally reach the file, you just have to take the union of the acls encountered on the way. Should only be a few cpu cycles. There's also the question of what to do about namespaces and bind mounts. If I run "mount --bind /foo /home/foo", then do I get different file permissions depending on what path I access the file by? What if I then run chroot("/foo"), do I get different file permissions then? What if I have two namespaces, each with their own root filesystem (say "root1" and "root2"), and I mount the other namespace's root filesystem in a subdir of each: NS1: mount /dev/root2 /otherns NS2: mount /dev/root1 /otherns Now I have the following paths to the same file, do these get different permissions or the same? NS1:/foo == NS2:/otherns/foo NS2:/bar == NS1:/otherns/bar If you answered that they get different permissions, then how do you handle the massive locking performance penalty due to the extra lookups? If you answered "same permissions", then how do you explain the obvious discrepancy to the admin? Good question. I would say the bind mount should have a flag to specify. That way the admin can choose where it should inherit from. I also don't see where this locking penalty is. The idea is nice, but as soon as you add multiple namespaces, chroots, or bind mounts the concept breaks down. For security-sensitive things like file permissions, you really do want determinate behavior regardless of the path used to access the data. How does it break down? Chroots have absolutely no impact at all, and the bind mounts/namespaces can be handled as I mentioned above. If you really want to be sure of the effective permissions on the file, then you simply flag it to not inherit from its parent or use an inherited rights mask to block the specific inherited permissions you want. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
--- Craig Ruff <[EMAIL PROTECTED]> wrote: > On Wed, Aug 15, 2007 at 10:30:19AM -0700, Marc > Perkel wrote: > > --- Kyle Moffett <[EMAIL PROTECTED]> wrote: > > > Except they do, and without directories the > > > performance of your average filesystem is going > to suck. > > > > Actually you would get a speed improvement. You > hash > > the full name and get the file number. You don't > have > > to break up the name into sections except for > > evaluating name permissions. > > > > The important concept here is that files and name > > aren't stored by levels of directories. The name > > points to the file number. Directory levels are > > emulated based on name separation characters or > any > > other algorithm that you want to use. > > > > One could create a file system and permission > system > > that gets rid of the concept of directories > entirely > > if one chooses to. > > I would like to add support for Kyle's assertion. > > The model described by Marc is exactly the method > used by the current > version of the NCAR Mass Storage Service (MSS), > which is data archive > of 4+ petabytes contained in 40+ million files. To > the user's point > of view, it looks somewhat like a POSIX file system > with both some > extensions and deficiencies. The MSS was designed > in the mid-1980s, > in an era where the costs of the supercomputers > (Cray-1s at that time) > were paramount. This lead to some MSS design > decisions to minimize the > need for users to rerun jobs on the expensive > supercomputer just because > they messed up their MSS file creation statements. > > Files names are a maximum of 128 bytes, with a > dynamically managed > directory structure indicated by '/' characters in > the name. The file > name is hashed, and the hash table provides the > internal file number (the > address in the Master File Directory (MFD)). Any > parent directories > are created automatically by the system upon file > creation, and are > automatically deleted if empty upon file deletion. > Directories also > have a self pointer, and both files and directories > are chained together > to allow the user to list (or otherwise manipulate) > the contents of > a directory. > > The biggest problem with this model is that to > manipulate the a directory > itself, you have to simulate the operation on all of > the files contained > within it. For example to rename a directory with > 'n' descendants, > you must perform: > > n+1 hash table removals > n+1 hash table insertions (with collision > detection) > n+1 MFD record updates > 1 directory chain removal > 1 directory chain insertion > > This is, needless to say, very painful when n is > large. Since users > must use directory trees to efficiently manage their > data holdings, > efficient directory manipulation is essential. > Contrast this with > the number of operations required for a directory > rename if files > do not record their complete pathname: > > 1 directory chain removal > 1 directory chain insertion > > Fortunately we are currently working to change from > using a model like > Marc describes to one Kyle describes. > I am describing a kind of functionality and not tied to the method that implements that functionality. Perhaps a straight hash of the name isn't the best way to implement it. Just because someone tried to do something like what I'm suggesting years ago and it didn't work doesn't mean that it can't be done. You just have to come up with a better method. Lets take this example. We are moving a million files from one branch if a tree to another. Do we wait for a million renames and hashes to occur? Of course not. So what to we do? We continue to be innovative. One must first adopt the attitude that anything can be done - you just have to be persistent until you figure out how. In this case we could have a name translation layer so if you want to do a move you change the translation layer indicating that a move occurred. Thus access to the new files get translated into the old name and accessed until the files are rehashed. Or - maybe there is some sort of tokenizer database for the names in the directory sections and you can just rename the section. Sort of a tree like database of hashes data within hashes. My point - you start with what you want to do and then you figure out how to make it happen. I can't answer all the details of how to make it happen but when I do something I start with the idea that if this were done right it would work this way and then I figure out how. Marc Perkel Junk Email Filter dot com http://www.junkemailfilter.com Take the Internet to Go: Yahoo!Go puts the Internet in your pocket: mail, news, photos & more. http://mobile.yahoo.com/go?refer=1GNXIC - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PRO
Re: Thinking outside the box on file systems
--- Kyle Moffett <[EMAIL PROTECTED]> wrote: > On Aug 15, 2007, at 14:05:23, Marc Perkel wrote: > > In this new system setfacl, chmod, chown, and > chgrp all go away > > except inside of an emulation layer. File and > directories no longer > > have permissions. People have permission to naming > patterns. So if > > you put a file into a tree or move a tree then > those who have > > permissions to the tree have access to the files. > > > > It eliminates the step of having to apply > permission after moving > > files into a tree. You don't have to change file > permissions > > because files no longer have permissions. > > And I'm trying to tell you that unless you have some > magic new > algorithm that turns NP-complete problems into > O(log(N)) problems, > your idea won't work. You can't just say "I just do > one little thing > (mv) and the entire rest of the computer > automagically changes to > match", because that would imply a single unscalable > global kernel > lock. "Pattern"-matching is either NP-complete or > high-polynomial- > order, depending on how its implemented, and if you > want to do a > recursive-chmod during a directory move then you're > going to have > race-conditions out the ass. If you have code or > solid math to back > up your postings then please do so, but otherwise > you're just wasting > time and bandwidth. > > Kyle - you are still missing the point. chmod goes away. File permissions goes away. Directories as you know them goes away. Marc Perkel Junk Email Filter dot com http://www.junkemailfilter.com Got a little couch potato? Check out fun summer activities for kids. http://search.yahoo.com/search?fr=oni_on_mail&p=summer+activities+for+kids&cs=bz - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
On Aug 15, 2007, at 15:26:07, Lennart Sorensen wrote: On Wed, Aug 15, 2007 at 10:59:12AM -0700, Marc Perkel wrote: When one thinks outside the box one has to think about evolving beyond what you are used to. When I moved beyond DOS I have to give up the idea of 8.3 file names. The idea here is to come up with a model that can emulate the existing system for backwards compatibility. But moving beyond 8.3 didn't prevent you from still using 8.3 names if you wanted too. Longer file names are just an extension of shorter ones. As another example, take a look at "git", the SCM we use for the kernel, as contrasted with the older CVS. You can import your complete CVS history into it without data loss, and then you can even continue to use it the exact same way you used to use CVS, with some slight differences in command-line syntax. Once you are ready to move further, though, you can create multiple local branches to have your co-workers pull from to test changes. You discover that merging branches is much easier in git than in CVS. Your company starts to use a more distributed development model, they implement a policy telling developers to break up their changes into smaller pieces and write better change-logs. Somebody suddenly discovers the ability to "sign" a particular release version with a private key, and you start doing that as part of your release management to ensure that the codebase marked with a client tag is the exact same one you actually shipped to that client. On a fundamental level, GIT is a completely different paradigm from CVS. Its internal operations are entirely differently organized, it uses different algorithms and different storage formats. The end result of that is that it's literally orders of magnitude faster on large codebases. But to the USER it can be used exactly the same; you could even write a little CVS-to-GIT wrapper which imported your CVS into a git repo and then let you operate on it using "gcvs" commands the same way you would have operated on real CVS repositories. Just some food for thought Cheers, Kyle Moffett - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
On 8/15/07, Marc Perkel <[EMAIL PROTECTED]> wrote: > I want to throw out some concepts about a new way of > thinking about file systems. But the first thing you > have to do is to forget what you know about file > systems now. This is a discussion about a new view of > looking a file storage that is radically different and > it's more easily undersood if you forget a lot of what > you know. The idea is to create what seems natural to > the user rather than what seems natural to the > programmer. I believe that kernel interface is not really meant to be operated on the level that's directly accessible by the end user. The food chain is a bit different. The human user interacts with userlevel apps, not with kernel API directly. The userlevel apps interact in turn with the kernel APIs, either directly or via the layers of libraries. The abstrations that's presented to the human use are not necessarily 1:1 reflection of the kernel APIs. For example, you could program your novel way of permissions as a new file manager application that actually uses existing posix permissions underneath. Your file manager could check with userlevel-stored policies to implement the permissions that you describe, without any changes in existing kernel and existing filesystems. To expect that posix-compliant kernel will drop its posix complicance for temporary experiment sounds a bit far-fetched to me. But the door for all kinds of experiments is widely open in the userlevel. Yakov - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
On Wed, Aug 15, 2007 at 10:59:12AM -0700, Marc Perkel wrote: > When one thinks outside the box one has to think about > evolving beyond what you are used to. When I moved > beyond DOS I have to give up the idea of 8.3 file > names. The idea here is to come up with a model that > can emulate the existing system for backwards > compatibility. But moving beyond 8.3 didn't prevent you from still using 8.3 names if you wanted too. Longer file names are just an extension of shorter ones. > The concept behind my model is to create a new layer > where you can do ANYTHING with file names and > permissions and create models that emulate Linux, DOS, > Windows, Mac, or anything else you can dream of. Then > you can create a Linux/Windows/Mac template to emulate > what you are used to. I am not even sure your idea could represent everything that is currently possible with Posix ACLs and the standard unix permisions, never mind SELinux. I am also almost entirely convinced that your idea would be amazingly slow and inefficient and a serious pain to use. And of course nothing seems to ever suceed without backwards compatibility to support legacy programs until new ones take over, unless it is in fact an entirely new concept for a new field and isn't trying to replace some existing working system. -- Len Sorensen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
On Wed, Aug 15, 2007 at 10:09:31AM -0700, Marc Perkel wrote: > Yep - way outside the box - and thus the title of the > thread. > > The idea is that people have permissions - not files. > By people I mean users, groups, managers, applications > etc. One might even specify that there are no > permission restrictions at all. Part of the process > would be that the kernel load what code it will use > for the permission system. It might even be a little > perl script you write. > > > Also - you aren't even giving permission to access > files. It's permission to access name patterns. One > could apply REGEX masks to names to determine > permissions. So if you have permission to the name you > have permission to the file. So if I have permission to access /foo/*x but no permission to access /foo/*y, do I have permission to rename /foo/123x to /foo/123y and if I do so, do I loose access to my file? Can I move it back? > Hard links would be multiple names pointing to the > same file. Simlinks would be name aliases. I think I prefer to keep my files inside the box. That way I won't need to get a bucket. :) -- Len Sorensen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
HI While I find your ideas intriguing, I'd like to offer a friendly suggestion: You're never going to convince anyone on LKML unless you do the following things: * Describe your idea in detail, including algorithms, pseudo code, pictures, or whatever. Vague hand-wavey stuff won't do it. * Don't accuse others of being close minded (i.e. "not thinking outside the box"). Explain why their assertions may not be correct according to your proposal. * Accept that others have far more experience, no matter how experienced you are. * Remember that the current assumptions can't just break when the idea is implemented: backwards compatibility is important. Also, be prepared to have your idea shot down. Very few ideas make it into the kernel, and those only get there after months of hashing out the details. Brian --- Kyle Moffett <[EMAIL PROTECTED]> wrote: > On Aug 15, 2007, at 13:19:16, Marc Perkel wrote: > > One of the problems with the Unix/Linux world is > that your minds > > are locked into this one model. In order to do it > right it requires > > the mental discipline to break out of that. > > The major thing that you are missing is that this > "one model" has > been very heavily tested over the years. People > understand it, know > how to use it and write software for it, and grok > its limitations. > There's also a vast amount of *existing* code that > you can't just > "deprecate" overnight; the world just doesn't work > that way. The > real way to get there (IE: a new model) from here > (IE: the old model) > is the way all Linux development is done with a lot > of sensible easy- > to-understand changes and refactorings. > > With that said, if you actually want to sit down and > start writing > *code* for your model, go ahead. If it turns out to > be better than > our existing model then I owe you a bottle of your > favorite beverage. > > Cheers, > Kyle Moffett > When one thinks outside the box one has to think about evolving beyond what you are used to. When I moved beyond DOS I have to give up the idea of 8.3 file names. The idea here is to come up with a model that can emulate the existing system for backwards compatibility. The concept behind my model is to create a new layer where you can do ANYTHING with file names and permissions and create models that emulate Linux, DOS, Windows, Mac, or anything else you can dream of. Then you can create a Linux/Windows/Mac template to emulate what you are used to. Marc Perkel Junk Email Filter dot com http://www.junkemailfilter.com - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
On Wed, Aug 15, 2007 at 10:30:19AM -0700, Marc Perkel wrote: > --- Kyle Moffett <[EMAIL PROTECTED]> wrote: > > Except they do, and without directories the > > performance of your average filesystem is going to suck. > > Actually you would get a speed improvement. You hash > the full name and get the file number. You don't have > to break up the name into sections except for > evaluating name permissions. > > The important concept here is that files and name > aren't stored by levels of directories. The name > points to the file number. Directory levels are > emulated based on name separation characters or any > other algorithm that you want to use. > > One could create a file system and permission system > that gets rid of the concept of directories entirely > if one chooses to. I would like to add support for Kyle's assertion. The model described by Marc is exactly the method used by the current version of the NCAR Mass Storage Service (MSS), which is data archive of 4+ petabytes contained in 40+ million files. To the user's point of view, it looks somewhat like a POSIX file system with both some extensions and deficiencies. The MSS was designed in the mid-1980s, in an era where the costs of the supercomputers (Cray-1s at that time) were paramount. This lead to some MSS design decisions to minimize the need for users to rerun jobs on the expensive supercomputer just because they messed up their MSS file creation statements. Files names are a maximum of 128 bytes, with a dynamically managed directory structure indicated by '/' characters in the name. The file name is hashed, and the hash table provides the internal file number (the address in the Master File Directory (MFD)). Any parent directories are created automatically by the system upon file creation, and are automatically deleted if empty upon file deletion. Directories also have a self pointer, and both files and directories are chained together to allow the user to list (or otherwise manipulate) the contents of a directory. The biggest problem with this model is that to manipulate the a directory itself, you have to simulate the operation on all of the files contained within it. For example to rename a directory with 'n' descendants, you must perform: n+1 hash table removals n+1 hash table insertions (with collision detection) n+1 MFD record updates 1 directory chain removal 1 directory chain insertion This is, needless to say, very painful when n is large. Since users must use directory trees to efficiently manage their data holdings, efficient directory manipulation is essential. Contrast this with the number of operations required for a directory rename if files do not record their complete pathname: 1 directory chain removal 1 directory chain insertion Fortunately we are currently working to change from using a model like Marc describes to one Kyle describes. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
On Aug 15, 2007, at 14:05:23, Marc Perkel wrote: In this new system setfacl, chmod, chown, and chgrp all go away except inside of an emulation layer. File and directories no longer have permissions. People have permission to naming patterns. So if you put a file into a tree or move a tree then those who have permissions to the tree have access to the files. It eliminates the step of having to apply permission after moving files into a tree. You don't have to change file permissions because files no longer have permissions. And I'm trying to tell you that unless you have some magic new algorithm that turns NP-complete problems into O(log(N)) problems, your idea won't work. You can't just say "I just do one little thing (mv) and the entire rest of the computer automagically changes to match", because that would imply a single unscalable global kernel lock. "Pattern"-matching is either NP-complete or high-polynomial- order, depending on how its implemented, and if you want to do a recursive-chmod during a directory move then you're going to have race-conditions out the ass. If you have code or solid math to back up your postings then please do so, but otherwise you're just wasting time and bandwidth. Cheers, Kyle Moffett - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
Kyle, In this new system setfacl, chmod, chown, and chgrp all go away except inside of an emulation layer. File and directories no longer have permissions. People have permission to naming patterns. So if you put a file into a tree or move a tree then those who have permissions to the tree have access to the files. It eliminates the step of having to apply permission after moving files into a tree. You don't have to change file permissions because files no longer have permissions. Marc Perkel Junk Email Filter dot com http://www.junkemailfilter.com Shape Yahoo! in your own image. Join our Network Research Panel today! http://surveylink.yahoo.com/gmrs/yahoo_panel_invite.asp?a=7 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
--- Kyle Moffett <[EMAIL PROTECTED]> wrote: > On Aug 15, 2007, at 13:19:16, Marc Perkel wrote: > > One of the problems with the Unix/Linux world is > that your minds > > are locked into this one model. In order to do it > right it requires > > the mental discipline to break out of that. > > The major thing that you are missing is that this > "one model" has > been very heavily tested over the years. People > understand it, know > how to use it and write software for it, and grok > its limitations. > There's also a vast amount of *existing* code that > you can't just > "deprecate" overnight; the world just doesn't work > that way. The > real way to get there (IE: a new model) from here > (IE: the old model) > is the way all Linux development is done with a lot > of sensible easy- > to-understand changes and refactorings. > > With that said, if you actually want to sit down and > start writing > *code* for your model, go ahead. If it turns out to > be better than > our existing model then I owe you a bottle of your > favorite beverage. > > Cheers, > Kyle Moffett > When one thinks outside the box one has to think about evolving beyond what you are used to. When I moved beyond DOS I have to give up the idea of 8.3 file names. The idea here is to come up with a model that can emulate the existing system for backwards compatibility. The concept behind my model is to create a new layer where you can do ANYTHING with file names and permissions and create models that emulate Linux, DOS, Windows, Mac, or anything else you can dream of. Then you can create a Linux/Windows/Mac template to emulate what you are used to. Marc Perkel Junk Email Filter dot com http://www.junkemailfilter.com Boardwalk for $500? In 2007? Ha! Play Monopoly Here and Now (it's updated for today's economy) at Yahoo! Games. http://get.games.yahoo.com/proddesc?gamekey=monopolyherenow - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
On Aug 15, 2007, at 13:34:44, Phillip Susi wrote: The problem that I have with this setup is that it specifies an ACL on EACH file. Yes, you can set a default on the directory for newly created files, but what if I want to add a user to the access list for that whole directory? I have to individually update every acl on every file in that directory. We've *always* had to do this; that's what "chmod -R" or "setfacl -R" are for :-D. The major problem is that the locking and lookup overhead gets really significant if you have to look at the entire directory tree in order to determine the permissions for one single object. I definitely agree that we need better GUIs for managing file permissions, but I don't see how you could modify the kernel in this case to do what you want. Also if you move a file created elsewhere into that directory, it retains its existing permissions doesn't it? So what would you have happen when you move another directory into that directory? Should it retain its permissions? If they change based on the new directory then do you recurse into each subdirectory? Such recursing to modify permissions also has significant performance implications. What about if the file is hardlinked elsewhere; do those permissions change? There's also the question of what to do about namespaces and bind mounts. If I run "mount --bind /foo /home/foo", then do I get different file permissions depending on what path I access the file by? What if I then run chroot("/foo"), do I get different file permissions then? What if I have two namespaces, each with their own root filesystem (say "root1" and "root2"), and I mount the other namespace's root filesystem in a subdir of each: NS1: mount /dev/root2 /otherns NS2: mount /dev/root1 /otherns Now I have the following paths to the same file, do these get different permissions or the same? NS1:/foo == NS2:/otherns/foo NS2:/bar == NS1:/otherns/bar If you answered that they get different permissions, then how do you handle the massive locking performance penalty due to the extra lookups? If you answered "same permissions", then how do you explain the obvious discrepancy to the admin? I would rather just add a new ace to the directory itself which specifies that it applies to the entire tree. Then you only need to store a single acl on disk, and only have to update one acl to add a new user. The idea is nice, but as soon as you add multiple namespaces, chroots, or bind mounts the concept breaks down. For security- sensitive things like file permissions, you really do want determinate behavior regardless of the path used to access the data. Cheers, Kyle Moffett - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
--- Phillip Susi <[EMAIL PROTECTED]> wrote: > Kyle Moffett wrote: > > Going even further in this direction, the > following POSIX ACL on the > > directories will do what you want: > > > > ## Note: file owner and group are kmoffett > > u::rw- > > g::rw- > > u:lsorens:rw- > > u:mtharp:rw- > > u:mperkel:rw- > > g:randomcvsdudes:r- > > default:u::rw- > > default:g::rw- > > default:u:lsorens > > default:u:mtharp:rw- > > default:u:mperkel:rw- > > default:g:randomcvsdudes:r- > > > The problem that I have with this setup is that it > specifies an ACL on > EACH file. Yes, you can set a default on the > directory for newly > created files, but what if I want to add a user to > the access list for > that whole directory? I have to individually update > every acl on every > file in that directory. Also if you move a file > created elsewhere into > that directory, it retains its existing permissions > doesn't it? I would > rather just add a new ace to the directory itself > which specifies that > it applies to the entire tree. Then you only need > to store a single acl > on disk, and only have to update one acl to add a > new user. > > In the model I'm suggesting files and directories no longer have permissions so ACLs go away. Only users, groups, managers, applications, and other objects have permissions. So if you move a file into the tree then everything that has permission to that tree has rights to the file. Marc Perkel Junk Email Filter dot com http://www.junkemailfilter.com Boardwalk for $500? In 2007? Ha! Play Monopoly Here and Now (it's updated for today's economy) at Yahoo! Games. http://get.games.yahoo.com/proddesc?gamekey=monopolyherenow - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
--- Michael Tharp <[EMAIL PROTECTED]> wrote: > Marc Perkel wrote: > > That not a problem - it's a feature. In such a > > situation the person would get a general file > creation > > error. > > Feature or not, it's still vulnerable to probing by > malicious users. If > there are create permissions on the directory, the > invisibility is not > perfect. In a real world situation I would think that users probing for invisible files is more secure that users knowing the names of files that they have no access to. > > > Although it isn't likely people would structure > > files with invisible files in directories that the > > user has create permissions [...] > > ... /tmp ... You're still thinking inside the box. Let's take the tmp directory for example. /tmp wpuld probably g away in favor of persomal /tmp directories. As we all know, /tmp is the source of a lot of vulnerabilities. One might put a name translation mask on the /tmp name in the file name translation system. For example: /tmp -> my /tmp Thus files written to /tmp would become /mperkel/tmp and users wouldn't be able to see other users /tmp files or have any name conflicts. Let me explain about the concept of thinking outside the box. If you run into a problem you figure out a new solution. It's about finding ways to make things work rather than finding ways to make things not work. So - we are not only talking about a name permission system but a file name translation system. Thus a user's view of the file system might not be the same for all users. In fact, let's say that mperkel is a Windows user and is just attacking to Linus as a file system. Because mperkel is in the windows group the file system appears as h:\home\mperkel on a native Linux level and mounts are drive letters. It would use a Windows name translation mask program that would be part of the permission/naming system. Marc Perkel Junk Email Filter dot com http://www.junkemailfilter.com Sick sense of humor? Visit Yahoo! TV's Comedy with an Edge to see what's on, when. http://tv.yahoo.com/collections/222 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
On Aug 15, 2007, at 13:19:16, Marc Perkel wrote: One of the problems with the Unix/Linux world is that your minds are locked into this one model. In order to do it right it requires the mental discipline to break out of that. The major thing that you are missing is that this "one model" has been very heavily tested over the years. People understand it, know how to use it and write software for it, and grok its limitations. There's also a vast amount of *existing* code that you can't just "deprecate" overnight; the world just doesn't work that way. The real way to get there (IE: a new model) from here (IE: the old model) is the way all Linux development is done with a lot of sensible easy- to-understand changes and refactorings. With that said, if you actually want to sit down and start writing *code* for your model, go ahead. If it turns out to be better than our existing model then I owe you a bottle of your favorite beverage. Cheers, Kyle Moffett - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
Kyle Moffett wrote: Going even further in this direction, the following POSIX ACL on the directories will do what you want: ## Note: file owner and group are kmoffett u::rw- g::rw- u:lsorens:rw- u:mtharp:rw- u:mperkel:rw- g:randomcvsdudes:r- default:u::rw- default:g::rw- default:u:lsorens default:u:mtharp:rw- default:u:mperkel:rw- default:g:randomcvsdudes:r- The problem that I have with this setup is that it specifies an ACL on EACH file. Yes, you can set a default on the directory for newly created files, but what if I want to add a user to the access list for that whole directory? I have to individually update every acl on every file in that directory. Also if you move a file created elsewhere into that directory, it retains its existing permissions doesn't it? I would rather just add a new ace to the directory itself which specifies that it applies to the entire tree. Then you only need to store a single acl on disk, and only have to update one acl to add a new user. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
--- Kyle Moffett <[EMAIL PROTECTED]> wrote: > On Aug 15, 2007, at 13:09:31, Marc Perkel wrote: > > The idea is that people have permissions - not > files. By people I > > mean users, groups, managers, applications > > etc. One might even specify that there are no > permission > > restrictions at all. Part of the process would be > that the kernel > > load what code it will use for the permission > system. It might even > > be a little perl script you write. > > > > Also - you aren't even giving permission to access > files. It's > > permission to access name patterns. One could > apply REGEX masks to > > names to determine permissions. So if you have > permission to the > > name you have permission to the file. > > Please excuse me, I'm going to go stand over in the > corner for a minute. > > *hahahahahaa hahahahahaaa hahaa hoo hee snicker > sniff* > > *wanders back into the conversation* > > Sorry about that, pardon me. > > I suspect you will find it somewhat hard to convince > *anybody* on > this list to put either a regex engine or a Perl > interpreter into the > kernel. I doubt you could even get a simple > shell-style pattern > matcher in. First of all, both of the former chew > up enormous gobs > of stack space *AND* they're NP-complete. You just > can't do such > matching even in polynomial time, let alone > something that scales > appropriately for an OS kernel like, say, O(log(n)). > > Cheers, > Kyle Moffett > Keep in mind that this is about thinking outside the box. Don't let new ideas scare you. I'm not suggesting that the kernel contain perl. I'm saying that you can let the kernel call a perl program in user space to control part of the permission system. There are examples of this in FUSE. What I'm suggesting would be very FUSE friendly. Marc Perkel Junk Email Filter dot com http://www.junkemailfilter.com Got a little couch potato? Check out fun summer activities for kids. http://search.yahoo.com/search?fr=oni_on_mail&p=summer+activities+for+kids&cs=bz - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
Marc Perkel wrote: > That not a problem - it's a feature. In such a > situation the person would get a general file creation > error. Feature or not, it's still vulnerable to probing by malicious users. If there are create permissions on the directory, the invisibility is not perfect. > Although it isn't likely people would structure > files with invisible files in directories that the > user has create permissions [...] ... /tmp ... > [...] it is logical that if I > put a file in a place where the user has no rights I > want it to stay there. Currently the user can delete > files where they have no rights. Indeed. The sticky bit works around this, but IMHO it's a hack. > I might also want to restrict the kind of a user can > createor give permission to create only certian file > names. > > /etc/vz/conf/*.conf - create - readonly - self-rw > /etc/vz/conf - deny > > This would allow the user to read all *.conf files, > create new *.conf files, and full permissions to > read/write/delete files that the user created but not > files that others created. If listing a directory then > only the *.conf files would appear even if other files > are in the directory. It'd be interesting to find a use case for this, but that's no reason not to provide the functionality. > Marc Perkel > Junk Email Filter dot com > http://www.junkemailfilter.com -- m. tharp - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
--- Kyle Moffett <[EMAIL PROTECTED]> wrote: . > > > The important point here is that directories don't > really exist. > > Except they do, and without directories the > performance of your > average filesystem is going to suck. > Actually you would get a speed improvement. You hash the full name and get the file number. You don't have to break up the name into sections except for evaluating name permissions. The important concept here is that files and name aren't stored by levels of directories. The name points to the file number. Directory levels are emulated based on name separation characters or any other algorithm that you want to use. One could create a file system and permission system that gets rid of the concept of directories entirely if one chooses to. That's outside the box big time. Marc Perkel Junk Email Filter dot com http://www.junkemailfilter.com Shape Yahoo! in your own image. Join our Network Research Panel today! http://surveylink.yahoo.com/gmrs/yahoo_panel_invite.asp?a=7 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
On Aug 15, 2007, at 13:09:31, Marc Perkel wrote: The idea is that people have permissions - not files. By people I mean users, groups, managers, applications etc. One might even specify that there are no permission restrictions at all. Part of the process would be that the kernel load what code it will use for the permission system. It might even be a little perl script you write. Also - you aren't even giving permission to access files. It's permission to access name patterns. One could apply REGEX masks to names to determine permissions. So if you have permission to the name you have permission to the file. Please excuse me, I'm going to go stand over in the corner for a minute. *hahahahahaa hahahahahaaa hahaa hoo hee snicker sniff* *wanders back into the conversation* Sorry about that, pardon me. I suspect you will find it somewhat hard to convince *anybody* on this list to put either a regex engine or a Perl interpreter into the kernel. I doubt you could even get a simple shell-style pattern matcher in. First of all, both of the former chew up enormous gobs of stack space *AND* they're NP-complete. You just can't do such matching even in polynomial time, let alone something that scales appropriately for an OS kernel like, say, O(log(n)). Cheers, Kyle Moffett - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
--- Kyle Moffett <[EMAIL PROTECTED]> wrote: > On Aug 15, 2007, at 12:02:41, Marc Perkel wrote: > > Kyle, thinking further outside the box, files > would no longer have > > owners or permissions. Nor would > > directories. People, groups, managers, and other > objects with have > > permissions. One might tag a file with the object > that created it > > so you could implement "self" rights which might > be use to replace > > the concept of /tmp directories. > > Well, that's actually kind of close to how SELinux > works. > > This is the real fundamental design gotcha: >Our current apps *AND* admins speak "UNIX" and > "POSIX". They > don't speak "MarcPerkelOS" (or even "SELinux"). As > long as there is > not a reasonably-close-to-1-to-1 mapping between > UNIX semantics and > your "outside the box" semantics, the latter can't > really be used. > It would just involve rewriting too much code *AND* > retraining too > many admins from scratch to make it work. Hell, > even Windows and Mac > have moved towards a UNIX-like permissions system, > precisely because > it's a simple model which is relatively easy to > teach people how to > use. ACLs are just a slight modification of that > model to allow two > things: >(A) Additional user/group permissions >(B) Default permissions for new child > files/dirs/etc > > People are having a huge problem with SELinux > permissions as is, and > portions of that are a fairly standard model that's > been worked over > in various OSes for many years. I seriously doubt > that anything that > far "outside the box" is going to be feasible, at > least in the near > term. > > Good new filesystem developments are likely to be > ones which preserve > the same outer model, yet allow for > deeper/more-powerful control for > those users/admins who need it. > > Cheers, > Kyle Moffett > > Kyle, What I'm suggesting is scrapping all existing concepts and replacing them with something entirely new. Posix, Unix, SELinux go away except for an emulation layer for backwards compatibility. What I'm suggesting is to start over and do it right. If this new idea is implemented then one could implement POSIX as one of many permission modules that one could load. One could also load a WINDOWS permission model that could be used with SAMBA. This would be a new more powerful underlying layer that can be used to emulate anything you want. And it would be great for people using FUSE who could make file systems look any way they want. One of the problems with the Unix/Linux world is that your minds are locked into this one model. In order to do it right it requires the mental discipline to break out of that. Marc Perkel Junk Email Filter dot com http://www.junkemailfilter.com Looking for a deal? Find great prices on flights and hotels with Yahoo! FareChase. http://farechase.yahoo.com/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
One note before you read the rest of this: The kinds of things you are discussing here are usually called "MAC" or "Mandatory Access Control", and they are always implemented on top of an LSM *after* ordinary "DAC" or "Discretionary Access Control" (IE: file permissions) are applied. If your MAC rules are good enough you could theoretically just chmod 6777 all the files on the system and make them all root-owned, but why throw away mostly- free security? On Aug 15, 2007, at 12:36:38, Marc Perkel wrote: --- Michael Tharp <[EMAIL PROTECTED]> wrote: Kyle Moffett wrote: Basically any newly-created item in such a directory will get the permissions described by the "default:" entries in the ACL, and subdirectories will get a copy of said "default:" entries. This would work well, although I would give write permissions to a group so the entire dir wouldn't need to be re-ACLed when a user is added. I may give this a shot; I've been avoiding ACLs because they have always sounded incomplete/not useful, but the inheritance aspect sounds rather nice. Michael, my idea in this model is that there will be no permissions stored in files. So you would not have to re-ACL anything. What I'm thinking is there would be a new permission system that is completely different. It might be something like this. I am loged in as mperkel. I get all the rights of mperkel and all other objects like groups or management lists that I am a member of. Once the system has a full list of my rights it starts to compare the file name I'm trying to access to the rights I have by testing each section of the name. So if the file is /home/mperkel/files/myfile then the test would be: Big flashing "WARNING" sign number 1: "Once the system has a _full_ _list_ of..." A "full list of" anything does not scale at all. When coming up with kernel ideas, think of the biggest possible box you can imagine, then scale it up by 2-3 orders of magnitude. If it still works then you're fine; otherwise /home/mperkel/files/myfile - nothing /home/mperkel/files - nothing /home/mperkel - match - mperkel granted tree permission Rights tests would be based on trees so if you hit a tree permission they you can access anything in the tree unless you have hit a deny in the branches. All of this is based on the text strings in the file name with the "/" separator for the tests. Big flashing "WARNING" sign number 2: You are doing privileges based on pathnames, which is a massive no-no. Please go see the huge AppArmor/SELinux flame-war that occurred a month or so ago for all the reasons why pathname-based security doesn't work (and furthermore, doesn't scale). Hint: It has to do with 4 syscalls: chroot(), mount(), clone(CLONE_NEWNS), and link() The correct way of thinking of this is applying permissions to name strings. Directories will become artificial constructs. For example, one might grant permissions for files: /etc/*.conf - read only /etc - deny And so when both /etc/shadow and /tmp/ file_about_to_be_nuked_by_a_daemon point to the same block of data, you now lose *ALL* of the benefits of that model. In this example the user would be able to read any file in the /etc directory that ended in *.conf but no other files. If the object listed the /etc directory it would only show the *.conf files and no other file would appear to exist. Big flashing "WARNING" sign number 3: This means putting some kind of pattern matcher in the kernel. I don't even want to *THINK* about how unholy of a hell that would be. The way SELinux does this is by using regexes in userspace to set the appropriate *initial* labels on all the files, and let the system's automatic type transitions take care of the rest. The important point here is that directories don't really exist. Except they do, and without directories the performance of your average filesystem is going to suck. Imagine that every file has an internal number that is linked to the blocks that contain that file. Then there are file names that link to that number directly. These are called "inodes" and "hardlinks". Every file and directory is a hardlink to the inode (using the inode number) containing its data. For directories the inode data blocks have more inode references, and for files they have actual data blocks. This is the model that UNIX operating systems have been using for years. Then there is a permission system that compares the name you are requesting to a permission algorithm that determines what you are allowed to do to the name that you are requesting. The "name" that you are requesting is a fundamentally bad security concept. Better is the attributes of the actual *inode* that you are requesting. Then you will not only be denied access to the /etc/passwd file, you wouldn't even be able to tell if it exists. You could theoretically do this wit
Re: Thinking outside the box on file systems
--- [EMAIL PROTECTED] wrote: > On Wed, 15 Aug 2007 09:02:41 PDT, Marc Perkel said: > > > Kyle, thinking further outside the box, files > would no > > longer have owners or permissions. Nor would > > directories. People, groups, managers, and other > > objects with have permissions. > > You gotta think *way* out of the box to come up with > a system where a "file" > isn't an object that can have some sort of ACL or > permissions on it. > Yep - way outside the box - and thus the title of the thread. The idea is that people have permissions - not files. By people I mean users, groups, managers, applications etc. One might even specify that there are no permission restrictions at all. Part of the process would be that the kernel load what code it will use for the permission system. It might even be a little perl script you write. Also - you aren't even giving permission to access files. It's permission to access name patterns. One could apply REGEX masks to names to determine permissions. So if you have permission to the name you have permission to the file. Hard links would be multiple names pointing to the same file. Simlinks would be name aliases. Marc Perkel Junk Email Filter dot com http://www.junkemailfilter.com Moody friends. Drama queens. Your life? Nope! - their life, your story. Play Sims Stories at Yahoo! Games. http://sims.yahoo.com/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
--- alan <[EMAIL PROTECTED]> wrote: > On Tue, 14 Aug 2007, Marc Perkel wrote: > > > For example. If you list a directory you only see > the > > files that you have some rights to and files where > you > > have no rights are invisible to you. If a file is > read > > only to you then you can't delete it either. > Having > > write access to a directory really means that you > have > > file create rights. You can also delete files that > you > > have write access to. You would also allocate > > permissions to manage file rights like being able > to > > set the rights of inferior users. > > Imagine the fun you will have trying to write a file > name and being told > you cannot write it for some unknown reason. > Unbeknownst to you, there is > a file there, but it is not owned by you, thus > invisible. > > Making a file system more user oriented would avoid > little gotchas like > this. The reason it is "programmer oriented" is > that those are the people > who have worked out why it works and why certain > things are bad ideas. > That not a problem - it's a feature. In such a situation the person would get a general file creation error. Although it isn't likely people would structure files with invisible files in directories that the user has create permissions it is logical that if I put a file in a place where the user has no rights I want it to stay there. Currently the user can delete files where they have no rights. I might also want to restrict the kind of a user can createor give permission to create only certian file names. /etc/vz/conf/*.conf - create - readonly - self-rw /etc/vz/conf - deny This would allow the user to read all *.conf files, create new *.conf files, and full permissions to read/write/delete files that the user created but not files that others created. If listing a directory then only the *.conf files would appear even if other files are in the directory. Marc Perkel Junk Email Filter dot com http://www.junkemailfilter.com Building a website is a piece of cake. Yahoo! Small Business gives you all the tools to get online. http://smallbusiness.yahoo.com/webhosting - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
On Wed, 15 Aug 2007 09:02:41 PDT, Marc Perkel said: > Kyle, thinking further outside the box, files would no > longer have owners or permissions. Nor would > directories. People, groups, managers, and other > objects with have permissions. You gotta think *way* out of the box to come up with a system where a "file" isn't an object that can have some sort of ACL or permissions on it. pgpenUmJkNIfu.pgp Description: PGP signature
Re: Thinking outside the box on file systems
On Aug 15, 2007, at 12:02:41, Marc Perkel wrote: Kyle, thinking further outside the box, files would no longer have owners or permissions. Nor would directories. People, groups, managers, and other objects with have permissions. One might tag a file with the object that created it so you could implement "self" rights which might be use to replace the concept of /tmp directories. Well, that's actually kind of close to how SELinux works. This is the real fundamental design gotcha: Our current apps *AND* admins speak "UNIX" and "POSIX". They don't speak "MarcPerkelOS" (or even "SELinux"). As long as there is not a reasonably-close-to-1-to-1 mapping between UNIX semantics and your "outside the box" semantics, the latter can't really be used. It would just involve rewriting too much code *AND* retraining too many admins from scratch to make it work. Hell, even Windows and Mac have moved towards a UNIX-like permissions system, precisely because it's a simple model which is relatively easy to teach people how to use. ACLs are just a slight modification of that model to allow two things: (A) Additional user/group permissions (B) Default permissions for new child files/dirs/etc People are having a huge problem with SELinux permissions as is, and portions of that are a fairly standard model that's been worked over in various OSes for many years. I seriously doubt that anything that far "outside the box" is going to be feasible, at least in the near term. Good new filesystem developments are likely to be ones which preserve the same outer model, yet allow for deeper/more-powerful control for those users/admins who need it. Cheers, Kyle Moffett - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
--- Michael Tharp <[EMAIL PROTECTED]> wrote: > Kyle Moffett wrote: > > Basically any newly-created item in such a > directory will get the > > permissions described by the "default:" entries in > the ACL, and > > subdirectories will get a copy of said "default:" > entries. > > This would work well, although I would give write > permissions to a group > so the entire dir wouldn't need to be re-ACLed when > a user is added. I > may give this a shot; I've been avoiding ACLs > because they have always > sounded incomplete/not useful, but the inheritance > aspect sounds rather > nice. Michael, my idea in this model is that there will be no permissions stored in files. So you would not have to re-ACL anything. What I'm thinking is there would be a new permission system that is completely different. It might be something like this. I am loged in as mperkel. I get all the rights of mperkel and all other objects like groups or management lists that I am a member of. Once the system has a full list of my rights it starts to compare the file name I'm trying to access to the rights I have by testing each section of the name. So if the file is /home/mperkel/files/myfile then the test would be: /home/mperkel/files/myfile - nothing /home/mperkel/files - nothing /home/mperkel - match - mperkel granted tree permission Rights tests would be based on trees so if you hit a tree permission they you can access anything in the tree unless you have hit a deny in the branches. All of this is based on the text strings in the file name with the "/" separator for the tests. The correct way of thinking of this is applying permissions to name strings. Directories will become artificial constructs. For example, one might grant permissions for files: /etc/*.conf - read only /etc - deny In this example the user would be able to read any file in the /etc directory that ended in *.conf but no other files. If the object listed the /etc directory it would only show the *.conf files and no other file would appear to exist. The important point here is that directories don't really exist. Imagine that every file has an internal number that is linked to the blocks that contain that file. Then there are file names that link to that number directly. Then there is a permission system that compares the name you are requesting to a permission algorithm that determines what you are allowed to do to the name that you are requesting. For example, you want to list all file names /etc/*. Each name is fetched and your permissions are compared to each item and you get a list of names that you have some permission to access. If you have no permission to a name that exists then you don't see the name. Thus, suppose you have this permission /etc/pass* - deny Then you will not only be denied access to the /etc/passwd file, you wouldn't even be able to tell if it exists. The root user for compatibility would have permissions to everything. It would be like a super manager. Managers would be objects that have limited ability to alter the permissions for other users or objects that they manage. I'm also thinking there would be a "kernel" user which would be a level above the root user where the kernel would have access to files that even the root user can't see (unless debug modes are set) so that some files can be system only or readable by root but writable by kernel. Marc Perkel Junk Email Filter dot com http://www.junkemailfilter.com Park yourself in front of a world of choices in alternative vehicles. Visit the Yahoo! Auto Green Center. http://autos.yahoo.com/green_center/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
--- Kyle Moffett <[EMAIL PROTECTED]> wrote: > > ## Note: file owner and group are kmoffett > u::rw- > g::rw- > u:lsorens:rw- > u:mtharp:rw- > u:mperkel:rw- > g:randomcvsdudes:r- > default:u::rw- > default:g::rw- > default:u:lsorens > default:u:mtharp:rw- > default:u:mperkel:rw- > default:g:randomcvsdudes:r- Kyle, thinking further outside the box, files would no longer have owners or permissions. Nor would directories. People, groups, managers, and other objects with have permissions. One might tag a file with the object that created it so you could implement "self" rights which might be use to replace the concept of /tmp directories. Marc Perkel Junk Email Filter dot com http://www.junkemailfilter.com Got a little couch potato? Check out fun summer activities for kids. http://search.yahoo.com/search?fr=oni_on_mail&p=summer+activities+for+kids&cs=bz - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
Kyle Moffett wrote: > Basically any newly-created item in such a directory will get the > permissions described by the "default:" entries in the ACL, and > subdirectories will get a copy of said "default:" entries. This would work well, although I would give write permissions to a group so the entire dir wouldn't need to be re-ACLed when a user is added. I may give this a shot; I've been avoiding ACLs because they have always sounded incomplete/not useful, but the inheritance aspect sounds rather nice. > So yes, such functionality is nice; even more so because we already have > it. I think if you were really going to "extend" a UNIX filesystem it > would need to be in 2 directions: > (A) Handling disk failures by keeping multiple copies of important > files. This is ZFS' bailiwick, no? I'd love to see the licensing issues resolved, because if it can control level of redundancy on a per-file/directory basis, I would be a very happy man. > (B) Have version-control support This might be pushing it, but hey, we *are* talking about the future here. > (C) Allowing distributed storage (also lazy synchronization and > offline modification support) I'd really love to see distributed storage not suck. Everything I've seen requires myriad daemons and ugly configuration. > With some appropriate modifications and hooks, GIT actually comes pretty > close here. For larger files it needs to use a "list-of-4MB-chunks" > approach to minimize the computation overhead for committing a > randomly-modified file. The "index" of course would be directly read > and modified by vfs calls and via mapped memory. Merge handling would > need careful integration, preferably with allowing custom > default-merge-handlers per subtree. There would be lots more design > issues to work out, but it's something to think about Now you're just being silly ;) > Cheers, > Kyle Moffett -- m. tharp - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
On Aug 15, 2007, at 09:30:21, Lennart Sorensen wrote: On Wed, Aug 15, 2007 at 09:02:37AM -0400, Michael Tharp wrote: Personally, what I'd like to see is a better way of dealing with propagation of ownership. Currently, in order to allow "collaboration" directories where a directory tree is owned by a certain group and anyone in that group can write and create files, one has to change the system umask, use a magical bit on the collaboration directory to propagate group ownership, and create a group for every user on the system in order to keep their personal files safe with the new umask. This seems highly flawed. I suggest that propagation of group ownership should be the default mode, not a special one, and that the group-writable permissions should also be propagated to new files and directories. This way, the user's home directory would remain 0755, while the collaboration directory could be 0775, without any changing of umasks. Posix ACLs seem to solve most group permissions issues and control of permission propegation. It actually works quite well on Linux. I am surprised if there aren't lots of people already using it. Going even further in this direction, the following POSIX ACL on the directories will do what you want: ## Note: file owner and group are kmoffett u::rw- g::rw- u:lsorens:rw- u:mtharp:rw- u:mperkel:rw- g:randomcvsdudes:r- default:u::rw- default:g::rw- default:u:lsorens default:u:mtharp:rw- default:u:mperkel:rw- default:g:randomcvsdudes:r- Basically any newly-created item in such a directory will get the permissions described by the "default:" entries in the ACL, and subdirectories will get a copy of said "default:" entries. So yes, such functionality is nice; even more so because we already have it. I think if you were really going to "extend" a UNIX filesystem it would need to be in 2 directions: (A) Handling disk failures by keeping multiple copies of important files. (B) Have version-control support (C) Allowing distributed storage (also lazy synchronization and offline modification support) With some appropriate modifications and hooks, GIT actually comes pretty close here. For larger files it needs to use a "list-of-4MB- chunks" approach to minimize the computation overhead for committing a randomly-modified file. The "index" of course would be directly read and modified by vfs calls and via mapped memory. Merge handling would need careful integration, preferably with allowing custom default-merge-handlers per subtree. There would be lots more design issues to work out, but it's something to think about Cheers, Kyle Moffett - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
On Wed, Aug 15, 2007 at 09:02:37AM -0400, Michael Tharp wrote: > This jumped out at me right away. In such a system, an attacker with > write permissions on a "sticky" directory like /tmp could probe for > others' files by attempting to create them and recording all cases where > permission was denied due to an existing, hidden file. But of course, > this was just an example of something a less UNIX-y permission scheme > could do, not a key part of such a design. > > Personally, what I'd like to see is a better way of dealing with > propagation of ownership. Currently, in order to allow "collaboration" > directories where a directory tree is owned by a certain group and > anyone in that group can write and create files, one has to change the > system umask, use a magical bit on the collaboration directory to > propagate group ownership, and create a group for every user on the > system in order to keep their personal files safe with the new umask. > This seems highly flawed. I suggest that propagation of group ownership > should be the default mode, not a special one, and that the > group-writable permissions should also be propagated to new files and > directories. This way, the user's home directory would remain 0755, > while the collaboration directory could be 0775, without any changing of > umasks. > > Of course, this would go against tradition, and cause some mayhem in the > logic responsible for magically determining permissions for new files, > but since we're talking about thinking outside of the box, I think > that's excusable :) Posix ACLs seem to solve most group permissions issues and control of permission propegation. It actually works quite well on Linux. I am surprised if there aren't lots of people already using it. Remember that existing software expects a unix style interface, since they are unix programs. Anything you invent MUST be compatible with the standard unix view of things, even if it offers additional options. -- Len Sorensen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
alan wrote: > Imagine the fun you will have trying to write a file name and being told > you cannot write it for some unknown reason. Unbeknownst to you, there > is a file there, but it is not owned by you, thus invisible. This jumped out at me right away. In such a system, an attacker with write permissions on a "sticky" directory like /tmp could probe for others' files by attempting to create them and recording all cases where permission was denied due to an existing, hidden file. But of course, this was just an example of something a less UNIX-y permission scheme could do, not a key part of such a design. Personally, what I'd like to see is a better way of dealing with propagation of ownership. Currently, in order to allow "collaboration" directories where a directory tree is owned by a certain group and anyone in that group can write and create files, one has to change the system umask, use a magical bit on the collaboration directory to propagate group ownership, and create a group for every user on the system in order to keep their personal files safe with the new umask. This seems highly flawed. I suggest that propagation of group ownership should be the default mode, not a special one, and that the group-writable permissions should also be propagated to new files and directories. This way, the user's home directory would remain 0755, while the collaboration directory could be 0775, without any changing of umasks. Of course, this would go against tradition, and cause some mayhem in the logic responsible for magically determining permissions for new files, but since we're talking about thinking outside of the box, I think that's excusable :) -- m. tharp - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
The ACLs that were added to Linux were a step in the right direction but very incomplete. What should be is a complex permission system that would allow fine grained permissions and inherentance masks to control what permission are granted when someone moves new files into a directory. Instead of just root and users there would be mid level roles where users and objects had management authority over parts of the system and the roles can be defined in a very flexible way. For example, rights might change during "business hours". The problem with complex permission systems is, well, they are complex... I'd still go for the UNIX KISS philosophy and the rather easy permission system, as it is easier to manage. Windows has all that great permission stuff, but if you look at the reality, hardly anybody uses it due to its complexity. Tim - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Thinking outside the box on file systems
On Tue, 14 Aug 2007, Marc Perkel wrote: For example. If you list a directory you only see the files that you have some rights to and files where you have no rights are invisible to you. If a file is read only to you then you can't delete it either. Having write access to a directory really means that you have file create rights. You can also delete files that you have write access to. You would also allocate permissions to manage file rights like being able to set the rights of inferior users. Imagine the fun you will have trying to write a file name and being told you cannot write it for some unknown reason. Unbeknownst to you, there is a file there, but it is not owned by you, thus invisible. Making a file system more user oriented would avoid little gotchas like this. The reason it is "programmer oriented" is that those are the people who have worked out why it works and why certain things are bad ideas. -- Refrigerator Rule #1: If you don't remember when you bought it, Don't eat it. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Thinking outside the box on file systems
I want to throw out some concepts about a new way of thinking about file systems. But the first thing you have to do is to forget what you know about file systems now. This is a discussion about a new view of looking a file storage that is radically different and it's more easily undersood if you forget a lot of what you know. The idea is to create what seems natural to the user rather than what seems natural to the programmer. For example, if a user has not read or write access to a file then why should they be able to delete the file - or even list the file in the directory? In order to grasp this idea the idea of directory permission as you now know them needs to go away. Imagine that the file system is a database that contains file data, name data, and permission data. Loose the idea that files have an owner and a group or the attributes that we are familiar with. Think instead that users, groups, managers, application, and such are objects and there is a complex rights system that gives access to names that point to file data. For example. If you list a directory you only see the files that you have some rights to and files where you have no rights are invisible to you. If a file is read only to you then you can't delete it either. Having write access to a directory really means that you have file create rights. You can also delete files that you have write access to. You would also allocate permissions to manage file rights like being able to set the rights of inferior users. The ACLs that were added to Linux were a step in the right direction but very incomplete. What should be is a complex permission system that would allow fine grained permissions and inherentance masks to control what permission are granted when someone moves new files into a directory. Instead of just root and users there would be mid level roles where users and objects had management authority over parts of the system and the roles can be defined in a very flexible way. For example, rights might change during "business hours". I want to throw these concepts out there to inspire a new way of thinging and let Linux evolve into a more natural kind of file system rather than staying ture to it's ancient roots. Of course there would be an emulation layer to keep existing apps happy but I think that Linux will never be truly what it could be unless it breaks away from the limitations of the past. Anyhow, I'm going to stop at this just to let these ideas settle in. In my mind there's a lot more detail but let's see where this goes. Marc Perkel Marc Perkel Junk Email Filter dot com http://www.junkemailfilter.com Fussy? Opinionated? Impossible to please? Perfect. Join Yahoo!'s user panel and lay it on us. http://surveylink.yahoo.com/gmrs/yahoo_panel_invite.asp?a=7 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/