Re: directories of interest, a multiplicity alternative to CWD
Good comments, David. David Green wrote: So the set of default standard dirs would just be a hash of IO objects: $IO::DOI{home}, $IO::DOI{docs}, etc. Actually, different OS's would provide different sets of standard named dirs, and you should be able to import them: # Assume I'm running on a Mac, so $IO::DOI::MacOSX is automatically loaded use IO::DOI ; # names that ::MacOSX makes available say $Home; # /Users/david say $Music;# /Users/david/Music say $Downloads;# /Users/david/Junk drawer Similarly, I think it very reasonable that for OS's like Windows where it is commonplace for paths to start with a particular disk volume, then $IO::DOI::Windows would automatically load a DOI for each mounted disk; eg that $C or $E etc would be pre-defined. There will be a few names that should be standard as much as possible across OS's, e.g. "Home" even though on a Mac the dir is actually called "Users". "Trash" might be another one (which will be undef if the OS doesn't handle it). "Trash" wouldn't be so simple since there are multiple ".trash" directories; for starters, at least one for each logical disk volume, and also one inside each home folder under Users. If we had a "Trash", it would have to pick one of those. Anyway, this is a minor issue that doesn't need to be worried about now. This doesn't address the security side of things; dir objects might have a flag you can set so that it will warn you (perhaps fatally) if you try to use $dir.parent or "$dir/..", etc., but you could always get to an outside dir some other way. I think a more explicit way to set a chroot is better, such as: $IO::Root = $*CWD; $IO::Root = $Home; temp $IO::Root = $IO::DOI{Docs}; I would say that by design a "$dir/.." or $dir.parent should simply be impossible; if someone tried to write that, they would get a subdir named "parent" or "..", or a fatal warning if that isn't possible. Similarly, if a Perl thread does not see any DOI at all, then they effectively are denied access to the filesystem period. Hm, undef $IO::Root? Of course, that still doesn't cover everything your suggestion does, because your way allows for multiple roots. But you also weren't suggesting a specific syntax, and I'm leaning to something like my example above. Perhaps along the lines of: $IO::Root{"file"} = "/"; # default root (assumes "file://") $IO::Root{"http"} = "http://";; # means any website $IO::Root{"ftp"} = "ftp://";;# etc. What I had in mind was that every canonical DOI just lives as a member of a single hash, and deleting the hash key is how you remove one. If you have a separate variable for a single one then any use of it would be semantically equivalent to substituting a hash access in its place, and the variable would be undefined if the corresponding DOI doesn't exist, like a Perl 5 weakref when its target is garbage-collected. Every time you use IO::Some_protocol_name, it would set a default $IO::Root{protocol-name}. But there's nothing special about the names; you can add or change $IO::Root as you wish. $IO::Root{"file"} = "/foo"; $IO::Root{"more files"} = "/bar"; # Now can access only files that come under /foo or /bar $IO::Root<$_>.delete for «file "more files"»; # Now there are no more file:// roots, cannot access any files $IO::Root = "http://localhost/~david/";; # Now can access only URLs from my section of the local website Hm, having just written that, it obviously should be the case that $IO::Root should be a hash with all the available "file:" roots, i.e. $IO::Root is a hash-of-hashes where the keys are {protocol-name}{arbitrary-name}. And the default arbitrary-name might just be "default". -- Darren Duncan
Re: directories of interest, a multiplicity alternative to CWD
On 2009-Aug-19, at 2:08 am, Darren Duncan wrote: %DOI{'mycwd'} = %DOI{'fscwd'}; %DOI{'mycwd'} ~= 'subdir'; # later my $fh = IO.open( 'mycwd/myfile.txt' ); For ease of use, we can still have vars like $*CWD, which might be an alias for a doi with a specific name. I've been thinking of something similar, but you should be able to do this with any directory object. my $dir = $*CWD; # returns an IO object, not merely a string my $dir ~= "subdir"; # $dir is now an object representing the subdir my $file = io "$dir/filename"; { temp $*CWD = $dir; ... } So the set of default standard dirs would just be a hash of IO objects: $IO::DOI{home}, $IO::DOI{docs}, etc. Actually, different OS's would provide different sets of standard named dirs, and you should be able to import them: # Assume I'm running on a Mac, so $IO::DOI::MacOSX is automatically loaded use IO::DOI ; # names that ::MacOSX makes available say $Home; # /Users/david say $Music;# /Users/david/Music say $Downloads;# /Users/david/Junk drawer There will be a few names that should be standard as much as possible across OS's, e.g. "Home" even though on a Mac the dir is actually called "Users". "Trash" might be another one (which will be undef if the OS doesn't handle it). This doesn't address the security side of things; dir objects might have a flag you can set so that it will warn you (perhaps fatally) if you try to use $dir.parent or "$dir/..", etc., but you could always get to an outside dir some other way. I think a more explicit way to set a chroot is better, such as: $IO::Root = $*CWD; $IO::Root = $Home; temp $IO::Root = $IO::DOI{Docs}; Similarly, if a Perl thread does not see any DOI at all, then they effectively are denied access to the filesystem period. Hm, undef $IO::Root? Of course, that still doesn't cover everything your suggestion does, because your way allows for multiple roots. But you also weren't suggesting a specific syntax, and I'm leaning to something like my example above. Perhaps along the lines of: $IO::Root{"file"} = "/"; # default root (assumes "file://") $IO::Root{"http"} = "http://";; # means any website $IO::Root{"ftp"} = "ftp://";;# etc. Every time you use IO::Some_protocol_name, it would set a default $IO::Root{protocol-name}. But there's nothing special about the names; you can add or change $IO::Root as you wish. $IO::Root{"file"} = "/foo"; $IO::Root{"more files"} = "/bar"; # Now can access only files that come under /foo or /bar $IO::Root<$_>.delete for «file "more files"»; # Now there are no more file:// roots, cannot access any files $IO::Root = "http://localhost/~david/";; # Now can access only URLs from my section of the local website Hm, having just written that, it obviously should be the case that $IO::Root should be a hash with all the available "file:" roots, i.e. $IO::Root is a hash-of-hashes where the keys are {protocol-name} {arbitrary-name}. And the default arbitrary-name might just be "default". -David
Re: directories of interest, a multiplicity alternative to CWD
Jon Lang wrote: 'home' should be spelled '~'. Yes, of course. And conceptually a DOI can be any string at all. Logically we'd probably have non-alpha names for many of the common/standard ones. -- Darren Duncan
Re: directories of interest, a multiplicity alternative to CWD
'home' should be spelled '~'. -- Jonathan "Dataweaver" Lang
Re: directories of interest, a multiplicity alternative to CWD
Darren Duncan wrote: The named filesystem roots can be defined or altered at runtime by Perl code, and each one is defined within the context of another. I should clarify my intention here, which is that each DOI is mapped behind the scenes by Perl to a standalone absolute filesystem url, and when you define one DOI in terms of another, their definitions are *not* bound together. For example, if you define DOI "b" as subdir "foo" of DOI "a", and then you later redefine what "a" is, then "b" does *not* automatically update to be the "foo" subdir of the altered "a", but rather keeps its original meaning. So this keeps things simpler and avoids action at a distance. Well the only "action at a distance" is that any subsequently executed code with a p{a/bar/baz} will use the new meaning of "a". -- Darren Duncan
Re: directories of interest, a multiplicity alternative to CWD
Richard Hainsworth wrote: I think this is a much more flexible system than those suggested so far. It seems to me that this approach - lets the OS and the implementation deal with pathways that are valid (taking into account locale and OS constraints). - defines only that part of the location/directory tree/file system on which perl6 programs operate, rather than having to understand the whole environment in which software could run - delegates to a module if perl6 is to be used to manage a specific environment/OS - offers more future proofing and portability than constraining what can or cannot be in a pathname. - allows users to configure via Policy.pm standard pathnames to something non-standard for development and testing, leaving the software itself unchanged. Richard, thanks for your feedback. I think you raise a lot of good points about further consequences of my proposal. I think I'll also raise the point that depending on how its further designed, my proposal could help with security as well, by implicitly defining sandboxes as it were. If it is interpreted that you can not refer to something that is a parent or sibling dir of a DOI by way of that DOI, hence treating it like an absolute root, then that means that any code can only access parts of the filesystem that are visible-to-it DOIs or subdirs of said, because there's simply no syntax to reference anything outside. There is no '..' in my scheme. For example, if a Perl thread starts out seeing only a single DOI representing the filesystem root, and they update it to some subdir, then they have just chrooted themselves. Similarly, if a Perl thread does not see any DOI at all, then they effectively are denied access to the filesystem period. So in general, I hope people can see that a natural consequence of my proposal is that we also have an elegant means for security or virtualization, which among other things can aid Perl's support for running in restrictive environments, such as a "cloud", or whatever. Note, I want to make clear that my proposal is *not* about abstracting away all kinds of IO into a single uri scheme, but only abstracting the file system. Not that it can't be further extended in appropriate ways. -- Darren Duncan
Re: directories of interest, a multiplicity alternative to CWD
Timothy, thanks for your feedback. My proposal was never about the syntax, which I stated at the end, but rather my syntax was just illustrative. I actually meant for p{} or whatever syntax to be used, but I didn't recall those details and was keeping it simple. And the use of a %DOI hash was just illustrative, meant to evoke how you set signal handlers in Perl 5, and in no way does that syntax need to be used. The idea of starting all paths with "/" rather than "/" was to first imply that there was no single root, and also for consistency in that the one representing "/" has the *same* syntax as all the others. I wasn't actually proposing using "fsroot", etc as the names; those were just examples, and in fact my proposal was about users picking what names they want, so my list of 6 examples, such as "docs", was meant to represent something users might pick and name for their own circumstances, not that these are built-ins. Come to think of it, if a "" is allowed to be any string at all, like a hash key can, then the system-defined one representing the real filesystem root, "/", could just be the empty string, which means that you end up using the exact same "/" syntax as you're already used to, and it means the same thing. So then under my proposal, its more that the syntax for relative urls like "foo/bar" has had its meaning changed so "foo" is a doi rather than a subdir of the single CWD. While I didn't explicitly state it, the roots that came system-defined could conceivably also be altered by the user, eg they could redefine whatever-"fsroot"-is-called, which incidentally would result in them chroot-ing themselves. This also means that part of what I had in mind is that you can't use a relative url to go to some parent/sibling of a "doi" by way of that "doi", meaning all urls visually are absolute. This might mean that the name "bookmark" isn't as appropriate if it implies you can go to a parent of the "doi" by way of it, but this is a minor quibble and if the feature is called "bookmark" I won't really object. -- Darren Duncan Timothy S. Nelson wrote: On Wed, 19 Aug 2009, Darren Duncan wrote: My proposal is to have all filesystem paths as seen within Perl being relative paths, and that there are multiple filesystem roots which can be referred to by name and each relative path is explicitly relative to a named root; each of these named roots is called a "directory of interest" or "doi". I briefly mentioned a similar idea, but with no theoretical background :). I would've called them "bookmarks" instead of "doi", though. For those who shied away from Darren's large theoretical background, fearing that it would lead to "pie-in-the-sky" or something, the simple version of his proposal was to have a magic global hash, and have it work like: %*DOI = "/home/foo"; $path = p{home/Music}; # $path points at /home/foo/Music I disagree with the syntax, but the idea has merit. Your idea for making everything a relative path isn't one that appeal to me. But having a predefined set of bookmarks that could be used is certainly an interesting idea. I don't like your syntax, though :). But see below. fsroot - the root of the real file system, analogous to "/" I think we should stick with / as it's well-understood. Unless the Windows users among us see some advantage to fsroot -- I'd be happy to discuss it with them. fscwd - the dir that was the fs CWD when Perl started up Stick with $*CWD for this, I reckon, and have it implicit in all paths that don't start with an absolute path. But all the other variables below should be absolute paths. docs - the dir that contains the usual files we want to work with temp - the dir where we can put temporary files home - the current user's home dir mycwd - some other cwd-dir, which is virtual-changeable in Perl Vote yes to temp and home. Don't understand what you mean by "docs". So to refer to common things on a Unix system like the fully-qualified ways, you can write paths like: fsroot/home/me/myfile fsroot/usr/bin/perl fsroot This is a great example of why fsroot is a bad idea. For something analogous to traditional CWD sans modifications: fscwd/mynewfile.txt fscwd/lib/doit.pl ...and a great example of why fscwd is a bad idea. To define a new doi at runtime, something like: %DOI{'mycwd'} = %DOI{'fscwd'}; %DOI{'mycwd'} ~= 'subdir'; # later my $fh = IO.open( 'mycwd/myfile.txt' ); If I were doing it, I'd make it look like this: %bookmark = $*CWD; %bookmark.push('subdir'); # later my $fh = IO.open( p{#mycwd#/myfile.txt} ); The only part I don't like is marking the "mycwd" with the hashes. I think we can't do without markers altogether because it would absolutely confuse everyone :). Putting a hash before and after is also a bad idea because it's like nothing else in Perl. The only other idea I can think of i
Re: directories of interest, a multiplicity alternative to CWD
On Wed, 19 Aug 2009, Darren Duncan wrote: My proposal is to have all filesystem paths as seen within Perl being relative paths, and that there are multiple filesystem roots which can be referred to by name and each relative path is explicitly relative to a named root; each of these named roots is called a "directory of interest" or "doi". I briefly mentioned a similar idea, but with no theoretical background :). I would've called them "bookmarks" instead of "doi", though. For those who shied away from Darren's large theoretical background, fearing that it would lead to "pie-in-the-sky" or something, the simple version of his proposal was to have a magic global hash, and have it work like: %*DOI = "/home/foo"; $path = p{home/Music}; # $path points at /home/foo/Music I disagree with the syntax, but the idea has merit. Your idea for making everything a relative path isn't one that appeal to me. But having a predefined set of bookmarks that could be used is certainly an interesting idea. I don't like your syntax, though :). But see below. fsroot - the root of the real file system, analogous to "/" I think we should stick with / as it's well-understood. Unless the Windows users among us see some advantage to fsroot -- I'd be happy to discuss it with them. fscwd - the dir that was the fs CWD when Perl started up Stick with $*CWD for this, I reckon, and have it implicit in all paths that don't start with an absolute path. But all the other variables below should be absolute paths. docs - the dir that contains the usual files we want to work with temp - the dir where we can put temporary files home - the current user's home dir mycwd - some other cwd-dir, which is virtual-changeable in Perl Vote yes to temp and home. Don't understand what you mean by "docs". So to refer to common things on a Unix system like the fully-qualified ways, you can write paths like: fsroot/home/me/myfile fsroot/usr/bin/perl fsroot This is a great example of why fsroot is a bad idea. For something analogous to traditional CWD sans modifications: fscwd/mynewfile.txt fscwd/lib/doit.pl ...and a great example of why fscwd is a bad idea. To define a new doi at runtime, something like: %DOI{'mycwd'} = %DOI{'fscwd'}; %DOI{'mycwd'} ~= 'subdir'; # later my $fh = IO.open( 'mycwd/myfile.txt' ); If I were doing it, I'd make it look like this: %bookmark = $*CWD; %bookmark.push('subdir'); # later my $fh = IO.open( p{#mycwd#/myfile.txt} ); The only part I don't like is marking the "mycwd" with the hashes. I think we can't do without markers altogether because it would absolutely confuse everyone :). Putting a hash before and after is also a bad idea because it's like nothing else in Perl. The only other idea I can think of is to come up with another quote adverb/pragma pair. Something like: use hashinterpolate %*bookmark; ...and have the :hi (short for :hashinterpolate) quote modifier make $mycwd refer to %*bookmark, and then have :path automatically do :hi as well, unless :qq is used, or something. Anyway, I home someone who knows more about Perl interpolation than me has an idea here. Anyway, I like my terminology better, because it's less scary, and I've shaved off some of the scariest bits of syntax from your proposal, but there is still some work to be done on it here. HTH, - | Name: Tim Nelson | Because the Creator is,| | E-mail: wayl...@wayland.id.au| I am | - BEGIN GEEK CODE BLOCK Version 3.12 GCS d+++ s+: a- C++$ U+++$ P+++$ L+++ E- W+ N+ w--- V- PE(+) Y+>++ PGP->+++ R(+) !tv b++ DI D G+ e++> h! y- -END GEEK CODE BLOCK-
Re: directories of interest, a multiplicity alternative to CWD
I think this is a much more flexible system than those suggested so far. It seems to me that this approach - lets the OS and the implementation deal with pathways that are valid (taking into account locale and OS constraints). - defines only that part of the location/directory tree/file system on which perl6 programs operate, rather than having to understand the whole environment in which software could run - delegates to a module if perl6 is to be used to manage a specific environment/OS - offers more future proofing and portability than constraining what can or cannot be in a pathname. - allows users to configure via Policy.pm standard pathnames to something non-standard for development and testing, leaving the software itself unchanged. Darren Duncan wrote: When you write a path, then rather than rooting it with "/", you instead root it with "/". Examples of "doi" can be: fsroot - the root of the real file system, analogous to "/" fscwd - the dir that was the fs CWD when Perl started up docs - the dir that contains the usual files we want to work with temp - the dir where we can put temporary files home - the current user's home dir mycwd - some other cwd-dir, which is virtual-changeable in Perl To define a new doi at runtime, something like: %DOI{'mycwd'} = %DOI{'fscwd'}; %DOI{'mycwd'} ~= 'subdir'; # later my $fh = IO.open( 'mycwd/myfile.txt' ); For ease of use, we can still have vars like $*CWD, which might be an alias for a doi with a specific name. Note, please ignore my specific syntax for denoting Path objects. I defer to other discussions and synopsis for those details, and mainly what I'm trying to put across here is concepts. My main point here is that we effectively can have multiple CWD in the same Perl thread, and it shouldn't be too much trouble to make this work well. -- Darren Duncan
directories of interest, a multiplicity alternative to CWD
All this discussion about file paths, particularly about "current working directory", has inspired me to raise another idea for how this could be done, which may be a little more abstract or different than you're used to, but may also be an elegant solution. My proposal draws inspiration from a design aspect of my Muldis D language in regards to how fully-qualified names for DBMS entities are written out, including the names of types, routines, variables, and so on, which is in turn inspired by how SQL DBMSs tend to name things, where you qualify them with a database name plus schema name plus package name if applicable and so on; for a DBMS, the hierarchy of namespaces is like a filesystem, since everything relevant lives in there, and so schema names etc are like directories and variables or routines etc are like files, or alternately they are like namespaces in a programming language defined by packages or classes etc for their routines or types or variables. The Muldis D concept is that there are some primary namespaces whose definition is relative to where they are invoked; and invoking entities by way of these means "I refer to this entity in terms of a particular context of mine", such as "in my package" ($pkg.foo) or "in my schema" ($sdp.foo or $dep.bar.foo etc). My proposal is to have all filesystem paths as seen within Perl being relative paths, and that there are multiple filesystem roots which can be referred to by name and each relative path is explicitly relative to a named root; each of these named roots is called a "directory of interest" or "doi". The named filesystem roots can be defined or altered at runtime by Perl code, and each one is defined within the context of another. Obviously at least one must be system-defined by Perl so to be able to genesis the others, and that one corresponds to the actual filesystem root (or some other dir if the Perl process is operating in a jail). When you write a path, then rather than rooting it with "/", you instead root it with "/". Examples of "doi" can be: fsroot - the root of the real file system, analogous to "/" fscwd - the dir that was the fs CWD when Perl started up docs - the dir that contains the usual files we want to work with temp - the dir where we can put temporary files home - the current user's home dir mycwd - some other cwd-dir, which is virtual-changeable in Perl So to refer to common things on a Unix system like the fully-qualified ways, you can write paths like: fsroot/home/me/myfile fsroot/usr/bin/perl fsroot For something analogous to traditional CWD sans modifications: fscwd/mynewfile.txt fscwd/lib/doit.pl To define a new doi at runtime, something like: %DOI{'mycwd'} = %DOI{'fscwd'}; %DOI{'mycwd'} ~= 'subdir'; # later my $fh = IO.open( 'mycwd/myfile.txt' ); For ease of use, we can still have vars like $*CWD, which might be an alias for a doi with a specific name. Note, please ignore my specific syntax for denoting Path objects. I defer to other discussions and synopsis for those details, and mainly what I'm trying to put across here is concepts. My main point here is that we effectively can have multiple CWD in the same Perl thread, and it shouldn't be too much trouble to make this work well. -- Darren Duncan