Re: [boost] boost::filesystem file restrictions

[Beman Dawes]

At 05:43 AM 8/18/2003, John Torjo wrote:
>> The current approach is clearly too restrictive and isn't satisfactory.
>> Beyond the problems you mention, there really isn't a single standard 
for
>> portability. Even 8.3 names aren't portable to systems which don't 
allow
>> periods in names. A whole family of checkers is needed, giving various
>> degrees of portability. Some should be supplied with the library, but
>users
>> also need to be able to provide their own.
>> [...]
>
>>  added this:
>>
>> typedef bool (*is_portable_test)( string & candidate );
>>
>> class scoped_portability_policy : noncopyable
>> {
>> public:
>>   explicit scoped_portabiity_policy( is_portable_test f );
>>   ~scoped_portabiity_policy();
>>  };
>>
>
>I'm not sure 'portability' is the right word here.
>
>Since it can be overridden by the user, maybe a better name would be:
>is_legal_name_test - and the user can override it to suit its needs.
>
>I don't quite like is_portable_test, since I assume there is only
>one 'portability', not more (at least, this is what I think, when
>discussing portability). For instance, when talking about a portable 
name,
>I assume there is a clear definition of what that means to everybody
>(and I don't assume users could/should override that ;).
>
>
>That said, instead of a scoped portability policy, which will go rather
>bad with thread-safety, maybe, just a simple
>set_legal_name_policy( is_legal_name_test f); would look better.
>
>Users would (should) set this in main(), while there are no more threads,
>and it could play nicely with thread-safety as well.

Yes. Plus there are some other issues.

The actual interface would include boost::filesystem::path constructors 
which take an additional argument to explicitly specify a name checker 
function. In working out use cases, it seems that temporary overrides of 
the default function are best handled via these constructors. That leaves 
only the case of wishing to permanently replace the default function, and 
the simpler approach you are talking about would be better for that.

For safety, such a set_legal_name_policy() would use the 
write-once-before-read idiom to avoid a dangerous global variable. (I'm 
actually thinking of "name_check" for the type, and "set_name_check" for 
the function name.)

I'm about to post a message asking for opinions on the details of the 
policy function pointer or object.

--Beman 

___
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Re: [boost] boost::filesystem file restrictions

[John Torjo]

> The current approach is clearly too restrictive and isn't satisfactory.
> Beyond the problems you mention, there really isn't a single standard for
> portability. Even 8.3 names aren't portable to systems which don't allow
> periods in names. A whole family of checkers is needed, giving various
> degrees of portability. Some should be supplied with the library, but
users
> also need to be able to provide their own.
> [...]

>  added this:
>
> typedef bool (*is_portable_test)( string & candidate );
>
> class scoped_portability_policy : noncopyable
> {
> public:
>   explicit scoped_portabiity_policy( is_portable_test f );
>   ~scoped_portabiity_policy();
>  };
>

I'm not sure 'portability' is the right word here.

Since it can be overridden by the user, maybe a better name would be:
is_legal_name_test - and the user can override it to suit its needs.

I don't quite like is_portable_test, since I assume there is only
one 'portability', not more (at least, this is what I think, when
discussing portability). For instance, when talking about a portable name,
I assume there is a clear definition of what that means to everybody
(and I don't assume users could/should override that ;).


That said, instead of a scoped portability policy, which will go rather
bad with thread-safety, maybe, just a simple
set_legal_name_policy( is_legal_name_test f); would look better.

Users would (should) set this in main(), while there are no more threads,
and it could play nicely with thread-safety as well.


Best,
John


___
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Re: [boost] boost::filesystem file restrictions

[Jeff Garland]

>  >"Peter Dimov" <[EMAIL PROTECTED]> wrote:
>> I am not sure that it should be the responsibility of the path class to 
>> enforce some notion of portability. 

I haven't been following this whole discussion, but I'll chime in here b/c
I believe some of this might be partially due to comments I made during 
initial development.  The basic scenario for this is:
1) I'm developing a script-like application that manipulates files. In 
particular it generates a bunch of file and pathnames.
2) I'd like to develop this on one platform and expect that it will port to 
others without modification.

Therefore, I want the library to help me with this by helping me aviod non-
portable paths. Of course if I'm developing for a single platform then I 
don't need this.  I think the current design does this pretty nicely.

Jeff




___
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Re: [boost] boost::filesystem file restrictions

[Beman Dawes]

At 04:23 PM 8/15/2003, Peter Dimov wrote:
>Beman Dawes wrote:
>> At 01:40 PM 8/14/2003, Peter Dimov wrote:
>>  >
>>  >I am not sure that it should be the responsibility of the path
>>  class to >enforce some notion of portability. Wouldn't it be more
>>  appropriate to >defer the portability check, if any, to the point
>>  where the path is >actually used in a filesystem operation?
>>
>> That's too late. A real path is often made up of some native elements
>> (which the portability check doesn't apply to) and some portable
>> elements (which the portability check should be applied to).
>>
>> The earlier the error can be detected, the better. Also, a path is
>> only constructed once, but may be use multiple times.
>
>[...]
>
>> That would be easy if we accepted the native platform as the default,
>> and portable cases had to be specially coded. But my interest is in
>> portable semantics as the default.
>
>I must be missing something. What is a "portable" path useful for? A
>portable path _grammar_ (element sequence separated by '/') is certainly
>useful, it allows me to write portable _code_ that deals with paths.
Yes, to the extent the elements (the names) are portable.

>Portable path _data_ is a different story.

For sure. But we have been talking only about names which are elements of 
paths and how to string them together via a portable grammar. We haven't 
been talking about file data content at all.

--Beman

___
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Re: [boost] boost::filesystem file restrictions

[Beman Dawes]

At 08:46 PM 8/14/2003, Walter Landry wrote:

>"Peter Dimov" <[EMAIL PROTECTED]> wrote:
>> I am not sure that it should be the responsibility of the path class to
>> enforce some notion of portability. Wouldn't it be more appropriate to
>> defer the portability check, if any, to the point where the path is
>> actually used in a filesystem operation?
>
>I agree, if only because I could imagine manipulating a bunch of
>non-legal paths before actually using a legal one.
I gave that some thought during the initial design. Let's say you want to 
do some manipulation of an element (that is, a name representing a node in 
the tree). Perhaps you have a home grown runtime macro expansion scheme. 
You are probably better off holding the data in a std::string until your 
manipulations (say macro expansion) have progressed to the point it is a 
valid native or portable name or path. At that point, assign the string to 
a boost::filesystem::path.

>  We still have to
>solve the problem, but at a different place.
Yes, exactly.

>  Beman's singleton stack
>seems like a reasonable solution.  We can argue over what the default
>portability policy should be, but it almost becomes irrelevant because
>it is easy to change and forget about it.
I've been turning that solution over and over in my mind, and while it has 
some mild blemishes, it seems clearly a big improvement over the current 
design.

Among side advantages, it doesn't break any current code. So unless someone 
comes up with a killer argument against the stack approach, I'll implement 
it within a day or two.

Part of the difficulty with the current approach is documentation; I'll try 
to rectify that in the process.

--Beman

___
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Re: [boost] boost::filesystem file restrictions

[Peter Dimov]

Beman Dawes wrote:
> At 01:40 PM 8/14/2003, Peter Dimov wrote:
>  >
>  >I am not sure that it should be the responsibility of the path
>  class to >enforce some notion of portability. Wouldn't it be more
>  appropriate to >defer the portability check, if any, to the point
>  where the path is >actually used in a filesystem operation?
>
> That's too late. A real path is often made up of some native elements
> (which the portability check doesn't apply to) and some portable
> elements (which the portability check should be applied to).
>
> The earlier the error can be detected, the better. Also, a path is
> only constructed once, but may be use multiple times.

[...]

> That would be easy if we accepted the native platform as the default,
> and portable cases had to be specially coded. But my interest is in
> portable semantics as the default.

I must be missing something. What is a "portable" path useful for? A
portable path _grammar_ (element sequence separated by '/') is certainly
useful, it allows me to write portable _code_ that deals with paths.
Portable path _data_ is a different story.

___
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Re: [boost] boost::filesystem file restrictions

[Beman Dawes]

At 01:40 PM 8/14/2003, Peter Dimov wrote:
>Beman Dawes wrote:
>>
>> The current approach is clearly too restrictive and isn't
>> satisfactory. Beyond the problems you mention, there really isn't a
>> single standard for portability. Even 8.3 names aren't portable to
>> systems which don't allow periods in names. A whole family of
>> checkers is needed, giving various degrees of portability. Some
>> should be supplied with the library, but users also need to be able
>> to provide their own.
>>
>> OTOH, a function that has to be explicitly called isn't going to be
>> effective. Manual checking is too laborious in code that does a lot
>> of path manipulation. A one time I took several pages of code and
>> tried to add explicit checks. The code turned into a mess. Manual
>> checking is also
>> likely to be ignored by the very programmers who need it the most;
>> those
>> who have firm but erroneous ideas about what is or isn't portable.
>
>I am not sure that it should be the responsibility of the path class to
>enforce some notion of portability. Wouldn't it be more appropriate to
>defer the portability check, if any, to the point where the path is
>actually used in a filesystem operation?
That's too late. A real path is often made up of some native elements 
(which the portability check doesn't apply to) and some portable elements 
(which the portability check should be applied to).

The earlier the error can be detected, the better. Also, a path is only 
constructed once, but may be use multiple times.

Because it is an invariant that any fully constructed path has already been 
error checked (and operations like appends in effect construct their 
argument first if necessary to guarantee the invariant), lots of other code 
doesn't have to worry about error detection.

I'm very comfortable with applying the error check a construction time (or 
equivalent for rhs arguments), and not applying it at all for native paths. 
What I'm having trouble which is the mechanism for selecting the particular 
error function to apply.

That would be easy if we accepted the native platform as the default, and 
portable cases had to be specially coded. But my interest is in portable 
semantics as the default.

--Beman

--Beman

--Beman

___
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Re: [boost] boost::filesystem file restrictions

[Dick . Bridges]

   
 
  "Edward Diener"  
 
  <[EMAIL PROTECTED] To:  [EMAIL PROTECTED]
 
  ft.com>cc:   
 
  Sent by:   Subject: [boost] Re: 
boost::filesystem file restrictions   
  [EMAIL PROTECTED]

  s.boost.org  
 
   
 
   
 
  08/15/2003 08:35 
 
  AM   
 
  Please respond to
 
  Boost mailing list   
 
   
 
   
 




David Abrahams wrote:
> Glen Knowles <[EMAIL PROTECTED]> writes:
>
>>> From: David Abrahams [mailto:[EMAIL PROTECTED]
> portable_path("/foo/bar") <- throws on Windows

 Not sure why this would throw, what is the purpose of
 portable_path? "/foo/bar" is perfectly reasonable on Windows.
>>>
>>> It's perfectly reasonable but it doesn't have a portable meaning.
>>> It
>>> is a relative path w.r.t. the drive of the current directory.
>>
>> Almost all paths are relative w.r.t. something, the current users
>> filesystem mapping, the computer, the network.
>
> I find that somewhat compelling... but in the end it doesn't hold up.
>
>> I don't see how
>> leaving out the drive makes it less portable then leaving out the
>> computer name.
>
> It's less portable because how it is to be interpreted *with respect
> to the filesystem* can change dynamically.  Remember the name of this
> library, "filesystem"? ;->
>
> Filesystems belong to computers.  A computer's filesystem is accessed
> via an infinite tree of names (**).  How those names which correspond
> actual storage are mapped can be modified dynamically in any number of
> ways: you can have symbolic and hard links, mount drives, remote
> computers can come online or go away, non-storage devices can be
> mounted at locations in the tree etc.  The one constant is the
> structure of the tree of names which allows us to access a virtual
> location in the filesystem (as opposed to physical).
>
> A path is a set of instructions for traversing the name tree.  By any
> reasonable definition, an absolute path identifies a single virtual
> location in a filesystem's name tree, not one that can change based on
> the process state.  A path on windows that starts with '/' is a set
> of instructions which begins: "go to the root of the current
> directory path".

Correction. It does not mean that. It means go to the root directory of the
current drive. It is still not an absolute path since the current drive
changes. If one specified 'a:/', then that is an absolute path as defined
under Windows. Even if 'a:' were a removable disk, and thus could be
physically changed, it would be considered an absolute path.



I'm just a confused lurker seeking some clarification.  I thought most OSs
allowed a process filesystem to be dynamically "re-rooted" and that '/'
refers to the "current root" - whatever the OS (assuming hierarchical
filesystem[s]).  If you introduce the extra-filesystem "a:/" wrt Windows,
then why not chroot for *NIXs or the 9P messages that "re-root" the Plan 9
filesystem to a different date or file server?

I was under the impression that "/foo/bar" is a relative path wrt the
current root of any given process state.  For any given process, the
physical location of "/foo/bar" may change between points in time and, for
any two processes, the physical location of "/foo/bar" may be different at
the same point in time.

Re: [boost] boost::filesystem file restrictions

[Walter Landry]

"Peter Dimov" <[EMAIL PROTECTED]> wrote:
> I am not sure that it should be the responsibility of the path class to
> enforce some notion of portability. Wouldn't it be more appropriate to defer
> the portability check, if any, to the point where the path is actually used
> in a filesystem operation?

I agree, if only because I could imagine manipulating a bunch of
non-legal paths before actually using a legal one.  We still have to
solve the problem, but at a different place.  Beman's singleton stack
seems like a reasonable solution.  We can argue over what the default
portability policy should be, but it almost becomes irrelevant because
it is easy to change and forget about it.

Regards,
Walter Landry
[EMAIL PROTECTED]

___
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Re: [boost] boost::filesystem file restrictions

[Beman Dawes]

At 06:03 AM 8/14/2003, Walter Landry wrote:

>Greetings,
>
>I've started using boost::filesystem recently, and I'm mostly very
>happy.
Wow! A very happy user. Or at least mostly very happy. That's good news:-) 
Seriously, it is a powerful motivator to get that kind of feedback.

>  One thing bothers me, though.  Why does it implement any
>restrictions, by default, on what kind of files it allows?
See below.

>...
>I also noticed that you can't open a directory named "." or "..",
>though I think "./" and "../" both work.
Hum... I'm not sure that is currently intended.

>  Files starting or ending with spaces also don't work.

That is intended, although it may change if the portability checking scheme 
changes.

>
>I understand that I can (painfully) work around it by using a
>different context, but I don't understand why boost::filesystem wants
>to restrict me to a set of filenames that are portable.  Isn't that a
>bit too much handholding?  I don't mind having an is_portable()
>command or something similar, but it is incredibly annoying to have to
>abide by someone else's filename restrictions.
The current approach is clearly too restrictive and isn't satisfactory. 
Beyond the problems you mention, there really isn't a single standard for 
portability. Even 8.3 names aren't portable to systems which don't allow 
periods in names. A whole family of checkers is needed, giving various 
degrees of portability. Some should be supplied with the library, but users 
also need to be able to provide their own.

OTOH, a function that has to be explicitly called isn't going to be 
effective. Manual checking is too laborious in code that does a lot of path 
manipulation. A one time I took several pages of code and tried to add 
explicit checks. The code turned into a mess. Manual checking is also 
likely to be ignored by the very programmers who need it the most; those 
who have firm but erroneous ideas about what is or isn't portable.

So it comes down to an interface problem. What is the best way for the user 
specify a portability checking function to override the default checking 
function?

It would be trivial to add an additional path constructor that takes a 
checking function or function object.

But that would kill ease-of-use. It is one thing to require an additional 
constructor argument in the fairly limited use "native" case, but the 
portability checking is applied to each and every path object constructed, 
including the many path temporaries created by the automatic conversions 
from char * and string.

(That "kill ease-of-use" point might be hard to understand if you haven't 
actually written code using the library. The automatic conversions really 
do allow "script-like" programming ease, and are reasonably safe given the 
conversion result is class path.)

Also, the portability checking policy often needs to propagate to called 
functions such as third party liberties. Adding overloads of functions to 
take an additional argument is also too messy, and doesn't provide for 
automatic pass-through to lower level functions. In most (but not all) 
programs it really would be convenient if portability checking policy was a 
global. But of course we all know that "global variables are consider 
harmful". There are also several valid use cases where a program needs to 
switch back and forth between portability policies.

Another approach is to provide a way to tell class path that within a file 
scope use a policy other than the default. It would have to be carefully 
worked out to avoid ODR violations, and to ensure propagation to called 
library functions. I have not been able to come up with a workable version 
of this approach.

That was about as far as my thinking had gotten in the past. While 
composing this reply, I came up with another possible solution. Let's say 
 added this:

   typedef bool (*is_portable_test)( string & candidate );

   class scoped_portability_policy : noncopyable
   {
   public:
 explicit scoped_portabiity_policy( is_portable_test f );
 ~scoped_portabiity_policy();
};
The effect of constructing a scoped_portability_policy is to push p onto a 
stack. The destructor pops the stack. Class path uses the top stack entry 
as the current portability checker. The stack is initialized with a default 
portability checker.

Most programs which wanted a policy other than the default would just being 
something like this:

  int main()
  {
 scoped_portability_policy( posix_portability_check );
 ...
Although the stack is global, when coupled with scoped_portability_policy, 
it is safer than a single global variable (because library functions which 
push are guaranteed to pop.)

Of course it can be subverted by static or heap allocation. (Aside: Is 
there a reliable way to prevent static or heap allocation?)

I'm curious to hear what others would think of the above scheme.

--Beman

___
Unsubscribe & other changes: http://lists.

RE: [boost] boost::filesystem file restrictions

[Michael van der Westhuizen]

-Original Message-
From: Beman Dawes [mailto:[EMAIL PROTECTED]
Sent: 14 August 2003 04:56
To: Boost mailing list; [EMAIL PROTECTED]
Subject: Re: [boost] boost::filesystem file restrictions


At 06:03 AM 8/14/2003, Walter Landry wrote:

 >I've started using boost::filesystem recently, and I'm mostly very
 >happy.

> Wow! A very happy user. Or at least mostly very happy. That's good news:-)

> Seriously, it is a powerful motivator to get that kind of feedback.

Well, here's another happy user :-)

-- snip --

> I'm curious to hear what others would think of the above scheme.

Oddly enough I had a similar thought halfway through reading your 
mail - by the time I got to your proposal all I could think was "yes".

As for the stack/heap question... I don't believe there is a way to
portably and reliably tell how memory was allocated (please correct me
if I'm wrong - code would be most welcome!).

Michael
___
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[boost] boost::filesystem file restrictions

[Walter Landry]

Greetings,

I've started using boost::filesystem recently, and I'm mostly very
happy.  One thing bothers me, though.  Why does it implement any
restrictions, by default, on what kind of files it allows?  From the
documentation:

  The following are not valid name char's: x01-x1F, <, >, :, ", /, \,
  |, *, ?. Although these characters are supported by some operating
  systems, they are disallowed by so many operating systems that they
  are banned altogether.

I also noticed that you can't open a directory named "." or "..",
though I think "./" and "../" both work.  Files starting or ending
with spaces also don't work.

I understand that I can (painfully) work around it by using a
different context, but I don't understand why boost::filesystem wants
to restrict me to a set of filenames that are portable.  Isn't that a
bit too much handholding?  I don't mind having an is_portable()
command or something similar, but it is incredibly annoying to have to
abide by someone else's filename restrictions.

I found a bug report at

  https://sourceforge.net/tracker/?func=detail&atid=107586&aid=776146&group_id=7586

and a discussion about this (though it focused more on "?" and "*") at

  http://lists.boost.org/MailArchives/boost/msg46073.php

In particular, Beman Dawes said:

  What is very explicitly not a supported use case is for providing
  simply a new but still non-portable interface to non-portable
  operating system API's.  If that is the need, just call the
  non-portable operating system API's directly.

If we were to take that to the extreme, then boost::filesystem should
only accept 8.3 filenames.  What Beman seems to be missing is that he
has made an extremely nice interface to filesystems.  It is much
simpler and easier to use within C++ than the non-portable API's.  But
it is unneccesarily difficult to look at whatever files the user may
have created.  This makes the library much less useful than it might
otherwise be.

It is a trivial change to fix this (just take out the check), and I've
done this in my own copy.  I'm attaching a patch with this email.

Thanks,
Walter Landry
[EMAIL PROTECTED]


--- /home/boo/arx/arx/src/boost/{arch}/++pristine-trees/unlocked/boost/boost--ar
x/boost--arx--1.0/[EMAIL PROTECTED]/boost--arx--1.0--patch-6/./libs/filesyst
em/src/path_posix_windows.cpp   2003-07-18 13:22:15.0 -0400
+++ /home/boo/arx/arx/src/boost/./libs/filesystem/src/path_posix_windows.cpp
2003-07-31 10:37:51.0 -0400
@@ -128,17 +128,6 @@
   {
 
 //  error checking functions  /
/
-
-bool generic_name( const std::string & name )
-{
-  return name.size() != 0
-&& name.find_first_of( invalid_generic ) == std::string::npos
-&& name != "."
-&& name != ".."
-&& *name.begin() != ' '
-&& *(name.end()-1) != ' '; 
-}
-
 bool posix_name( const std::string & name )
 {
   return name.find_first_not_of( valid_posix ) == std::string::npos
@@ -308,13 +297,6 @@
 #   endif
 );
 
-  if ( context == generic && !generic_name( name ) )
-  {
-boost::throw_exception( filesystem_error(
-  "boost::filesystem::path",
-  "invalid name \"" + name + "\" in path: \"" + src + "\"" ) );
-  }
-
   m_path += name;
 }
 
___
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Re: [boost] boost::filesystem file restrictions

[Peter Dimov]

Beman Dawes wrote:
>
> The current approach is clearly too restrictive and isn't
> satisfactory. Beyond the problems you mention, there really isn't a
> single standard for portability. Even 8.3 names aren't portable to
> systems which don't allow periods in names. A whole family of
> checkers is needed, giving various degrees of portability. Some
> should be supplied with the library, but users also need to be able
> to provide their own.
>
> OTOH, a function that has to be explicitly called isn't going to be
> effective. Manual checking is too laborious in code that does a lot
> of path manipulation. A one time I took several pages of code and
> tried to add explicit checks. The code turned into a mess. Manual
> checking is also
> likely to be ignored by the very programmers who need it the most;
> those
> who have firm but erroneous ideas about what is or isn't portable.

I am not sure that it should be the responsibility of the path class to
enforce some notion of portability. Wouldn't it be more appropriate to defer
the portability check, if any, to the point where the path is actually used
in a filesystem operation?

___
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost