Re: Path as a dictionary tree key? (was Re: PEP on path module for standard library)

[Ron Adam]

Here's an example of how path objects could possibly be used to store 
and retrieve data from tree_dictionary class.

I used lists here in place of path objects, but I think path objects 
would be better.  I think paths used this way creates a consistant way 
to access data stored in both internal and external tree structurs.

A more realistic example would be to store formated (html) strings in 
the tree and then use imbeded paths to jump from page to page.

The tree class defined here is not complete, it's just the minimum to 
get this example to work.  I wouldn't mind at all if anyone posted 
improvements.  (hint hint) ;-)

Cheers,
Ron Adam


+  output --

Define paths:
path1 = ['hello', 'world']
path2 = ['hello', 'there', 'world']
path3 = ['hello', 'there', 'wide', 'world']

Store path keys in tree:
  hello
world
  None
there
  world
None
  wide
world
  None

Get path list from tree:
['hello', 'world']
['hello', 'there', 'world']
['hello', 'there', 'wide', 'world']

Put items in tree using path:
  hello
world
  1
there
  world
2
  wide
world
  3

Get items from tree using path:
path1: 1
path2: 2
path3: 3


+ source code ---

# Dictionary tree class  (not finished)
class Tree(dict):
 #TODO - remove_item method,
 #   remove_path method,
 #   __init__ method if needed.

 def get_item(self, path):
 d = self
 for key in path[:-1]:
 d=d[key]
 return d[path[-1]]

 def put_item(self, path, item):
 d = self
 for key in path[:-1]:
 d=d[key]
 d[path[-1]]=item

 def store_path(self, path_=None):
 # store item
 key = path_[0]
 if len(path_)==1:
 self[key]=None
 return
 try:
 self[key]
 except KeyError:
 self[key]=Tree()
 self[key].store_path(path_[1:])

 def get_paths(self, key=None, path=[], pathlist=[]):
 if key==None:
 key=self
 keys = key.keys()
 for k in keys:
 if type(self[k]) is Tree:
 path.append(k)
 self[k].get_paths(key[k], path, pathlist)
 else:
 pathlist.append(path+[k])
 return pathlist


def pretty_print_tree(t, tab=0):
 for k in t.keys():
 print '  '*tab, k
 if type(t[k]) is Tree:
 pretty_print_tree(t[k], tab+1)
 else:
 print '  '*(tab+1), t[k]

# Store data in a dictionary.

print 'Define paths:'
path1 = ['hello','world']
print 'path1 =', path1
path2 = ['hello','there','world']
print 'path2 =', path2
path3 = ['hello','there','wide','world']
print 'path3 =', path3

print '\nStore path keys in tree:'
tree = Tree()
tree.store_path(path1)
tree.store_path(path2)
tree.store_path(path3)
pretty_print_tree(tree)

print '\nGet path list from tree:'
path_list = tree.get_paths()
for path in path_list:
 print path

print '\nPut items in tree using path:'
tree.put_item(path1, '1')
tree.put_item(path2, '2')
tree.put_item(path3, '3')
pretty_print_tree(tree)

print '\nGet items from tree using path:'
print 'path1:', tree.get_item(path1)
print 'path2:', tree.get_item(path2)
print 'path3:', tree.get_item(path3)


+ end ---



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Path as a dictionary tree key? (was Re: PEP on path module for standard library)

[Ron Adam]

Brian Beck wrote:
> Ron Adam wrote:
> 
>> This give a more general purpose for path objects.  Working out ways 
>> to retrieve path objects from a dictionary_tree also would be useful I 
>> think.  I think a Tree class would also be a useful addition as well.
>>
>> Any thoughts on this?
> 
> 
> I don't think this would be as useful as you think, for Path objects at 
> least.  Path objects represent *a* path, and building a tree as you have 
> proposed involves storing much more information than necessary.  For 
> instance, if I have the path /A/B/X, a tree-structured object would 
> logically also store the siblings of X and B (subpaths of B and A).


But you miss understand...  I'm not suggesting a path object be a tree, 
but it should/could be used as a key for accessing data stored in a tree 
structure.

The path object it self would be much the same thing as what is already 
being discussed.  Using it as a key to a tree data structure is an 
additional use for it.

Regards,
Ron



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Path as a dictionary tree key? (was Re: PEP on path module for standard library)

[Brian Beck]

Ron Adam wrote:
> This give a more general purpose for path objects.  Working out ways to 
> retrieve path objects from a dictionary_tree also would be useful I 
> think.  I think a Tree class would also be a useful addition as well.
> 
> Any thoughts on this?

I don't think this would be as useful as you think, for Path objects at 
least.  Path objects represent *a* path, and building a tree as you have 
proposed involves storing much more information than necessary.  For 
instance, if I have the path /A/B/X, a tree-structured object would 
logically also store the siblings of X and B (subpaths of B and A).

-- 
Brian Beck
Adventurer of the First Order
-- 
http://mail.python.org/mailman/listinfo/python-list

Path as a dictionary tree key? (was Re: PEP on path module for standard library)

[Ron Adam]

I'm wondering if a class that acts as a interface to a tree data 
structure stored in a dictionary could also be useful as a base class 
for accessing filesystems, urls, and zip (or rar) files.

Then a path object could then be used as a dictionary_tree key.

This idea seems much more useful to me than the specific file path 
object being proposed.  It could be used for all sorts of things and 
extends Python in a more general way.

So given a toy example and a not yet written Tree class which is just a 
dictionary with alternate methods for managing tree data.

D = {'a': {'b': 'data1', 'c': 'data2'}}
D_tree = Tree(D)

As:
'a'
'b'
'data1'
'c'
'data2'

A path object to get 'data1' would be.

path = Path('a','b')

item = D_tree[path]

or

item = D_tree[Path('a','b')]


That would be in place of..

item = D[path[0]][path[1]] -> item = D['a']['b']


This give a more general purpose for path objects.  Working out ways to 
retrieve path objects from a dictionary_tree also would be useful I 
think.  I think a Tree class would also be a useful addition as well.

Any thoughts on this?


Cheers,
Ron

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[qvx]

Ron Adam wrote:
> Bengt Richter wrote:
>
> > http://msdn.microsoft.com/workshop/networking/pluggable/overview/aplugprot_overviews_entry.asp.


Qvx

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Andrew Dalke]

Peter Hansen wrote:
> A scattered assortment of module-level global function names, and 
> builtins such as open(), make it extraordinarily difficult to do 
> effective and efficient automated testing with "mock" objects.

I have been able to do this by inserting my own module-scope function
that intercepts the lookup before it gets to builtins.  A problem
though is that a future (Python 3K?) Python may not allow that.

For example,

module.open = mock_open
try:
  ...
finally:
  module.open = open

By looking at the call stack it is possible to replace the built-in
open to have new behavior only when called from specific modules or
functions, but that gets to be rather hairy.

> Object-oriented solutions like Path make it near trivial to substitute a 
> mock or other specialized object which (duck typing) acts like a Path 
> except perhaps for actually writing the file to disk, or whatever other 
> difference you like.

By analogy to the other builtins, another solution is to have a
protocol by which open() dispatches to an instance defined method.

> So, for the PEP, another justification for Path is that its use can 
> encourage better use of automated testing techniques and thereby improve 
> the quality of Python software, including in the standard library.

But then what does the constructor for the file object take?

I've also heard mention that a future (Py3K era) 'open' may allow
URLs and not just a path string.

Andrew
[EMAIL PROTECTED]

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Ron Adam]

Bengt Richter wrote:


>  >



> 
> Say, how about
> 
> if Pathobject('gui://message_box/yn/continue 
> processing?').open().read().lower()!='y':
> raise SystemExit, "Ok, really not continuing ;-)"
> 
> An appropriate registered subclass for the given platform, returned when the
> Pathobject base class instantiates and looks at the first element on open() 
> and delegates
> would make that possible, and spelled platform-independently as in the code 
> above.

I like it.  ;-)

No reason why a path can't be associated to any tree based object.

> 
> 
> Regards,
> Bengt Richter



I wasn't sure what to comment on, but it does point out some interesting 
possibilities I think.

A path could be associated to any file-like object in an external (or 
internal) tree structure.  I don't see any reason why not.

In the case of an internal file-like object, it could be a series of 
keys in nested dictionaries.  Why not use a path as a dictionary 
interface?

So it sort of raises the question of how tightly a path object should be 
associated to a data structure?  When and where should the path object 
determine what the final path form should be?  And how smart should it 
be as a potential file-like object?

Currently the device name is considered part of the path, but if instead 
you treat the device as an object, it could open up more options.

(Which would extend the pattern of your example above. I think.)

(also a sketch..  so something like...)

# Initiate a device path object.
apath = device('C:').path(initial_path)

# Use it to get and put data
afile = apath.open(mode,position,units)  # defaults ('r','line',next)
aline = afile.read().next()# read next unit, or as an iterator.
afile.write(line)
afile.close()

# Manually manipulate the path
apath.append('something') # add to end of path
apath.remove()# remove end of path
alist = apath.split()  # convert to a list
apath.join(alist) # convert list to a path
astring = str(apath()) # get string from path
apath('astring')  # set path to string
apath.validate()   # make sure it's valid

# Iterate and navigate the path
apath.next()# iterate path objects
apath.next(search_string)# iterate with search string
apath.previous()# go back
apath.enter()   # enter directory
apath.exit()# exit directory

# Close it when done.
apath.close()

etc...


With this you can iterate a file system as well as it's files.  ;-) 

 (Add more or less methods as needed of course.)


apath = device(dev_obj).path(some_path_sting)
apath.open().write(data).close()


or if you like...

device(dev_obj).append(path_sting).open().write(data).close()



Just a few thoughts,

Cheers,
Ron






-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Mike Orr]

Sorry for the formatting of my first message; I haven't posted via
Google Groups before.

Reinhold Birkenfeld wrote:
> > - Some way to pretty-print paths embedded in lists/dicts as strings.  I
> > have a
> >   .repr_as_str class attribute that toggles this.
>
> I think str() does its job nicely there.

But when you str() a list or other collection, it repr()'s the
elements.  This makes programs difficult to debug because you can't
just insert a "print my_list"; you then have to weed through all the
"path(" prefixes in the output.So you have to write a custom for
loop, which you may only use for five minutes, to convert the paths or
print them separately.  With a list you can use a list comprehension,
but not with a more complex structure.

I don't know if there's a good solution to this other than a path_debug
function that takes an object and converts all the embedded paths to
strings,
because people would object to repr() returning a plain string, but I
want to at least acknowledge it's a problem.

> > Less important but non-controversial:
> >
> > - Path.tempfile(), Path.tempdir():
> >   Create a temporary file using tempfile.mkstemp, tempfile.mkdtemp
> >
> > - Path.tempfileobject():
> >   Create a temporary file object using tempfile.TemporaryFile.
> >   (Static method.)
>
> Don't know whether these belong here. They are all not acting upon the
> path, but create the file/dir at another location.

More controversial than I thought.  OK, they can stay in a subclass.

My reasoning is that, if you're using paths, you'll need a Path for the
file/dir anyway, so why not do it in one step?  It also organizes the
operations, the way Path organizes os.path and shutil.  tempfileobject
is less closely related to path, but it would allow the deprecation of
the tempfile module, rather than just two functions in it.

> > Controversial:
> >
> > - Delete methods and mkdir should succeed silently if the operation is
> >   already done.  Otherwise you always have to put them in an if:
> >   'if foo.exists(): foo.remove()'.
>
> That's a good thing. The delete methods shouldn't act any different from
> the corresponding os functions.

Jason said the same thing.  I disagree.  But it shows why a "literal"
superclass in the stdlib and higher-level subclasses would be useful.

I don't see why a little change to avoid an unnecessary IOError is a
bad thing.  The only programs it would hurt are those that really want
to know whether the thing already existed, and those programs can
explicitly test for it *and already do*.  But most programs just want
to make sure the way is clear for their files, and would like a
one-line call to do it.

-- Mike Orr <[EMAIL PROTECTED]>

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Reinhold Birkenfeld]

Peter Hansen wrote:
> Reinhold Birkenfeld wrote:
>> Mike Orr wrote:
>>>- I saw in a thread that .name and .parent were removed.  I use them
>>>very frequently, along with .ext and .namebase.  I don't want to call
>>>methods for these.
>> 
>> They are now named .basename and .directory.
> 
> Reinhold, is ".parent" now ".directory"?  Does that still make sense 
> when the Path contains a directory and not a file name?  It seems to me 
> more general to have .parent in there where it makes sense for both cases.

Hm. It was suggested on python-dev and among my first modifications. I didn't
see objections till now, but as you're saying, parent is probably better.

Reinhold
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Peter Hansen]

Reinhold Birkenfeld wrote:
> Mike Orr wrote:
>>- Who needs .open()?  open(myPath) is fine.  But it can stay for
>>backward compatibility.
> 
> Right. I think it's nice for people who like a more OO approach.

There's one very good reason to use .open() on a Path object, and in 
fact it's a (for me) strong justification for the existence of Path in 
the standard library (and, even more, for porting everything else in the 
standard library to use Path).

A scattered assortment of module-level global function names, and 
builtins such as open(), make it extraordinarily difficult to do 
effective and efficient automated testing with "mock" objects.  Although 
one can fairly easily shadow the definition of "open" in any given 
module under test, if the code under test instead calls another standard 
library routine which internally uses open(), it becomes necessary to 
replace the builtin definition of open() and affect all code throughout 
the program.

Object-oriented solutions like Path make it near trivial to substitute a 
mock or other specialized object which (duck typing) acts like a Path 
except perhaps for actually writing the file to disk, or whatever other 
difference you like.

I haven't gotten around to doing a mock Path object yet, though given 
the work that's gone into my existing mock FileSystem object which lets 
you replace open() and test code in a fully controlled and safe 
environment, it shouldn't be hard to do.

So, for the PEP, another justification for Path is that its use can 
encourage better use of automated testing techniques and thereby improve 
the quality of Python software, including in the standard library.

-Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Peter Hansen]

Reinhold Birkenfeld wrote:
> Mike Orr wrote:
>>- I saw in a thread that .name and .parent were removed.  I use them
>>very frequently, along with .ext and .namebase.  I don't want to call
>>methods for these.
> 
> They are now named .basename and .directory.

Reinhold, is ".parent" now ".directory"?  Does that still make sense 
when the Path contains a directory and not a file name?  It seems to me 
more general to have .parent in there where it makes sense for both cases.

-Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Steven D'Aprano]

On Sat, 30 Jul 2005 14:43:27 +0200, Reinhold Birkenfeld wrote:

>> Less important but non-controversial:
>>
>> - Path.tempfile(), Path.tempdir():
>>   Create a temporary file using tempfile.mkstemp, tempfile.mkdtemp
>>
>> - Path.tempfileobject():
>>   Create a temporary file object using tempfile.TemporaryFile.
>>   (Static method.)
> 
> Don't know whether these belong here. They are all not acting upon the
> path, but create the file/dir at another location.

Agreed that they do not belong.

>> - .delete_dammit() recursively deletes it, whatever it is, without you
>>   having to do the if file/if directory dance.  I suppose it would need
>>   a politically correct name.
>>   which you really have to do every time you delete.
> 
> *grin* Any naming suggestions?

force_delete()

-- 
Steven.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Reinhold Birkenfeld]

Mike Orr wrote:

> The main changes I'd like to see in Path (some of these have been made
> in Python CVS at nondist/sandbox/path/path.py) are:

Thanks for the comments! They are greatly appreciated.

> - When methods create path objects, use self.__class__() instead of
> Path().
>   This makes it possible to subclass Path.  Otherwise you
>   have to rewrite the whole thing due to the hardcoded name.
>
> - Path.cwd() should be a class method instead of a static method, for
> the same
>   reason.

Changed by now.

> - The constructor behavior in CVS is good.  Path(''), Path.cwd(),
>   Path() => Path.cwd().

Note that Path() results in os.curdir, while Path.cwd() results in the
absolute current directory.

> - Need .chdir() method, otherwise you have to import os for it.

Added by now.

> - Some way to pretty-print paths embedded in lists/dicts as strings.  I
> have a
>   .repr_as_str class attribute that toggles this.

I think str() does its job nicely there.

> - .ancestor(3) is the same as .parent.parent.parent, but more concise.

Good proposal, will consider this.

> - I saw in a thread that .name and .parent were removed.  I use them
> very
>   frequently, along with .ext and .namebase.  I don't want to call
> methods for
>   these.

They are now named .basename and .directory.

> - '/' vs .joinpath(), no big deal.  I've been using '/' because it's
> there.
>   .joinpath must take multiple *args, to prevent .joinpath().joinpath()
> .

That's already in.

> - .joinancestor(N, *components) is a combination of .ancestor and
> .joinpath.
>   If curdir is /home/joe/Webware/www, Path.cwd().joinancestor(1, 'lib')
> is
>   /home/joe/Webware/lib.

I think Path.cwd().parent.joinwith('lib') is readable enough.

> - Keep .lisdir() as in CVS; it acts like os.listdir().  This is useful
> even
>   with paths, when you're just going to use the string basename anyway.

Right. In addition, it's confusing if os.listdir is behaving differently
from Path.listdir().

> - Who needs .open()?  open(myPath) is fine.  But it can stay for
> backward
>   compatibility.

Right. I think it's nice for people who like a more OO approach.

> Less important but non-controversial:
>
> - Path.tempfile(), Path.tempdir():
>   Create a temporary file using tempfile.mkstemp, tempfile.mkdtemp
>
> - Path.tempfileobject():
>   Create a temporary file object using tempfile.TemporaryFile.
>   (Static method.)

Don't know whether these belong here. They are all not acting upon the
path, but create the file/dir at another location.

> Controversial:
>
> - Delete methods and mkdir should succeed silently if the operation is
>   already done.  Otherwise you always have to put them in an if:
>   'if foo.exists(): foo.remove()'.

That's a good thing. The delete methods shouldn't act any different from
the corresponding os functions.

> - .delete_dammit() recursively deletes it, whatever it is, without you
>   having to do the if file/if directory dance.  I suppose it would need
> a
>   politically correct name.
>   which you really have to do every time you delete.

*grin* Any naming suggestions?

> - All .dirs*, .files*, .walk* methods have a symlinks flag, default
> True.
>   If false, don't yield symlinks.  Add .symlinks and .walksymlinks
> methods.
>   This eliminates an 'if' for programs that want to treat symlinks
> specially.

Sounds good. Will see.

Reinhold
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Mike Orr]

Michael Hoffman wrote:
> I use path in more of my modules and scripts than any other third-party
> module, and I know it will be very helpful when I no longer have to
> worry about deploying it.

Same here.  An object-oriented path module is extremely useful, and
makes one's code much less cluttered.  I've written an enhpath module
that I used in my projects, with several convenience methods and some
magic behavior.  It's too
ambitious for the standard library, but I'd like if people could at
least look at the docstring and see whether some features might be
useful in Path.  I'd also like to see Path made subclass-friendly so I
could implement this as a subclass, and others could make other
subclasses.  The docstring itself could also be ported to Path.  The
source and test suite (py.test) are here:
http://cafepy.com/quixote_extras/rex/path/enhpath.py
http://cafepy.com/quixote_extras/rex/path/enhpath_test.py
append "?download=1" for download-friendly format.
I sent an earlier version to Jason Orendorff and he liked some of the
changes and had some ideas of his own, but said he was too busy to do
much implementation work, and then my further letters never got a
reply.

The main changes I'd like to see in Path (some of these have been made
in Python CVS at nondist/sandbox/path/path.py) are:

- When methods create path objects, use self.__class__() instead of
Path().
  This makes it possible to subclass Path.  Otherwise you
  have to rewrite the whole thing due to the hardcoded name.

- Path.cwd() should be a class method instead of a static method, for
the same
  reason.

- The constructor behavior in CVS is good.  Path(''), Path.cwd(),
  Path() => Path.cwd().

- Need .chdir() method, otherwise you have to import os for it.

- Some way to pretty-print paths embedded in lists/dicts as strings.  I
have a
  .repr_as_str class attribute that toggles this.

- .ancestor(3) is the same as .parent.parent.parent, but more concise.

- I saw in a thread that .name and .parent were removed.  I use them
very
  frequently, along with .ext and .namebase.  I don't want to call
methods for
  these.

- '/' vs .joinpath(), no big deal.  I've been using '/' because it's
there.
  .joinpath must take multiple *args, to prevent .joinpath().joinpath()
.

- .joinancestor(N, *components) is a combination of .ancestor and
.joinpath.
  If curdir is /home/joe/Webware/www, Path.cwd().joinancestor(1, 'lib')
is
  /home/joe/Webware/lib.

- Keep .lisdir() as in CVS; it acts like os.listdir().  This is useful
even
  with paths, when you're just going to use the string basename anyway.

- Who needs .open()?  open(myPath) is fine.  But it can stay for
backward
  compatibility.



Less important but non-controversial:

- Path.tempfile(), Path.tempdir():
  Create a temporary file using tempfile.mkstemp, tempfile.mkdtemp

- Path.tempfileobject():
  Create a temporary file object using tempfile.TemporaryFile.
  (Static method.)

Controversial:

- Delete methods and mkdir should succeed silently if the operation is
  already done.  Otherwise you always have to put them in an if:
  'if foo.exists(): foo.remove()'.

- .delete_dammit() recursively deletes it, whatever it is, without you
  having to do the if file/if directory dance.  I suppose it would need
a
  politically correct name.
  which you really have to do every time you delete.

- All .dirs*, .files*, .walk* methods have a symlinks flag, default
True.
  If false, don't yield symlinks.  Add .symlinks and .walksymlinks
methods.
  This eliminates an 'if' for programs that want to treat symlinks
specially.

- I have a .move method that combines .rename, .renames, and .move; and
a .copy
  method that combines .copy, .copy2, and .copytree .

I'd appreciate a Cc on replies if your newsreader allows it.

-- Mike Orr <[EMAIL PROTECTED]>

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Daniel Dittmar]

Terry Reedy wrote:
> for dir in pathobject:
>   if isdir(dir): cd(dir)
> 
> *is*, in essence, what the OS mainly does with paths (after splitting the 
> string representation into pieces).

That's why there is rarely a need to to it in Python code.

> Directory walks also work with paths as sequences (stacks, in particular).

I'd say it works with stacks of pathes, not with stacks of path elements.

I'm not saying that there isn't any use for having a list of path 
elements. But it isn't that common, so it should get an methodname to 
make it more explicit.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Daniel Dittmar]

John Roth wrote:
> However, a path as a sequence of characters has even less
> meaning - I can't think of a use, while I have an application

That's true. But the arguments for path objects as strings go more in 
the direction of using existing  functions that expect strings.

> where traversing a path as a sequence of path elements makes
> perfect sense: I need to descend the directory structure, directory
> by directory, looking for specific files and types.

But then your loop doesn't need the individual path elements, but rather 
  sub-path objects

for p in pathobj.stepdown ('/usr/local/bin'):
 if p.join (searchedFile):
 whatever

I'm not saying that there isn't any use for having a list of path 
elements. But it isn't that common, so it should get an methodname to 
make it more explicit.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Bengt Richter]

On Sat, 23 Jul 2005 07:05:05 +1000, John Machin <[EMAIL PROTECTED]> wrote:

>Daniel Dittmar wrote:
>> Duncan Booth wrote:
>> 
>>>  I would have expected a path object to be a sequence of path elements 
>>> rather than a sequence of characters. 
>> 
>> 
>> Maybe it's nitpicking, but I don't think that a path object should be a 
>> 'sequence of path elements' in an iterator context.
>> 
>> This means that
>> 
>> for element in pathobject:
>> 
>> has no intuitive meaning for me, so it shouldn't be allowed.
>
>Try this:
>
>A file-system is a maze of twisty little passages, all alike. Junction 
>== directory. Cul-de-sac == file. Fortunately it is signposted. You are 
>dropped off at one of the entrance points ("current directory", say). 
>You are given a route (a "path") to your destination. The route consists 
>of a list of intermediate destinations.
>
>for element in pathobject:
>follow_sign_post_to(element)
>
>Exception-handling strategy: Don't forget to pack a big ball of string. 
>Anecdotal evidence is that breadcrumbs are unreliable.
>


ISTM a path is essentially a representation of a script whose interpretation
by an orderly choice of interpreters finally leads to accessing to some entity,
typically a serial data representation, through an object, perhaps a local 
proxy,
that has standard methods for accessing the utimate object's desired info.

IOW, a path sequence is like a script text that has been .splitline()'d and
and the whole sequence fed to a local interpreter, which might chew through 
multiple
lines on its own, or might invoke interpreters on another network to deal with 
the
rest of the script, or might use local interpreters for various different kinds 
of
access (e.g., after seeing 'c:' vs 'http://' vs '/c' vs '//c' etc. on the 
platform
defining the interpretation of the head element).

Turning a single path string into a complete sequence of elements is not 
generally possible
unless you have local knowledge of the syntax of the entire tail beyond the the 
prefix
you have to deal with. Therefore, a local platform-dependent Pathobject class 
should, I think,
only recognize prefixes that it knows how to process or delegate processing 
for, leaving
the interpretation of the tail to the next Pathobject instance, however 
selected and/or
located.

So say (this is just a sketch, mind ;-)

po = Pathobject()

results in a po that splits out (perhaps by regex) a prefix, a first 
separator/delimiter,
and the remaining tail. E.g., in class Pathobject,
def __init__(self, pathstring=None)
if pathstring is None: #do useful default??
self.pathstring = pathstring
self.prefix, self.sep, self.tail = self.splitter(pathstring)
if self.prefix in self.registered_prefixes:
self.child = self.registered_prefixes[self.prefix](self.tail)
else:
self.child = []
self.opened_obj = None

Then the loop inside a local pathobject's open method po.open()
might go something like

def open(self, *mode, **kw):
if self.child:
self.opened_obj = self.child.open(self.tail, *mode, **kw)
else:
self.opened_obj = file(self.pathstring, *mode)
return self

And closing would just go to the immediately apparent opened object, and
if that had complex closing to do, it would be its responsibility to deal
with itself and its child-derived objects.

def close(self):
self.opened_object.close()


The point is that a given pathobject could produce a new or modified pathobject 
child
which might be parsing urls instead of windows file system path strings or could
yield an access object producing something entirely synthetic.

A synthetic capability could easily be introduced if the local element 
pathobject
instance looked for e.g., 'synthetic://' as a possible first element (prefix) 
string representation,
and then passed the tail to a subclass defining synthetic:// path 
interpretation.
E.g., 'synthetic://temp_free_diskspace' could be a platform-independent way to 
get such info as that.

Opening 'testdata:// ...' might be an interesting way to feed test suites, if 
pathobject subclasses
could be registered locally and found via the head element's string 
representation.'

One point from this is that a path string represents an ordered sequence of 
elements, but is heterogenous,
and therefore has potentially heterogenous syntax reflected in string tails 
with syntax that should be
interpreted differently from the prefix syntax.

Each successive element of a path string effectively requires an interpreter 
for that stage of access
pursuit, and the chain of processing may result in different path 
entities/objects/representations
on different systems, with different interpretations going on, sharing only 
that they are part of the
process of getting access to something and providing access services, if it's 
not a one-shot access.

This would also be a potential way to create access to a fore

Re: PEP on path module for standard library

[Duncan Booth]

Peter Hansen wrote:
> Just a note, though you probably know, that this is intended to be 
> written this way with path:
> 
> >>> p / q
> path(u'a/b/c/d')

I know, but it really doesn't look right to me.

I think that my fundamental problem with all of this is that by making path 
a subclass of str/unicode it inherits inappropriate definitions of some 
common operations, most obviously addition, iteration and subscripting. 

These operations have obvious meaning for paths which is not the same as 
the meaning for string. Therefore (in my opinion) the two ought to be 
distinct.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Peter Hansen]

George Sakkis wrote:
> "Andrew Dalke" <[EMAIL PROTECTED]> wrote:
>>I think that the string representation of a path is so important that
>>it *is* the path.
> 
> There are (at least) two frequently used path string representations, 
 > the absolute and the relative to the working directory. Which one
 > *is* the path ? Depending on the application, one of them woud
> be more natural choice than the other.

Sorry, George, but that's now how it works.

Whether using the regular string-based Python paths or the new path 
module, a path *is* either absolute or relative, but cannot be both at 
the same time.

This is therefore not an issue of "representation" but one of state.

-Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[paul]

Michael Hoffman wrote:
> Reinhold Birkenfeld wrote:
> > Probably as Terry said: a path is both a list and a string.

[...]

> One way to divide this is solely based on path separators:
>
> ['c:', 'windows', 'system32:altstream', 'test.dir',
> 'myfile.txt.zip:altstream']

I would argue that any proposed solution has to work with VMS
pathnames. ;-)

> The current stdlib solution, os.path.splitext(os.path.splitext(filename)
> [0])[0] is extremely clunky, and I have long desired something better.
> (OK, using filename.split(os.extsep) works a little better, but you get
> the idea.)

And also with unusual (eg. RISC OS) filename extensions.

To do any justice to the existing solutions, any PEP should review at
least the following projects:

 * The path module (of course):
   http://www.jorendorff.com/articles/python/path/

 * The py.path module (or at least the ideas for it):
   http://codespeak.net/py/current/doc/future.html

 * itools.uri
   http://www.ikaaro.org/itools

 * Results from the "Object-Oriented File System Virtualisation"
   project in the "Summer of Code" programme:
   http://wiki.python.org/moin/SummerOfCode

And I hope that the latter project is reviewing some of the other work,
if only to avoid the "framework proliferation" that people keep
complaining about.

Paul

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Andrew Dalke]

George Sakkis wrote:
> That's why phone numbers would be a subset of integers, i.e. not every
> integer would correspond to a valid number, but with the exception of
> numbers starting with zeros, all valid numbers would be an integers.

But it's that exception which violates the LSP.

With numbers, if x==y then (x,y) = (y,x) makes no difference.
If phone numbers are integers then 001... == 01... but swapping
those two numbers makes a difference.  Hence they cannot be modeled
as integers.

> Regardless, this was not my point; the point was that adding
> two phone numbers or subtracting them never makes sense semantically.

I agree. But modeling them as integers doesn't make sense either.
Your example of adding phone numbers depends on them being represented
as integers.  Since that representation doesn't work, it makes sense
that addition of phone number is suspect.

> There are (at least) two frequently used path string representations,
> the absolute and the relative to the working directory. Which one *is*
> the path ? Depending on the application, one of them woud be more
> natural choice than the other.

Both.  I don't know why one is more natural than the other.

>> I trust my intuition on this, I just don't know how to justify it, or
>> correct it if I'm wrong.
> 
> My intuition also happens to support subclassing string, but for
> practical reasons rather than conceptual.

As you may have read elsewhere in this thread, I give some examples
of why subclassing from string fits best with existing code.

Even if there was no code base, I think deriving from string is the
right approach.  I have a hard time figuring out why though.  I think
if the lowest level Python/C interface used a "get the filename"
interface then perhaps it wouldn't make a difference.  Which means
I'm also more guided by practical reasons than conceptual.

Andrew
[EMAIL PROTECTED]

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[George Sakkis]

"Andrew Dalke" <[EMAIL PROTECTED]> wrote:

> [snipped]
>
> > Take for example the case where a PhoneNumber class is subclass
> > of int. According to LSP, it is perfectly ok to add phone numbers
> > together, subtract them, etc, but the result, even if it's a valid
> > phone number, just doesn't make sense.
>
> Mmm, I don't think an integer is a good model of a phone number.
> For example, in the US
>   00148762040828
> will ring a mobile number in Sweden while
>   148762040828
> will give a "this isn't a valid phone number" message.

That's why phone numbers would be a subset of integers, i.e. not every integer 
would correspond to a
valid number, but with the exception of numbers starting with zeros, all valid 
numbers would be an
integers. Regardless, this was not my point; the point was that adding two 
phone numbers or
subtracting them never makes sense semantically.

> [snipped]
>
> > I just noted that conceptually a path is a composite object consisting of
> > many properties (dirname, extension, etc.) and its string representation
> > is just one of them. Still, I'm not suggesting that a 'pure' solution is
> > better that a more practical that covers most usual cases.
>
> For some reason I think that
>
>   path.dirname()
>
> is better than
>
>   path.dirname
>
> Python has properties now so the implementation of the latter is
> trivial - put a @property on the line before the "def dirname(self):".

Sorry, I used the term 'property' in the broad sense, as the whole exposed API, 
not the specific
python feature; I've no strong preference between path.dirname and 
path.dirname().

> I think that the string representation of a path is so important that
> it *is* the path.

There are (at least) two frequently used path string representations, the 
absolute and the relative
to the working directory. Which one *is* the path ? Depending on the 
application, one of them woud
be more natural choice than the other.

> I trust my intuition on this, I just don't know how to justify it, or
> correct it if I'm wrong.

My intuition also happens to support subclassing string, but for practical 
reasons rather than
conceptual.

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Andrew Dalke]

George Sakkis wrote:
> Bringing up how C models files (or anything else other than primitive types
> for that matter) is not a particularly strong argument in a discussion on
> OO design ;-)

While I have worked with C libraries which had a well-developed
OO-like interface, I take your point.

Still, I think that the C model of a file system should be a
good fit since after all C and Unix were developed hand-in-hand.  If
there wasn't a good match then some of the C path APIs should be
confusing or complicated.  Since I don't see that it suggests that
the "path is-a string" is at least reasonable.

> Liskov substitution principle imposes a rather weak constraint

Agreed.  I used that as an example of the direction I wanted to
go.  What principles guide your intuition of what is a "is-a"
vs a "has-a"?

> Take for example the case where a PhoneNumber class is subclass
> of int. According to LSP, it is perfectly ok to add phone numbers
> together, subtract them, etc, but the result, even if it's a valid
> phone number, just doesn't make sense.

Mmm, I don't think an integer is a good model of a phone number.
For example, in the US
  00148762040828
will ring a mobile number in Sweden while
  148762040828
will give a "this isn't a valid phone number" message.

Yet both have the same base-10 representation.  (I'm not using
a syntax where leading '0' indicates an octal number. :)

> I wouldn't say more complicated, but perhaps less intuitive in a few cases, 
> e.g.:
> 
>> path(r'C:\Documents and Settings\Guest\Local Settings').split()
> ['C:\\Documents', 'and', 'Settings\\Guest\\Local', 'Settings']
> instead of
> ['C:', 'Documents and Settings', 'Guest', 'Local Settings']

That is why the path module using a different method to split
on pathsep vs. whitespace.  I get what you are saying, I just think
it's roughly equivalent to appealing to LSP in terms of weight.

Mmm, then there's a question of the usefulness of ".lower()" and
".expandtabs()" and similar methods.  Hmmm

> I just noted that conceptually a path is a composite object consisting of
> many properties (dirname, extension, etc.) and its string representation
> is just one of them. Still, I'm not suggesting that a 'pure' solution is
> better that a more practical that covers most usual cases.

For some reason I think that

  path.dirname()

is better than

  path.dirname

Python has properties now so the implementation of the latter is
trivial - put a @property on the line before the "def dirname(self):".

I think that the string representation of a path is so important that
it *is* the path.  The other things you call properties aren't quite
properties in my model of a path and are more like computable values.

I trust my intuition on this, I just don't know how to justify it, or
correct it if I'm wrong.

Andrew
[EMAIL PROTECTED]

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Neil Hodgson]

Scott David Daniels:

> Isn't it even worse than this?
> On Win2K & XP, don't the file systems have something to do with the
> encoding?  So D: (a FAT drive) might naturally be str, while C:
> (an NTFS drive) might naturally be unicode.  

This is generally safe as Windows is using unicode internally and 
provides full-fidelity access to the FAT drive using unicode strings. 
You can produce failures if you try to create files with names that can 
not be represented but you would see a similar failure with byte string 
access.

 > Even worse, would be a
> path that switches in the middle (which it might do if we get to a
> ZIP file or use the newer dir-in-file file systems.

If you are promoting from byte strings with a known encoding to 
unicode path objects then this should always work.

Neil
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Michael Hoffman]

Peter Hansen wrote:

> Practically everything that path does, with a few useful exceptions, is 
> a thin wrapper around the existing calls.

If the implementation is easy to explain, it may be a good idea.

OT: I just realized you can now type in "python -m this" at the command 
line, which is convenient, but strange.
-- 
Michael Hoffman
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[George Sakkis]

"Andrew Dalke" <[EMAIL PROTECTED]> wrote:

> George Sakkis wrote:
> > You're right, conceptually a path
> > HAS_A string description, not IS_A string, so from a pure OO point of
> > view, it should not inherit string.
>
> How did you decide it's "has-a" vs. "is-a"?
>
> All C calls use a "char *" for filenames and paths,
> meaning the C model file for the filesystem says
> paths are strings.

Bringing up how C models files (or anything else other than primitive types for 
that matter) is not
a particularly strong argument in a discussion on OO design ;-)

> Paths as strings fit the Liskov substitution principle
> in that any path object can be used any time a
> string is used (eg, "loading from " + filename)

Liskov substitution principle imposes a rather weak constraint on when 
inheritance should not be
used, i.e. it is a necessary condition, but not sufficient. Take for example 
the case where a
PhoneNumber class is subclass of int. According to LSP, it is perfectly ok to 
add phone numbers
together, subtract them, etc, but the result, even if it's a valid phone 
number, just doesn't make
sense.

> Good information hiding suggests that a better API
> is one that requires less knowledge.  I haven't
> seen an example of how deriving from (unicode)
> string makes things more complicated than not doing so.

I wouldn't say more complicated, but perhaps less intuitive in a few cases, 
e.g.:

> path(r'C:\Documents and Settings\Guest\Local Settings').split()
['C:\\Documents', 'and', 'Settings\\Guest\\Local', 'Settings']
instead of
['C:', 'Documents and Settings', 'Guest', 'Local Settings']

I just noted that conceptually a path is a composite object consisting of many 
properties (dirname,
extension, etc.) and its string representation is just one of them. Still, I'm 
not suggesting that a
'pure' solution is better that a more practical that covers most usual cases.

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Peter Hansen]

Michael Hoffman wrote:
> Peter Hansen wrote:
> 
>> When files are opened through a "path" object -- e.g. 
>> path('name').open() -- then file.name returns the path object that was 
>> used to open it.
> 
> Also works if you use file(path('name')) or open(path('name')).

Since that's exactly what the path module does, it's not surprising. 
Practically everything that path does, with a few useful exceptions, is 
a thin wrapper around the existing calls.  path.open, for example is 
merely this:

   def open(self, mode='r'):
   return file(self, mode)

-Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[John Machin]

Michael Hoffman wrote:
> John Roth wrote:
> 
>> However, a path as a sequence of characters has even less
>> meaning - I can't think of a use, while I have an application
>> where traversing a path as a sequence of path elements makes
>> perfect sense: I need to descend the directory structure, directory
>> by directory, looking for specific files and types.
> 
> 
> I *have* used a path as a sequence of characters before. I had to deal 
> with a bunch of filenames that were formatted like "file_02832.a.txt"

Ya, ya , ya, only two days ago I had to lash up a quick script to find 
all files matching r"\d{8,8}[A-Za-z]{0,3}$" and check that the expected 
number were present in each sub-directory (don't ask!).

BUT you are NOT using "a path as a sequence of characters". Your 
filename is a path consisting of one element. The *element* is an 
instance of basestring, to which you can apply all the string methods 
and the re module.


> 
> I can see the case for a path as a sequence of elements, although in 
> practice, drive letters, extensions, and alternate streams complicate 
> things.
> 



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[John Machin]

Daniel Dittmar wrote:
> Duncan Booth wrote:
> 
>>  I would have expected a path object to be a sequence of path elements 
>> rather than a sequence of characters. 
> 
> 
> Maybe it's nitpicking, but I don't think that a path object should be a 
> 'sequence of path elements' in an iterator context.
> 
> This means that
> 
> for element in pathobject:
> 
> has no intuitive meaning for me, so it shouldn't be allowed.

Try this:

A file-system is a maze of twisty little passages, all alike. Junction 
== directory. Cul-de-sac == file. Fortunately it is signposted. You are 
dropped off at one of the entrance points ("current directory", say). 
You are given a route (a "path") to your destination. The route consists 
of a list of intermediate destinations.

for element in pathobject:
follow_sign_post_to(element)

Exception-handling strategy: Don't forget to pack a big ball of string. 
Anecdotal evidence is that breadcrumbs are unreliable.

Cheers,
John
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Michael Hoffman]

Peter Hansen wrote:

>> Most prominent change is that it doesn't inherit from str/unicode 
>> anymore.
>> I found this distinction important, because as a str subclass the Path 
>> object
>> has many methods that don't make sense for it.
> 
> On this topic, has anyone ask the original author (Jason Orendorff) 
> whether he has some background on this decision that might benefit the 
> discussion?

My impression is that he doesn't have a lot of spare cycles for this. He 
didn't have anything to add to the python-dev discussion when I informed 
him of it. I'd love to hear what he had to say about the design.
-- 
Michael Hoffman
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Michael Hoffman]

Reinhold Birkenfeld wrote:
> Andrew Dalke wrote:
> 
>>I disagree.  I've tried using a class which wasn't derived from
>>a basestring and kept running into places where it didn't work well.
>>For example, "open" and "mkdir" take strings as input.  There is no
>>automatic coercion.
> 
> Well, as a Path object provides both open() and mkdir() functions, these use
> cases are covered. And that's the point of the Path class: Every common use
> you may have for a path is implemented as a method.

Except when you pass the path to a function written by someone else

> So, it's maybe a good thing that for uncommon uses you have to explicitly
> "cast" the path to a string.

Where uncommon uses include passing the path object to any code you 
don't control? The stdlib can be fixed, this other stuff can't.

>>The solutions to this are:
>>  1) make the path object be derived from str or unicode.  Doing
>>this does not conflict with any OO design practice (eg, Liskov
>>substitution).
>>
>>  2) develop a new "I represent a filename" protocol, probably done
>>via adapt().
>>
>>I've considered the second of these but I think it's a more
>>complicated solution and it won't fit well with existing APIs
>>which do things like
>>
>>
>>  if isinstance(input, basestring):
>>input = open(input, "rU")
>>  for line in input:
>>print line
>>
>>I showed several places in the stdlib and in 3rd party packages
>>where this is used.
> 
> That's a valid point. However, if Path is not introduced as a string,
> most new users will not try to use a Path instance where a string is
> needed, just as you wouldn't try to pass a list where a string is wanted.

But many functions were written expecting lists as arguments but also 
work for strings, and do not require an explicit list(mystring) before 
calling the function.
-- 
Michael Hoffman
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Reinhold Birkenfeld]

Andrew Dalke wrote:
> Duncan Booth wrote:
>> Personally I think the concept of a specific path type is a good one, but 
>> subclassing string just cries out to me as the wrong thing to do.
> 
> I disagree.  I've tried using a class which wasn't derived from
> a basestring and kept running into places where it didn't work well.
> For example, "open" and "mkdir" take strings as input.  There is no
> automatic coercion.

Well, as a Path object provides both open() and mkdir() functions, these use
cases are covered. And that's the point of the Path class: Every common use
you may have for a path is implemented as a method.

So, it's maybe a good thing that for uncommon uses you have to explicitly
"cast" the path to a string.

 class Spam:
> ...   def __getattr__(self, name):
> ... print "Want", repr(name)
> ... raise AttributeError, name
> ... 
 open(Spam())
> Traceback (most recent call last):
>   File "", line 1, in ?
> TypeError: coercing to Unicode: need string or buffer, instance found
 import os
 os.mkdir(Spam())
> Traceback (most recent call last):
>   File "", line 1, in ?
> TypeError: coercing to Unicode: need string or buffer, instance found
 
> 
> The solutions to this are:
>   1) make the path object be derived from str or unicode.  Doing
> this does not conflict with any OO design practice (eg, Liskov
> substitution).
> 
>   2) develop a new "I represent a filename" protocol, probably done
> via adapt().
> 
> I've considered the second of these but I think it's a more
> complicated solution and it won't fit well with existing APIs
> which do things like
> 
> 
>   if isinstance(input, basestring):
> input = open(input, "rU")
>   for line in input:
> print line
> 
> I showed several places in the stdlib and in 3rd party packages
> where this is used.

That's a valid point. However, if Path is not introduced as a string,
most new users will not try to use a Path instance where a string is
needed, just as you wouldn't try to pass a list where a string is wanted.

>> You should need an explicit call to convert a path to a string and that 
>> forces you when passing the path to something that requires a string to 
>> think whether you wanted the string relative, absolute, UNC, uri etc.
> 
> You are broadening the definition of a file path to include URIs?
> That's making life more complicated.  Eg, the rules for joining
> file paths may be different than the rules for joining URIs.
> Consider if I have a file named "mail:[EMAIL PROTECTED]" and I
> join that with "file://home/dalke/badfiles/".
> 
> Additionally, the actions done on URIs are different than on file
> paths.  What should os.listdir("http://www.python.org/";) do?

I agree. Path is only for local filesystem paths (well, in UNIX they could
as well be remote, but that's thanks to the abstraction the filesystem
provides, not Python).

> As I mentioned, I tried some classes which emulated file
> paths.  One was something like
> 
> class TempDir:
>   """removes the directory when the refcount goes to 0"""
>   def __init__(self):
> self.filename = ... use a function from the tempfile module
>   def __del__(self):
> if os.path.exists(self.filename):
>   shutil.rmtree(self.filename)
>   def __str__(self):
> return self.filename
> 
> I could do
> 
>   dirname = TempDir()
> 
> but then instead of
> 
>   os.mkdir(dirname)
>   tmpfile = os.path.join(dirname, "blah.txt")
> 
> I needed to write it as
> 
>   os.mkdir(str(dirname))
>   tmpfile = os.path.join(str(dirname), "blah.txt"))
> 
> or have two variables, one which could delete the
> directory and the other for the name.  I didn't think
> that was good design.

I can't follow. That's clearly not a Path but a custom object of yours.
However, I would have done it differently: provide a "name" property
for the object, and don't call the variable "dirname", which is confusing.

> If I had derived from str/unicode then things would
> have been cleaner.
> 
> Please note, btw, that some filesystems are unicode
> based and others are not.  As I recall, one nice thing
> about the path module is that it chooses the appropriate
> base class at import time.  My "str()" example above
> does not and would fail on a Unicode filesystem aware
> Python build.

There's no difference. The only points where the type of a Path
object' underlying string is decided are Path.cwd() and the
Path constructor.


Reinhold
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Michael Hoffman]

Peter Hansen wrote:

> When files are opened through a "path" object -- e.g. 
> path('name').open() -- then file.name returns the path object that was 
> used to open it.

Also works if you use file(path('name')) or open(path('name')).
-- 
Michael Hoffman
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Michael Hoffman]

Scott David Daniels wrote:
> Duncan Booth wrote:
> 
>> BTW, does it matter at all in practical use that the base class of 
>> path varies between str and unicode depending on the platform?
> 
> Isn't it even worse than this?
> On Win2K & XP, don't the file systems have something to do with the
> encoding?  So D: (a FAT drive) might naturally be str, while C:
> (an NTFS drive) might naturally be unicode.

The current path module handles these situations at least as well as the 
libraries that come with Python do. ;-)
-- 
Michael Hoffman
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Terry Reedy]

"Michael Hoffman" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
>> on 22.07.2005 00:21 Michael Hoffman said the following:
>>> Any Java hackers here want to tell us of the wonders of the Java Path
>>> class?
> Ah, here it is:
>
> http://java.sun.com/j2se/1.5.0/docs/api/java/io/File.html

Very interesting.  Java's File class is a system-independent "abstract 
representation of file and directory pathnames" which is constructed from 
and converted to system-dependent string-form pathnames (including URL/URI 
file:... forms).  A File consist of an optional prefix and a *sequence* of 
zero or more string names.

In other words, Java's File class is what Duncan and I thought Python's 
Path class might or possibly should be.  So this internal representation 
might be worth considering as an option.  Of course, if the interface is 
done right, it mostly should not make too much difference to the user.

Terry J. Reedy



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Oliver Andrich]

Hi,

2005/7/22, Michael Hoffman <[EMAIL PROTECTED]>:
> What is this Java Path class? I have been STFWing and have found nothing
> on it in the. Indeed if you search for "Java Path class" (with quotes)
> almost half of the pages are this message from Guido. ;)
> 
> Any Java hackers here want to tell us of the wonders of the Java Path
> class? I would be interested in seeing how other OO languages deal with
> paths.

I guess the nearest Java class comparable with path is the File class.

http://java.sun.com/j2se/1.5.0/docs/api/java/io/File.html

And as I am a so called Java hacker, I highly appreciate path as a
module for my python projects and in my eyes it is the natural way to
address files/paths. At least it is more natural to me then the os,
os.path, etc. pp. bundle, that has grown over the time.

I would love to see path inside Python's stdlib.

Best regards,
Oliver 

-- 
Oliver Andrich <[EMAIL PROTECTED]> --- http://fitheach.de/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Peter Hansen]

Reinhold Birkenfeld wrote:
> FYI: I modified the path module a bit so that it fits many of the suggestions
> from python-dev, and put the result in the Python CVS tree under
> nondist/sandbox/path.

By the way, thanks for doing this Reinhold!

> Most prominent change is that it doesn't inherit from str/unicode anymore.
> I found this distinction important, because as a str subclass the Path object
> has many methods that don't make sense for it.

On this topic, has anyone ask the original author (Jason Orendorff) 
whether he has some background on this decision that might benefit the 
discussion?  Given the elegance of the design of the path module, I 
would think if he has an opinion on the matter it is probably based on 
more thought than any of us have given it so far.

And maybe he would even say that it was a wrong decision at the time and 
he'd do it differently the next time.

-Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Peter Hansen]

Duncan Booth wrote:
> As I said elsewhere I haven't 
> used path for anything real, so I'm still finding surprises such as why 
> this doesn't do what I expect:
> 
p = path('a/b')
q = path('c/d')
p+q
> 
> path(u'a/bc/d')

Just a note, though you probably know, that this is intended to be 
written this way with path:

 >>> p / q
path(u'a/b/c/d')

-Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Peter Hansen]

Stefan Rank wrote:

> (It would be nice to get `path`(s) easily from a `file`, at the moment 
> there is only file.name if I'm not mistaken).

When files are opened through a "path" object -- e.g. 
path('name').open() -- then file.name returns the path object that was 
used to open it.

-Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Peter Hansen]

Duncan Booth wrote:
> BTW, does it matter at all in practical use that the base class of path 
> varies between str and unicode depending on the platform?

I haven't seen any problem.  I confess I can't even imagine exactly what 
the problem might be, since they're both subclasses of basestring, 
aren't they?

And current code should have exactly the same issues when using str or 
unicode in all the calls that path() merely wraps.

So does it matter in practical use when one faces this issue and is 
*not* using "path"?

-Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Terry Reedy]

"Daniel Dittmar" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]

> Duncan Booth wrote:
>>  I would have expected a
>> path object to be a sequence of path elements rather than a sequence of
>> characters.

Glad I'm not the only oddball.

> Maybe it's nitpicking, but I don't think that a path object should be a
> 'sequence of path elements' in an iterator context.
>
> This means that
>
> for element in pathobject:
>
> has no intuitive meaning for me, so it shouldn't be allowed.

  The internal equivalent of (simplified, omitting error checking, 
etc.)

for dir in pathobject:
  if isdir(dir): cd(dir)

*is*, in essence, what the OS mainly does with paths (after splitting the 
string representation into pieces).

Directory walks also work with paths as sequences (stacks, in particular).

Terry J. Reedy



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Michael Hoffman]

John Roth wrote:

> However, a path as a sequence of characters has even less
> meaning - I can't think of a use, while I have an application
> where traversing a path as a sequence of path elements makes
> perfect sense: I need to descend the directory structure, directory
> by directory, looking for specific files and types.

I *have* used a path as a sequence of characters before. I had to deal 
with a bunch of filenames that were formatted like "file_02832.a.txt"

I can see the case for a path as a sequence of elements, although in 
practice, drive letters, extensions, and alternate streams complicate 
things.

But as the discussion here unfolds I'm starting to feel that the 
advantages of using a possibly more "meaningful" approach to path as a 
sequence of elements are overwhelmed by the practical advantages of 
using a basestring. Mainly, that you can use it anywhere a basestring 
would be used today and it Just Works.
-- 
Michael Hoffman
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Scott David Daniels]

Duncan Booth wrote:
> BTW, does it matter at all in practical use that the base class of path 
> varies between str and unicode depending on the platform?

Isn't it even worse than this?
On Win2K & XP, don't the file systems have something to do with the
encoding?  So D: (a FAT drive) might naturally be str, while C:
(an NTFS drive) might naturally be unicode.  Even worse, would be a
path that switches in the middle (which it might do if we get to a
ZIP file or use the newer dir-in-file file systems.

--Scott David Daniels
[EMAIL PROTECTED]
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Andrew Dalke]

George Sakkis wrote:
> You're right, conceptually a path
> HAS_A string description, not IS_A string, so from a pure OO point of
> view, it should not inherit string.

How did you decide it's "has-a" vs. "is-a"?

All C calls use a "char *" for filenames and paths,
meaning the C model file for the filesystem says
paths are strings.

Paths as strings fit the Liskov substitution principle
in that any path object can be used any time a
string is used (eg, "loading from " + filename)

Good information hiding suggests that a better API
is one that requires less knowledge.  I haven't
seen an example of how deriving from (unicode)
string makes things more complicated than not doing so.

Andrew
[EMAIL PROTECTED]

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Andrew Dalke]

Duncan Booth wrote:
> Personally I think the concept of a specific path type is a good one, but 
> subclassing string just cries out to me as the wrong thing to do.

I disagree.  I've tried using a class which wasn't derived from
a basestring and kept running into places where it didn't work well.
For example, "open" and "mkdir" take strings as input.  There is no
automatic coercion.

>>> class Spam:
...   def __getattr__(self, name):
... print "Want", repr(name)
... raise AttributeError, name
... 
>>> open(Spam())
Traceback (most recent call last):
  File "", line 1, in ?
TypeError: coercing to Unicode: need string or buffer, instance found
>>> import os
>>> os.mkdir(Spam())
Traceback (most recent call last):
  File "", line 1, in ?
TypeError: coercing to Unicode: need string or buffer, instance found
>>> 

The solutions to this are:
  1) make the path object be derived from str or unicode.  Doing
this does not conflict with any OO design practice (eg, Liskov
substitution).

  2) develop a new "I represent a filename" protocol, probably done
via adapt().

I've considered the second of these but I think it's a more
complicated solution and it won't fit well with existing APIs
which do things like


  if isinstance(input, basestring):
input = open(input, "rU")
  for line in input:
print line

I showed several places in the stdlib and in 3rd party packages
where this is used.


> In other words, to me a path represents something in a filesystem,

Being picky - or something that could be in a filesystem.

> the fact that it 
> has one, or indeed several string representations does not mean that the 
> path itself is simply a more specific type of string.

I didn't follow this.

> You should need an explicit call to convert a path to a string and that 
> forces you when passing the path to something that requires a string to 
> think whether you wanted the string relative, absolute, UNC, uri etc.

You are broadening the definition of a file path to include URIs?
That's making life more complicated.  Eg, the rules for joining
file paths may be different than the rules for joining URIs.
Consider if I have a file named "mail:[EMAIL PROTECTED]" and I
join that with "file://home/dalke/badfiles/".

Additionally, the actions done on URIs are different than on file
paths.  What should os.listdir("http://www.python.org/";) do?

As I mentioned, I tried some classes which emulated file
paths.  One was something like

class TempDir:
  """removes the directory when the refcount goes to 0"""
  def __init__(self):
self.filename = ... use a function from the tempfile module
  def __del__(self):
if os.path.exists(self.filename):
  shutil.rmtree(self.filename)
  def __str__(self):
return self.filename

I could do

  dirname = TempDir()

but then instead of

  os.mkdir(dirname)
  tmpfile = os.path.join(dirname, "blah.txt")

I needed to write it as

  os.mkdir(str(dirname))
  tmpfile = os.path.join(str(dirname), "blah.txt"))

or have two variables, one which could delete the
directory and the other for the name.  I didn't think
that was good design.


If I had derived from str/unicode then things would
have been cleaner.

Please note, btw, that some filesystems are unicode
based and others are not.  As I recall, one nice thing
about the path module is that it chooses the appropriate
base class at import time.  My "str()" example above
does not and would fail on a Unicode filesystem aware
Python build.

> It may even be that we need a hierarchy of path
> classes: URLs need similar but not identical manipulations
> to file paths, so if we want to address the failings
> of os.path perhaps we should also look at the failings 
> of urlparse at the same time.

I've found that hierarchies are rarely useful compared
to the number of times they are proposed and used.  One
of the joys to me of Python is its deemphasis of class
hierarchies.

I think the same is true here.  File paths and URIs are
sufficiently different that there are only a few bits
of commonality between them.  Consider 'split' which
for files creates (dirname, filename) while for urls
it creates (scheme, netloc, path, query, fragment)

Andrew
[EMAIL PROTECTED]

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Duncan Booth]

Michael Hoffman wrote:

> Here's some code I just wrote seconds ago to construct a path for a
> scp upload:
> 
> """
> DST_DIRPATH = path("host:~/destination")
> RSS_EXT = "rss"
> 
> dst_filenamebase = os.extsep.join([postcode.lower(), RSS_EXT])
> dst_filepath = DST_DIRPATH.joinpath(dst_filenamebase)
> """
> 
> With the current path implementation, this Just Works.

It isn't at all obvious to me that it works:

>>> import os
>>> from path import path
>>> postcode = "AA2 9ZZ"
>>> DST_DIRPATH = path("host:~/destination")
>>> RSS_EXT = "rss"
>>>
>>> dst_filenamebase = os.extsep.join([postcode.lower(), RSS_EXT])
>>> dst_filepath = DST_DIRPATH.joinpath(dst_filenamebase)
>>> print dst_filepath
host:~/destination\aa2 9zz.rss


> If I were using 
> something that parsed and understood paths, the scp/rcp convention of 
> host:filename would either cause an error or have to be programmed in 
> separately. The current implementation is much more flexible.

You still have to program your scp path separately from your filesystem 
path in order to handle the different conventions for path separator 
characters and maybe also escaping special characters in the path (I don't 
use scp much so I don't know if this is required).

> What are the practical advantages and conveniences of *not*
> subclassing from basestring?

Simplification of the api: not having methods such as center, expandtabs 
and zfill. 

Not having the base class change from str to unicode depending on which 
system you run your code?

Fewer undetected bugs (explicit is better than implicit)?

Perhaps none of these matter in practice. As I said elsewhere I haven't 
used path for anything real, so I'm still finding surprises such as why 
this doesn't do what I expect:

>>> p = path('a/b')
>>> q = path('c/d')
>>> p+q
path(u'a/bc/d')

If path didn't subclass string then either this would have been 
implemented, and probably would Do The Right Thing, or it wouldn't be 
implemented so I'd quickly realise I needed to do something else. Instead 
it does something suprising.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[John Roth]

"Daniel Dittmar" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
> Duncan Booth wrote:
>>  I would have expected a path object to be a sequence of path elements 
>> rather than a sequence of characters.
>
> Maybe it's nitpicking, but I don't think that a path object should be a 
> 'sequence of path elements' in an iterator context.
>
> This means that
>
> for element in pathobject:
>
> has no intuitive meaning for me, so it shouldn't be allowed.

However, a path as a sequence of characters has even less
meaning - I can't think of a use, while I have an application
where traversing a path as a sequence of path elements makes
perfect sense: I need to descend the directory structure, directory
by directory, looking for specific files and types.

John Roth
>
> Daniel 

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[John Roth]

"Duncan Booth" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
>
> John Roth wrote:
>> You have to start somewhere. One of the lessons that's beginning
>> to seep into people's minds is that getting something that works
>> out there is almost always preferable to (over) design by committee.
>
> Dead right, but once it goes into the standard library it has to pretty
> well stop evolving, so it needs to be right, or as close as possible 
> before
> that happens.

It has to stop evolving in incompatible directions, at least. Although
there is a precident with the process functions, classes, module,
whatever it is. It's up to five versions now, isn't it?

AFAICT, from a very broad brush perspective, there is really
only one substantive issue: how to handle multiple path-like
"things". URLs  have been mentioned in this thread, different
file systems and a possible in-memory file system have been
mentioned in other threads.

So whatever gets out there first shouldn't preempt the ability
to eventually fit into a wider structure without substantial
and incompatible modifications.

John Roth
> 

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Daniel Dittmar]

Duncan Booth wrote:
>  I would have expected a 
> path object to be a sequence of path elements rather than a sequence of 
> characters. 

Maybe it's nitpicking, but I don't think that a path object should be a 
'sequence of path elements' in an iterator context.

This means that

for element in pathobject:

has no intuitive meaning for me, so it shouldn't be allowed.

Daniel
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Duncan Booth]

Peter Hansen wrote:

> Duncan, are you another formerly non-user of path who has this opinion, 
> or have you already attempted to use path extensively in your code?

I'm a currently non-user of path who would probably use it if it were in 
the standard library but so far have been satisfied to use os.path.

> I'm not saying I dismiss the opinions of those who haven't actually 
> tried working with a string-based path object, but it's worth 
> considering that you might adopt a different opinion after using it for 
> a while.

I fully accept that. My point is simply that as a non-user, it sounds to me 
as though subclassing string is the wrong approach. I would have expected a 
path object to be a sequence of path elements rather than a sequence of 
characters. This is basically just a gut feeling though, so I'm perfectly 
happy to be told that I'm wrong.

BTW, does it matter at all in practical use that the base class of path 
varies between str and unicode depending on the platform?

John Roth wrote:
> You have to start somewhere. One of the lessons that's beginning
> to seep into people's minds is that getting something that works
> out there is almost always preferable to (over) design by committee.

Dead right, but once it goes into the standard library it has to pretty 
well stop evolving, so it needs to be right, or as close as possible before 
that happens.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[George Sakkis]

"Duncan Booth" <[EMAIL PROTECTED]> wrote:

> Personally I think the concept of a specific path type is a good one, but
> subclassing string just cries out to me as the wrong thing to do. In other
> words, to me a path represents something in a filesystem, the fact that it
> has one, or indeed several string representations does not mean that the
> path itself is simply a more specific type of string.
>
> You should need an explicit call to convert a path to a string and that
> forces you when passing the path to something that requires a string to
> think whether you wanted the string relative, absolute, UNC, uri etc.

First off, I find this is a relatively small detail overall, that is, 
regardless of whether path
subclasses string or not, its addition in the standard library will be a step 
forward. Havind said
that, I think the choice between subclassing or not is going to be a 
practicality-vs-purity
decision. You're right, conceptually a path HAS_A string description, not IS_A 
string, so from a
pure OO point of view, it should not inherit string. OTOH, people in favor of 
the subclassing point
out the convenience for many (or most) common cases. It's a tradeoff, so 
arguments for both cases
should be discussed.

George




-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Michael Hoffman]

George Sakkis wrote:

> Havind said that, I think
> the choice between subclassing or not is going to be a
> practicality-vs-purity decision. You're right, conceptually a path
> HAS_A string description, not IS_A string, so from a pure OO point of
> view, it should not inherit string. OTOH, people in favor of the
> subclassing point out the convenience for many (or most) common cases.

It would be an entirely different matter if we were designing a language 
from scratch. But we have to deal with an existing codebase that expects 
strings.

Here's some code I just wrote seconds ago to construct a path for a scp 
upload:

"""
DST_DIRPATH = path("host:~/destination")
RSS_EXT = "rss"

dst_filenamebase = os.extsep.join([postcode.lower(), RSS_EXT])
dst_filepath = DST_DIRPATH.joinpath(dst_filenamebase)
"""

With the current path implementation, this Just Works. If I were using 
something that parsed and understood paths, the scp/rcp convention of 
host:filename would either cause an error or have to be programmed in 
separately. The current implementation is much more flexible.

What are the practical advantages and conveniences of *not* subclassing 
from basestring?
-- 
Michael Hoffman
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Stefan Rank]

on 22.07.2005 16:14 George Sakkis said the following:
> "Duncan Booth" <[EMAIL PROTECTED]> wrote:
> 
>>Personally I think the concept of a specific path type is a good one, but
>>subclassing string just cries out to me as the wrong thing to do. In other
>>words, to me a path represents something in a filesystem, the fact that it
>>has one, or indeed several string representations does not mean that the
>>path itself is simply a more specific type of string.
>>
[snip]
> practicality-vs-purity decision. You're right, conceptually a path
> HAS_A string description, not IS_A string, so from a pure OO point of
> view, it should not inherit string.

Java has `File` which mixes the concepts "an object in the filesystem" 
and "a structured locator for such objects (in a hierarchical fs) that 
might or might not correspond to an object that is actually there".

`file` and `path` separate that. I think this is very reasonable.

(It would be nice to get `path`(s) easily from a `file`, at the moment 
there is only file.name if I'm not mistaken).

And a `path`, to me, actually IS_A string (unicode string) that happens 
to have special behaviour (such as os dependent quirks like a 
pathseparator that automatically get translated, comparable to '\n' used 
internally in strings translated to '\n'|'\r\n')

stefan

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[John Roth]

"Duncan Booth" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
> George Sakkis wrote:
>
>
> You should need an explicit call to convert a path to a string and that
> forces you when passing the path to something that requires a string to
> think whether you wanted the string relative, absolute, UNC, uri etc.
>
> It may even be that we need a hierarchy of path classes: URLs need similar
> but not identical manipulations to file paths, so if we want to address 
> the
> failings of os.path perhaps we should also look at the failings of 
> urlparse
> at the same time.

You have to start somewhere. One of the lessons that's beginning
to seep into people's minds is that getting something that works
out there is almost always preferable to (over) design by committee.

How to do a comprehensive, covers all the corner cases file
system object (or object hierarchy, etc) has been discussed before,
and nothing has ever come of it. Starting with an object that
actually does something some people want gives the designers a
chance to look at things in the wild.

John Roth 

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[George Sakkis]

"Duncan Booth" <[EMAIL PROTECTED]> wrote:

> Personally I think the concept of a specific path type is a good one, but
> subclassing string just cries out to me as the wrong thing to do. In other
> words, to me a path represents something in a filesystem, the fact that it
> has one, or indeed several string representations does not mean that the
> path itself is simply a more specific type of string.
>
> You should need an explicit call to convert a path to a string and that
> forces you when passing the path to something that requires a string to
> think whether you wanted the string relative, absolute, UNC, uri etc.

First off, I find this is a relatively small detail overall, that is,
regardless of whether path subclasses string or not, its addition in
the standard library will be a step forward. Havind said that, I think
the choice between subclassing or not is going to be a
practicality-vs-purity decision. You're right, conceptually a path
HAS_A string description, not IS_A string, so from a pure OO point of
view, it should not inherit string. OTOH, people in favor of the
subclassing point out the convenience for many (or most) common cases.
It's a tradeoff, so arguments for both cases should be discussed.

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Peter Hansen]

Duncan Booth wrote:
> Personally I think the concept of a specific path type is a good one, but 
> subclassing string just cries out to me as the wrong thing to do. In other 
> words, to me a path represents something in a filesystem, the fact that it 
> has one, or indeed several string representations does not mean that the 
> path itself is simply a more specific type of string.
> 
> You should need an explicit call to convert a path to a string and that 
> forces you when passing the path to something that requires a string to 
> think whether you wanted the string relative, absolute, UNC, uri etc.

Duncan, are you another formerly non-user of path who has this opinion, 
or have you already attempted to use path extensively in your code?

I'm not saying I dismiss the opinions of those who haven't actually 
tried working with a string-based path object, but it's worth 
considering that you might adopt a different opinion after using it for 
a while.

I did.

-Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Peter Hansen]

Reinhold Birkenfeld wrote:
> Michael Hoffman wrote:
>>What do you gain from removing these methods? A smaller dir()?
> 
> It made sense to me at the time I changed this, although at the moment
> I can't exactly recall the reasons.

Along with some of the others (and as a fairly heavy user of "path"), I 
would caution strongly against jumping to do make this change.

Given that a strong part of the justification for path's inclusion in 
the standard library (as expressed here in the past) is that it is 
stable and widely used, making such a fundamental change at this late 
stage just prior to its possible acceptance seems to me very risky.

I have noticed in a number of cases where a "path" was usable as a 
drop-in replacement for strings that previously contained paths.  I 
can't say for sure, but I strongly suspect some of that could would be 
broken if "paths" weren't basestrings.  I'll attempt to check in my own 
code.

Even if those uses don't actually break, it can *also* be useful to have 
the string methods available on a path object, so I'm also uncertain 
what you gain by removing that connection, though it's clear what things 
you might be losing.

-2 for this idea.

-Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Harald Armin Massa]

>Having path descend from str/unicode is extremely useful since I can
>then pass a path object to any function someone else wrote without
>having to worry about whether they were checking for basestring.
I use path.py from Jason to encapsulate a lot of the windows plattform
specialities of path dealing.
Being able to use path-opjects at every place where I would use str or
unicode is very essential, because I often use Python to tame Excel and
Word. To open files within these programms needs some "plain str" as
"PATH" for the file. (which, of course, can also be down by ways to
"convert" PATH to STRING.

Harald

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Harald Armin Massa]

> When you try to 'freeze' an application module,
>and Jason's 'path' module is present in any of the directories that are
>looked at by freeze's module finder (your app doesn't have to import
>it), freeze goes into an infinite loop of imports, eventually getting a
>'maximum recursion depth' exception.  This seems to be related to
>freeze getting confused between 'os.path' and Jason's 'path'.

This is a bug in distutils. Thomas Hellers py2exe encounters the same
bug. As much as I remember our conversation, he submitted a patch to
distutils.

In the meanwhile I renamed path.py to jpath.py, usings Jason's first
letter in a motion of honour while circumventing this bug.

Harald

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Thomas Heller]

[EMAIL PROTECTED] writes:

> I really love Jason's 'path' module.  Sadly, I've encountered a serious
> problem with using it.  When you try to 'freeze' an application module,
> and Jason's 'path' module is present in any of the directories that are
> looked at by freeze's module finder (your app doesn't have to import
> it), freeze goes into an infinite loop of imports, eventually getting a
> 'maximum recursion depth' exception.  This seems to be related to
> freeze getting confused between 'os.path' and Jason's 'path'.
>
> I encountered this using Jason's latest 'path' module and Python 2.3.2.
>  I was able to solve it for my use by renaming path.py to newpath.py
> and using 'from newpath import path' in my modules.
>
> I've just notified Jason about this.  I presume a solution like mine
> will be used, and look forward to seeing Jason's module in stdlib.

That was a bug in modulefinder, which was fixed in Python 2.4 and 2.3.4.
See http://www.python.org/sf/876278.

Thomas
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Michael Hoffman]

Duncan Booth wrote:

> You should need an explicit call to convert a path to a string and that 
> forces you when passing the path to something that requires a string to 
> think whether you wanted the string relative, absolute, UNC, uri etc.

Egad. I'm not sure if that will really make people's lives easier.
-- 
Michael Hoffman
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Duncan Booth]

George Sakkis wrote:

>> Read literally, this says (at least to me) "I don't want to fix it
>> because I don't think it's broke."
> 
> Or rather "I prefer a single existing mediocre solution than two
> solutions (even if the second was better)".
> 
Except that he is open to persuasion, so the PEP has to demonstrate that 
the duplication is worth the benefit.

Personally I think the concept of a specific path type is a good one, but 
subclassing string just cries out to me as the wrong thing to do. In other 
words, to me a path represents something in a filesystem, the fact that it 
has one, or indeed several string representations does not mean that the 
path itself is simply a more specific type of string.

You should need an explicit call to convert a path to a string and that 
forces you when passing the path to something that requires a string to 
think whether you wanted the string relative, absolute, UNC, uri etc.

It may even be that we need a hierarchy of path classes: URLs need similar 
but not identical manipulations to file paths, so if we want to address the 
failings of os.path perhaps we should also look at the failings of urlparse 
at the same time.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[George Sakkis]

"John Roth" <[EMAIL PROTECTED]> wrote:

> "Reinhold Birkenfeld" <[EMAIL PROTECTED]> wrote
> > He said,
> >
> > """
> > Whoa! Do we really need a completely different mechanism for doing the
> > same stuff we can already do? The path module seems mostly useful for
> > folks coming from Java who are used to the Java Path class. With the
> > massive duplication of functionality we should also consider what to
> > recommend for the future: will the old os.path module be deprecated,
> > or are we going to maintain both alternatives forever? (And what about
> > all the duplication with the os module itself, like the cwd()
> > constructor?) Remember TOOWTDI.
> > """
>
> Read literally, this says (at least to me) "I don't want to fix it because
> I don't think it's broke."

Or rather "I prefer a single existing mediocre solution than two solutions 
(even if the second was
better)".

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Neil Hodgson]

Reinhold Birkenfeld:

> And it is much more "Pythonic" in my eyes. Though that word may be inaccurate
> when it comes from someone else that Guido, I feel that endless chains of
> '[os.path.join(os.path.join(z, "a"), x) for x in 
> os.path.listdir(os.path.join(z, "a") if os.path.isfile(os.path.join(
> are not qualified as being Pythonic.

I like Path but the above is trying too hard to be poor code. 
os.path.join takes more than 2 arguments, so that should be
[os.path.join(z, "a", x) for x in os.path.listdir(os.path.join(z, "a") 
if os.path.isfile(os.path.join(

Neil
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[George Sakkis]

"Reinhold Birkenfeld" <[EMAIL PROTECTED]> wrote:

> > Why did Guido want a PEP? Is it because he likes the idea but
> > feels the feature set needs to be examined a bit more by the wider
> > community, or is it some other reason?
>
> He said,
>
> """
> Whoa! Do we really need a completely different mechanism for doing the
> same stuff we can already do? The path module seems mostly useful for
> folks coming from Java who are used to the Java Path class. With the
> massive duplication of functionality we should also consider what to
> recommend for the future: will the old os.path module be deprecated,
> or are we going to maintain both alternatives forever? (And what about
> all the duplication with the os module itself, like the cwd()
> constructor?) Remember TOOWTDI.
> """

Duplication is a valid point for debate, so the PEP should definitely address 
it. IMO os.path and
most (if not all) other equivalent modules and functions should be deprecated, 
though still working
until 2.9 for backwards compatibility, and dropped for python 3K.

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[John Roth]

"Reinhold Birkenfeld" <[EMAIL PROTECTED]> wrote in 
message news:[EMAIL PROTECTED]
> John Roth wrote:
>> "Michael Hoffman" <[EMAIL PROTECTED]> wrote in message
>> news:[EMAIL PROTECTED]
>>> Many of you are familiar with Jason Orendorff's path module
>>> , which is frequently
>>> recommended here on c.l.p. I submitted an RFE to add it to the Python
>>> standard library, and Reinhold Birkenfeld started a discussion on it in
>>> python-dev
>>> .
>>>
>>> The upshot of the discussion was that many python-dev'ers wanted path
>>> added to the stdlib, but Guido was not convinced and said it must have a
>>> PEP.
>>
>> Why did Guido want a PEP? Is it because he likes the idea but
>> feels the feature set needs to be examined a bit more by the wider
>> community, or is it some other reason?
>
> He said,
>
> """
> Whoa! Do we really need a completely different mechanism for doing the
> same stuff we can already do? The path module seems mostly useful for
> folks coming from Java who are used to the Java Path class. With the
> massive duplication of functionality we should also consider what to
> recommend for the future: will the old os.path module be deprecated,
> or are we going to maintain both alternatives forever? (And what about
> all the duplication with the os module itself, like the cwd()
> constructor?) Remember TOOWTDI.
> """

Read literally, this says (at least to me) "I don't want to fix it because
I don't think it's broke."

As far as the Java remark is concerned, I suspect that it's because
in Java there is no such thing as a function; everything is either a
method on an object or a static method on a class.

And as far as I'm concerned, it's broke. I could see getting rid of the
bits and  pieces of  procedural code in 3.0, but not sooner.

John Roth
>
> Reinhold 

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Reinhold Birkenfeld]

John Roth wrote:
> "Michael Hoffman" <[EMAIL PROTECTED]> wrote in message 
> news:[EMAIL PROTECTED]
>> Many of you are familiar with Jason Orendorff's path module 
>> , which is frequently 
>> recommended here on c.l.p. I submitted an RFE to add it to the Python 
>> standard library, and Reinhold Birkenfeld started a discussion on it in 
>> python-dev 
>> .
>>
>> The upshot of the discussion was that many python-dev'ers wanted path 
>> added to the stdlib, but Guido was not convinced and said it must have a 
>> PEP.
> 
> Why did Guido want a PEP? Is it because he likes the idea but
> feels the feature set needs to be examined a bit more by the wider
> community, or is it some other reason?

He said,

"""
Whoa! Do we really need a completely different mechanism for doing the
same stuff we can already do? The path module seems mostly useful for
folks coming from Java who are used to the Java Path class. With the
massive duplication of functionality we should also consider what to
recommend for the future: will the old os.path module be deprecated,
or are we going to maintain both alternatives forever? (And what about
all the duplication with the os module itself, like the cwd()
constructor?) Remember TOOWTDI.
"""

Reinhold
-- 
http://mail.python.org/mailman/listinfo/python-list

!Re: PEP on path module for standard library

[John Roth]

"Michael Hoffman" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
> Many of you are familiar with Jason Orendorff's path module 
> , which is frequently 
> recommended here on c.l.p. I submitted an RFE to add it to the Python 
> standard library, and Reinhold Birkenfeld started a discussion on it in 
> python-dev 
> .
>
> The upshot of the discussion was that many python-dev'ers wanted path 
> added to the stdlib, but Guido was not convinced and said it must have a 
> PEP.

Why did Guido want a PEP? Is it because he likes the idea but
feels the feature set needs to be examined a bit more by the wider
community, or is it some other reason?

I'm all in favor of having something that gives an alternative to
the kludge of functions that are "just a thin wrapper on the C
standard library." Considering the known problems with the
C standard library and the fact that it's strictly procedural, that
statement doesn't fill me with confidence. Rather it creates a
mild sense of dread: nobody has thought out how to do those
functions in a useful oo manner.

Path looks useable to me. Do I think it's going to be the
last word? I sincerely hope not! The only way we're going
to find out where it really needs to go from here, though, is
to put it out and find out how the wider community uses
and abuses it.

John Roth

> Thanks in advance,
> -- 
> Michael Hoffman 

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Reinhold Birkenfeld]

Michael Hoffman wrote:
> [EMAIL PROTECTED] wrote:
>> I really love Jason's 'path' module.  Sadly, I've encountered a serious
>> problem with using it.  When you try to 'freeze' an application module,
>> and Jason's 'path' module is present in any of the directories that are
>> looked at by freeze's module finder (your app doesn't have to import
>> it), freeze goes into an infinite loop of imports, eventually getting a
>> 'maximum recursion depth' exception.  This seems to be related to
>> freeze getting confused between 'os.path' and Jason's 'path'.
>> 
>> I encountered this using Jason's latest 'path' module and Python 2.3.2.
>>  I was able to solve it for my use by renaming path.py to newpath.py
>> and using 'from newpath import path' in my modules.
>> 
>> I've just notified Jason about this.  I presume a solution like mine
>> will be used, and look forward to seeing Jason's module in stdlib.
> 
> This sounds like a bug in "freeze" rather than something that should be 
> worked around in the standard library. Although there have already been 
> people opposed to naming it path because the duplication with os.path 
> might confuse humans.

As the most likely placement will be a class named "Path" inside the "os.path"
module, that bug with freeze won't apply to the "stdlib version" of Path.

Reinhold
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Reinhold Birkenfeld]

Michael Hoffman wrote:
> Reinhold Birkenfeld wrote:
>> FYI: I modified the path module a bit so that it fits many of the suggestions
>> from python-dev, and put the result in the Python CVS tree under
>> nondist/sandbox/path.
>> 
>> Most prominent change is that it doesn't inherit from str/unicode anymore.
>> I found this distinction important, because as a str subclass the Path object
>> has many methods that don't make sense for it.
> 
> Having path descend from str/unicode is extremely useful since I can 
> then pass a path object to any function someone else wrote without 
> having to worry about whether they were checking for basestring. I think 
> there is a widely used pattern of accepting either a basestring[1] or a 
> file-like object as a function argument, and using isinstance() to 
> figure out which it is.

Where do you see that pattern? IIRC it's not in the stdlib.

> What do you gain from removing these methods? A smaller dir()?

It made sense to me at the time I changed this, although at the moment
I can't exactly recall the reasons.

Probably as Terry said: a path is both a list and a string.

Reinhold
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Michael Hoffman]

Reinhold Birkenfeld wrote:
> FYI: I modified the path module a bit so that it fits many of the suggestions
> from python-dev, and put the result in the Python CVS tree under
> nondist/sandbox/path.
> 
> Most prominent change is that it doesn't inherit from str/unicode anymore.
> I found this distinction important, because as a str subclass the Path object
> has many methods that don't make sense for it.

Having path descend from str/unicode is extremely useful since I can 
then pass a path object to any function someone else wrote without 
having to worry about whether they were checking for basestring. I think 
there is a widely used pattern of accepting either a basestring[1] or a 
file-like object as a function argument, and using isinstance() to 
figure out which it is.

What do you gain from removing these methods? A smaller dir()?

[1] Probably str in actuality.
-- 
Michael Hoffman
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Michael Hoffman]

Stefan Rank wrote:
> on 22.07.2005 00:21 Michael Hoffman said the following:
>> Any Java hackers here want to tell us of the wonders of the Java Path 
>> class?
> 
> no such thing exists.
> 
> there is only the `File` class that incorporates a little bit of the 
> `path` functionality and some of the python built-in `file` functionality.

Ah, here it is:

http://java.sun.com/j2se/1.5.0/docs/api/java/io/File.html
-- 
Michael Hoffman
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Andrew Dalke]

Michael Hoffman wrote:
> Having path descend from str/unicode is extremely useful since I can 
> then pass a path object to any function someone else wrote without 
> having to worry about whether they were checking for basestring. I think 
> there is a widely used pattern of accepting either a basestring[1] or a 
> file-like object as a function argument, and using isinstance() to 
> figure out which it is.

Reinhold Birkenfeld wrote:
> Where do you see that pattern? IIRC it's not in the stdlib.

Here's the first place that comes to mind for me

xml.sax.saxutils

def prepare_input_source(source, base = ""):
"""This function takes an InputSource and an optional base URL and
returns a fully resolved InputSource object ready for reading."""

if type(source) in _StringTypes:
source = xmlreader.InputSource(source)
elif hasattr(source, "read"):
f = source
source = xmlreader.InputSource()
source.setByteStream(f)
if hasattr(f, "name"):
source.setSystemId(f.name)


and xml.dom.pulldom

def parse(stream_or_string, parser=None, bufsize=None):
if bufsize is None:
bufsize = default_bufsize
if type(stream_or_string) in _StringTypes:
stream = open(stream_or_string)
else:
stream = stream_or_string
if not parser:
parser = xml.sax.make_parser()
return DOMEventStream(stream, parser, bufsize)

Using the power of grep

aifc.py
def __init__(self, f):
if type(f) == type(''):
f = __builtin__.open(f, 'rb')
# else, assume it is an open file object already
self.initfp(f)

binhex.py
class HexBin:
def __init__(self, ifp):
if type(ifp) == type(''):
ifp = open(ifp)

imghdr.py
if type(file) == type(''):
f = open(file, 'rb')
h = f.read(32)
else:
location = file.tell()
h = file.read(32)
file.seek(location)
f = None

mimify.py
if type(infile) == type(''):
ifile = open(infile)
if type(outfile) == type('') and infile == outfile:
import os
d, f = os.path.split(infile)
os.rename(infile, os.path.join(d, ',' + f))
else:
ifile = infile

wave.py
def __init__(self, f):
self._i_opened_the_file = None
if type(f) == type(''):
f = __builtin__.open(f, 'rb')
self._i_opened_the_file = f
# else, assume it is an open file object already
self.initfp(f)


compiler/transformer.py:

if type(file) == type(''):
file = open(file)
return self.parsesuite(file.read())

plat-mac/applesingle.py
if type(input) == type(''):
input = open(input, 'rb')
# Should we also test for FSSpecs or FSRefs?
header = input.read(AS_HEADER_LENGTH)

site-packages/ZODB/ExportImport.py
if file is None: file=TemporaryFile()
elif type(file) is StringType: file=open(file,'w+b')


site-packages/numarray/ndarray.py
if type(file) == type(""):
name = 1
file = open(file, 'wb')


site-packages/kiva/imaging/GdImageFile.py
if type(fp) == type(""):
import __builtin__
filename = fp
fp = __builtin__.open(fp, "rb")
else:
filename = ""

site-packages/reportlab/graphics/renderPM.py
   if type(image.path) is type(''):
im = _getImage().open(image.path).convert('RGB')
   else:
im = image.path.convert('RGB')


site-packages/twisted/protocols/irc.py
def __init__(self, file):
if type(file) is types.StringType:
self.file = open(file, 'r')

(hmm, that last one looks buggy.  It should
have a "else: self.file = file" afterwards.)


Used in the std. lib and used by many different
people.  (I excluded the Biopython libraries
in this list, btw, because I may have influenced
the use of this sort of type check.)

Andrew
[EMAIL PROTECTED]

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Stefan Rank]

on 22.07.2005 00:21 Michael Hoffman said the following:
> Reinhold Birkenfeld wrote:
> 
>>John Roth wrote:
>>
>>
>>>Why did Guido want a PEP?
>>
>>He said,
>>
>>"""
>>Whoa! Do we really need a completely different mechanism for doing the
>>same stuff we can already do? The path module seems mostly useful for
>>folks coming from Java who are used to the Java Path class.
>  > """
> 
> What is this Java Path class? I have been STFWing and have found nothing 
> on it in the. Indeed if you search for "Java Path class" (with quotes) 
> almost half of the pages are this message from Guido. ;)
> 
> Any Java hackers here want to tell us of the wonders of the Java Path 
> class?

no such thing exists.

there is only the `File` class that incorporates a little bit of the 
`path` functionality and some of the python built-in `file` functionality.

my little self would actually propose to make *path* a built-in type as 
an intermediate between file and basestring/str/unicode (the latter is 
probably needed).

*ducks*

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[mkent]

I really love Jason's 'path' module.  Sadly, I've encountered a serious
problem with using it.  When you try to 'freeze' an application module,
and Jason's 'path' module is present in any of the directories that are
looked at by freeze's module finder (your app doesn't have to import
it), freeze goes into an infinite loop of imports, eventually getting a
'maximum recursion depth' exception.  This seems to be related to
freeze getting confused between 'os.path' and Jason's 'path'.

I encountered this using Jason's latest 'path' module and Python 2.3.2.
 I was able to solve it for my use by renaming path.py to newpath.py
and using 'from newpath import path' in my modules.

I've just notified Jason about this.  I presume a solution like mine
will be used, and look forward to seeing Jason's module in stdlib.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Michael Hoffman]

Reinhold Birkenfeld wrote:
> John Roth wrote:
> 
>>Why did Guido want a PEP?
> 
> He said,
> 
> """
> Whoa! Do we really need a completely different mechanism for doing the
> same stuff we can already do? The path module seems mostly useful for
> folks coming from Java who are used to the Java Path class.
 > """

What is this Java Path class? I have been STFWing and have found nothing 
on it in the. Indeed if you search for "Java Path class" (with quotes) 
almost half of the pages are this message from Guido. ;)

Any Java hackers here want to tell us of the wonders of the Java Path 
class? I would be interested in seeing how other OO languages deal with 
paths.
-- 
Michael Hoffman
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Michael Hoffman]

Reinhold Birkenfeld wrote:
> Michael Hoffman wrote:
> 
>>Having path descend from str/unicode is extremely useful since I can 
>>then pass a path object to any function someone else wrote without 
>>having to worry about whether they were checking for basestring. I think 
>>there is a widely used pattern of accepting either a basestring[1] or a 
>>file-like object as a function argument, and using isinstance() to 
>>figure out which it is.
> 
> Where do you see that pattern? IIRC it's not in the stdlib.

I do not think it is a *good* pattern, but it is used in Biopython. Of 
course, there they ARE using things like type("") so on a unicode 
filesystem it would already break. I seem to recall seeing it elsewhere, 
but I can't remember where.

If you remove the basestring superclass, then you remove the ability to 
use path objects as a drop-in replacement for any path string right now. 
You will either have to use str(pathobj) or carefully check that the 
function/framework you are passing the path to does not use isinstance() 
or any of the string methods that are now gone.

>>What do you gain from removing these methods? A smaller dir()?
> 
> It made sense to me at the time I changed this, although at the moment
> I can't exactly recall the reasons.
> 
> Probably as Terry said: a path is both a list and a string.

I can see the case for thinking of it in both of those ways. In the end 
a path is a sequence object. But a sequence of what?

I have a path that looks like this:

r"c:\windows\system32:altstream\test.dir\myfile.txt.zip:altstream"

One way to divide this is solely based on path separators:

['c:', 'windows', 'system32:altstream', 'test.dir', 
'myfile.txt.zip:altstream']

But then some of the elements of this sequence have more meaning than 
just being strings. "c:" is certainly something different from 
"windows." The file name and alternate data stream name of each element 
could be represented as a tuple.

The extensions can also be dealt with as a sequence. I have dealt with 
things like filename = "filename.x.y.z" and wanted to get "filename.x" 
before. The current stdlib solution, 
os.path.splitext(os.path.splitext(filename)[0])[0] is extremely clunky, 
and I have long desired something better. (OK, using 
filename.split(os.extsep) works a little better, but you get the idea.)

So if you start breaking the path into a sequence of bigger items than 
single character, where does it stop? What is a good design for this?
-- 
Michael Hoffman
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Michael Hoffman]

[EMAIL PROTECTED] wrote:
> I really love Jason's 'path' module.  Sadly, I've encountered a serious
> problem with using it.  When you try to 'freeze' an application module,
> and Jason's 'path' module is present in any of the directories that are
> looked at by freeze's module finder (your app doesn't have to import
> it), freeze goes into an infinite loop of imports, eventually getting a
> 'maximum recursion depth' exception.  This seems to be related to
> freeze getting confused between 'os.path' and Jason's 'path'.
> 
> I encountered this using Jason's latest 'path' module and Python 2.3.2.
>  I was able to solve it for my use by renaming path.py to newpath.py
> and using 'from newpath import path' in my modules.
> 
> I've just notified Jason about this.  I presume a solution like mine
> will be used, and look forward to seeing Jason's module in stdlib.

This sounds like a bug in "freeze" rather than something that should be 
worked around in the standard library. Although there have already been 
people opposed to naming it path because the duplication with os.path 
might confuse humans.
-- 
Michael Hoffman
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Terry Reedy]

"Reinhold Birkenfeld" <[EMAIL PROTECTED]> wrote in 
message news:[EMAIL PROTECTED]
> Most prominent change is that it doesn't inherit from str/unicode 
> anymore.
> I found this distinction important, because as a str subclass the Path 
> object
> has many methods that don't make sense for it.

While I am - on adding more to learn, I am + on collecting scattered 
filesystem functions into methods of coherent classes for eventually 
replacement of the former.  And I understand that this will mean a period 
of duplication.  But it is important to get the replacement right.

My only knowledge of the path module is what has been posted.  However, it 
seems to me that a path is conceptually a sequence of strings, rather than 
the single joined-string representation thereof.  If so, then of course it 
is not and should not be a subclass of single strings.  But that Path was 
so defined makes me wonder, in my ignorance, whether the current 
implementation is the best we can do for the future.

One advantage, for instance, of a split-up list implementation is that a 
set of paths with a common prefix could be represented by replacing the 
last string with a set of strings.

A more OO-friendly OS than the dominant ones today would accept a path as a 
list (sequence) instead of requiring that the list be joined (by an 
artifactual character) just to be split back into a list again.

My thoughts anyway.

Terry J. Reedy



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Reinhold Birkenfeld]

FYI: I modified the path module a bit so that it fits many of the suggestions
from python-dev, and put the result in the Python CVS tree under
nondist/sandbox/path.

Most prominent change is that it doesn't inherit from str/unicode anymore.
I found this distinction important, because as a str subclass the Path object
has many methods that don't make sense for it.

Peter Hansen wrote:
> Michael Hoffman wrote:
>> For the PEP, do any of you have arguments for or against including path? 
>> Code samples that are much easier or more difficult with this class 
>> would also be most helpful.
> 
> I believe the strongest argument for "path" can be made for how it 
> integrates functionality which, although closely related conceptually, 
> is currently distributed across a half dozen or more different modules 
> in the standard library.  Especially for newbies (I can only imagine, at 
> this stage) it would make working with files much easier in a many ways.
> 
> "Easier" or "more difficult" is a subjective thing, of course, but one 
> can't argue with the fact that path can sometimes do through a single 
> object what would otherwise require several imports and a bunch of calls 
> into things like open(), os.path, grep, and shutil.

Correct.

> Examples showing effective uses of path that simplify those cases would 
> probably merit the label "easier" even in Guido's mind, though 
> unfortunately that's not certain.  "Easier" in some minds might simply 
> translate to "many lines less code", and while path can sometimes do 
> that, aside from the ease of splitting and joining stuff without 
> multiple calls to os.path.this-and-that, it really doesn't often reduce 
> code size _that_ much, in my experience.  (Postings to c.l.p showing a 
> 50% reduction in code size for contrived examples notwithstanding.)

Well, these examples are the ones we'd like to see here. So, people:
If you posted examples to c.l.py in the past, please try to collect
them here!

> A related thoughts: since paths are objects, they have attributes or 
> properties, and having things like ".basename" and ".parent" readily 
> available without having to do obscure things like 
> os.path.split(somepath)[0] makes things much easier to read (therefore 
> more maintainable).  In fact, I'd propose that as another strong 
> argument in path's favour: it makes code much more readable, even if not 
> "easier" to write.
> 
> Hmm... does "easier" or "more difficult" apply to the writing of the 
> code or the reading of it?  I find it self-evident that code written 
> using "path" is much easier to read, not necessarily much easier to 
> write (for non-newbies).

And it is much more "Pythonic" in my eyes. Though that word may be inaccurate
when it comes from someone else that Guido, I feel that endless chains of
'[os.path.join(os.path.join(z, "a"), x) for x in 
os.path.listdir(os.path.join(z, "a") if os.path.isfile(os.path.join(
are not qualified as being Pythonic.

> I'd summarize this by saying that the integration of "path" in the 
> stdlib would make it easier for newbies to write code (that might not be 
> obvious to a non-newbie... shall we ask some to help?), and easier for 
> everyone to read code (self-evident, no?), and if that's not a 
> sufficient condition for inclusion I don't know what is.

Reinhold
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Fernando Perez]

Peter Hansen wrote:

> Michael Hoffman wrote:
>> For the PEP, do any of you have arguments for or against including path?
>> Code samples that are much easier or more difficult with this class
>> would also be most helpful.
> 
> I believe the strongest argument for "path" can be made for how it
> integrates functionality which, although closely related conceptually,
> is currently distributed across a half dozen or more different modules
> in the standard library.  Especially for newbies (I can only imagine, at
> this stage) it would make working with files much easier in a many ways.

+10

One of the few things that annoys me about the stdlib is what one could call
performing 'shell-scripting-like' tasks, and precisely because of the problem
you point out.  A number of conceptually related and common tasks are
scattered all over, and every time I need to write this kind of code, I find
myself paging over the docs for multiple modules, with no real intuition as to
where I could even guess where to find things.  This is very unusual for
python, where in most cases things are so well organized, that blind guessing
tends to work remarkably well.  

Personally I like the path module _a lot_, though I'm sure a thorough once-over
from c.l.py and python-dev, via a PEP, can only make it better and smooth out
hidden rough edges and corner cases.  But I'll be very happy if it does go
into the stdlib in the future.

Just my .02.

Best,

f

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PEP on path module for standard library

[Peter Hansen]

Michael Hoffman wrote:
> For the PEP, do any of you have arguments for or against including path? 
> Code samples that are much easier or more difficult with this class 
> would also be most helpful.

I believe the strongest argument for "path" can be made for how it 
integrates functionality which, although closely related conceptually, 
is currently distributed across a half dozen or more different modules 
in the standard library.  Especially for newbies (I can only imagine, at 
this stage) it would make working with files much easier in a many ways.

"Easier" or "more difficult" is a subjective thing, of course, but one 
can't argue with the fact that path can sometimes do through a single 
object what would otherwise require several imports and a bunch of calls 
into things like open(), os.path, grep, and shutil.

Examples showing effective uses of path that simplify those cases would 
probably merit the label "easier" even in Guido's mind, though 
unfortunately that's not certain.  "Easier" in some minds might simply 
translate to "many lines less code", and while path can sometimes do 
that, aside from the ease of splitting and joining stuff without 
multiple calls to os.path.this-and-that, it really doesn't often reduce 
code size _that_ much, in my experience.  (Postings to c.l.p showing a 
50% reduction in code size for contrived examples notwithstanding.)

A related thoughts: since paths are objects, they have attributes or 
properties, and having things like ".basename" and ".parent" readily 
available without having to do obscure things like 
os.path.split(somepath)[0] makes things much easier to read (therefore 
more maintainable).  In fact, I'd propose that as another strong 
argument in path's favour: it makes code much more readable, even if not 
"easier" to write.

Hmm... does "easier" or "more difficult" apply to the writing of the 
code or the reading of it?  I find it self-evident that code written 
using "path" is much easier to read, not necessarily much easier to 
write (for non-newbies).

I'd summarize this by saying that the integration of "path" in the 
stdlib would make it easier for newbies to write code (that might not be 
obvious to a non-newbie... shall we ask some to help?), and easier for 
everyone to read code (self-evident, no?), and if that's not a 
sufficient condition for inclusion I don't know what is.

-Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

PEP on path module for standard library

[Michael Hoffman]

Many of you are familiar with Jason Orendorff's path module 
, which is frequently 
recommended here on c.l.p. I submitted an RFE to add it to the Python 
standard library, and Reinhold Birkenfeld started a discussion on it in 
python-dev 
.

The upshot of the discussion was that many python-dev'ers wanted path 
added to the stdlib, but Guido was not convinced and said it must have a 
PEP. So Reinhold and I are going to work on one. Reinhold has already 
made some changes to the module to fit the python-dev discussion and put 
it in CPython CVS at nondist/sandbox/path.

For the PEP, do any of you have arguments for or against including path? 
Code samples that are much easier or more difficult with this class 
would also be most helpful.

I use path in more of my modules and scripts than any other third-party 
module, and I know it will be very helpful when I no longer have to 
worry about deploying it.

Thanks in advance,
-- 
Michael Hoffman
-- 
http://mail.python.org/mailman/listinfo/python-list