Re: [Python-Dev] / as path join operator
Jason == Jason Orendorff [EMAIL PROTECTED] writes: Jason You seem to think that because I said operating systems, Jason I'm talking about kernel algorithms and such. I can see how you'd get that impression, but it's not true. My reason for mentioning OS-level filesystem was to show that even in that limited domain, treating paths as strings leads to bugs. Jason I think paths are used more for communication, less for Jason computation. True. For that purpose it is absolutely essential to have a string represention. However, I was discussing the use of / to invoke path composition, which is a computation. Nobody will use that for communication (except to describe a path expression in graph theoretic terms), and I don't think it's a good idea to use that particular symbol for that operation. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of TsukubaTennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can do free software business; ask what your business can do for free software. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] / as path join operator (was: Re: The path module PEP)
Steven Bethard wrote:
My only fear with the / operator is that we'll end up with the same
problems we have for using % in string formatting -- the order of
operations might not be what users expect. Since join is conceptually
an addition-like operator, I would expect:
Path('home') / 'a' * 5
to give me:
home/a
If I understand it right, it would actually give me something like:
home/ahome/ahome/ahome/ahome/a
Is there any very deep magic that says / is the One True Operator
to use for this, given that there's to be an operator for it? For
instance, has lower correct precedence (so that Path('home') 'a'*5
does something less unexpected), and doesn't look quite so much
as if it denotes arithmetic, and avoids semantic interference
from the idea that / should divide things or make them smaller.
(Though, for what it's worth, I think sticking another subdirectory
onto a path *is* dividing and making smaller: think of a path as
representing a subtree.) You do lose the pun on the Unix path separator,
which is a shame.
--
g
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] / as path join operator
Stephen J. Turnbull wrote: Jason Filesystem paths are in fact strings on all operating Jason systems I'm aware of. I have no idea what you could mean by that. The data structure used to represent a filesystem on all OS filesystems I've used is a graph of directories and files. A filesystem object is located by traversing a path in that graph. Of course there's a string representation, especially for human use if you define everything that can be identified by one or more well-defined strings as a string, everything is a string. /F ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] / as path join operator
It's controversial that Path subclasses str. Some people think it's
flat-out wrong. Even Bjorn argues that it's a practicality-vs-purity
tradeoff. But a strong argument can be made that Path *should* be a
string subclass, practicality be damned. Proof follows.
I. Here's an example of the sort of thing you might say if you did
*not* think of paths as strings:
On 1/25/06, Stephen J. Turnbull [EMAIL PROTECTED] wrote:
I think it's logical to expect that
Path('home') / 'and/or'
points to a file named and/or in directory home, not to a file
named or in directory home/and.
This makes no sense whatsoever. Ergo, by reductio ad absurdum, paths
are strings.
II. And here is the sort of thing you'd say if you thought of paths
*solely* as strings:
(2) Note that '/' is also the path separator used by URIs, which RFC
2396 gives different semantics from Unix. Most of my Python usage to
date has been heavily web-oriented, and I'd have little use for /
unless it follows RFC 2396.
The quandary is resolved by pointing out that URIs are not paths (in
the sense of os.path and generally this whole horrible thread). Thus
not all strings are paths.
Hence the paths are a proper subset of the strings. By the existence
of os.path, they have their own commonly-used operations. By
definition, then, Path is a subclass of string, QED.
Do I really buy all this? I dunno. To say paths aren't strings is
all very well, and in a very abstract sense I almost agree--but you
have to admit it sort of flies in the face of, you know, reality.
Filesystem paths are in fact strings on all operating systems I'm
aware of. And it's no accident or performance optimization. It's
good design.
-j
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] / as path join operator
On Fri, Jan 27, 2006 at 06:19:52PM -0500, Jason Orendorff wrote: To say paths aren't strings is all very well, and in a very abstract sense I almost agree--but you have to admit it sort of flies in the face of, you know, reality. Filesystem paths are in fact strings on all operating systems I'm aware of. And it's no accident or performance optimization. It's good design. The question isn't whether Path objects should *act* like strings. I haven't seen anyone argue that they shouldn't, except for a few specific aspects, like iteration, and those are argued on both sides of the subclassing camp. The question is whether they should be actual subclasses of the Python string type. As for what platforms do, if we want to stick to the platform handling of paths, we change nothing. That's apparently not what people want ;) -- Thomas Wouters [EMAIL PROTECTED] Hi! I'm a .signature virus! copy me into your .signature file to help me spread! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] / as path join operator
[Jason Orendorff] Filesystem paths are in fact strings on all operating systems I'm aware of. And it's no accident or performance optimization. It's good design. Isn't that simply because filesystems aren't object orientated? I can't call methods of a path through the filesystem. There's a difference between a path, which is, yes, always (?) a string, and a Path object that provides convenient methods/properties. (Maybe one of the experimental object-orientated file systems has non- string paths. I have no idea). =Tony.Meyer ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] / as path join operator
On 1/27/06, Jason Orendorff [EMAIL PROTECTED] wrote:
On 1/25/06, Stephen J. Turnbull [EMAIL PROTECTED] wrote:
I think it's logical to expect that
Path('home') / 'and/or'
points to a file named and/or in directory home, not to a file
named or in directory home/and.
This makes no sense whatsoever. Ergo, by reductio ad absurdum, paths
are strings.
It makes perfect sense to me. However, since posix doesn't permit '/'
in file names I would expect it to emit an error:
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
However, I'm not sure if the error be emitted when the Path is
created, or when it's passed to open(). The former implies a set of
OS-specific Path classes, and would allow subclassing from str. The
latter allows (but does not require) a single universal Path class,
and that would prohibit subclassing from str (because you need a
higher-level representation to store path segments before converting
them to a platform-specific format.)
I'm -0 on subclassing str in the shortterm and -1 on it in the
longterm. It's cruft and not something we want to be stuck with.
--
Adam Olsen, aka Rhamphoryncus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] / as path join operator
Jason == Jason Orendorff [EMAIL PROTECTED] writes: Jason I. Here's an example of the sort of thing you might say if Jason you did *not* think of paths as strings: [...] Jason II. And here is the sort of thing you'd say if you thought Jason of paths *solely* as strings: Please note that my point was entirely different from trying to decide whether to subclass strings. My point was precisely that because of this schizophrenia in the use of / as a path join operator in various string representations of paths, it's a bad choice. People are naturally going to write buggy code because they don't have the implemented spec in mind. Jason Filesystem paths are in fact strings on all operating Jason systems I'm aware of. I have no idea what you could mean by that. The data structure used to represent a filesystem on all OS filesystems I've used is a graph of directories and files. A filesystem object is located by traversing a path in that graph. Of course there's a string representation, especially for human use, but manipulating that representation as a string in programs is a regular source of bugs. In most cases, the graph is sufficiently constrained that string manipulations are mostly accurate representations of graph traversal, but not always, and you get defects. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of TsukubaTennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can do free software business; ask what your business can do for free software. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] / as path join operator (was: Re: The path module PEP)
I think that everything that can be said aboud __div__() has already been said. But this argument was really convincing: [Tony Meyer] The vast majority of people (at least at the time) were either +0 or -0, not +1. +0's are not justification for including something. There is no clear consensus either way. Ultimately, Guido will decide if he thinks it is clever or not. Meanwhile I'll remove it from the PEP and keep it as an optional extension. Also, like Jason said, the removal of __div__() leads to the ultimate demise of joinpath(), woho! [Jason Orendorff] in which case I propose using the Path constructor as the replacement for os.path.join(). In that case, Path.joinpath can be dropped. Path.cwd() / foobar == Path(Path.cwd(), foobar) Path(foo) / bar / baz == Path(foo, bar, baz) Still, in the simpler cases, __div__() looks really handy: os.chdir(pkgdir / include) == os.chdir(Path(pkgdir, include)) Oh well. You can't have everything, can you? The updated PEP and an implementation is availible from http://wiki.python.org/moin/PathClass. -- mvh Björn ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] / as path join operator (was: Re: The path module PEP)
[John J Lee] But it's a very readable way to write a common operation. Perhaps one reason the discrepancy you point out doesn't bother me is that division is the least-used of the +-*/ arithmetic operations. Do you have evidence to back that up? It seems a strange claim. Outside of doing 'maths-y' work, I would think I'd use + most (but for strings), then / (for percentages). Also, , | and ^ seem like some sort of precedent, to my brain (if they don't to yours, that's fine and I believe you ;-). I don't follow this, sorry. You're referring to the bitwise operations? [Ian Bicking] Curious how often I use os.path.join and division, I searched a project of mine, and in 12k lines there were 34 uses of join, and 1 use of division. In smaller scripts os.path.join tends to show up a lot more (per line). I'm sure there's people who use division far more than I, and os.path.join less, but I'm guessing the majority of users are more like me. The problem with these sorts of guesses is that there's no evidence. (Maybe the suggestion that Brett's PhD should collect a corpus of Python scripts was a good one wink). Are mathematicians that under represented? Is file processing that highly represented? I have no idea. =Tony.Meyer ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] / as path join operator (was: Re: The path module PEP)
[John J Lee] But it's a very readable way to write a common operation. Perhaps one reason the discrepancy you point out doesn't bother me is that division is the least-used of the +-*/ arithmetic operations. [Tony Meyer] Do you have evidence to back that up? No. :) [Ian Bicking] of mine, and in 12k lines there were 34 uses of join, and 1 use of division. In smaller scripts os.path.join tends to show up a lot more [Tony] The problem with these sorts of guesses is that there's no evidence. (Maybe the suggestion that Brett's PhD should collect a corpus of Python scripts was a good one wink). Are mathematicians that under represented? Is file processing that highly represented? I have no idea. A second data point: I looked at ~10k lines of physical data analysis code I have lying around -- presumably a relatively rare and extreme example as the Python-world in general goes. Result: 140 occurences of os.path.join 170 physical lines (as opposed to syntactical lines) containing / as a division operator (very few lines contained 1 use of '/', so you can multiply 170 by 1.25 to get an upper bound of 213 uses in total) (To get the second number, I used find and grep heavily but very cautiously, and final manual count of stubborn lines of grep output with no use of '/' as division operator) The fact that even in this extreme case os.path.join is close on the tail of '/' strongly backs up Ian's guess that, in most Python code, / as division is rare compared to path joining. Should we deprecate use of '/' and '//' for division in Python 3.0? is-he-joking?-ly y'rs John ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] / as path join operator
Steven Bethard wrote:
John J Lee wrote:
On Thu, 26 Jan 2006, Tony Meyer wrote:
[...]
Well, if you include the much larger discussion on python-list,
people (including me) have said that removing __div__ is a good
idea. If it's included in the PEP, please at least include a
justification and cover the problems with it. The vast majority of
people (at least at the time) were either +0 or -0, not +1. +0's are
not justification for including something.
bikeshed
FWLIW, I'm definitely +1 on using / as a path join operator.
My only fear with the / operator is that we'll end up with the same
problems we have for using % in string formatting -- the order of
operations might not be what users expect. Since join is conceptually
an addition-like operator, I would expect:
Path('home') / 'a' * 5
to give me:
home/a
If I understand it right, it would actually give me something like:
home/ahome/ahome/ahome/ahome/a
I don't want to claim this is the most common use case, but I've
certainly seen auto-generated paths that look like 'a' * 20, and it
would be a pity if using the / operator for Path objects did the wrong
thing by default here...
What if we used subpath as the name instead of joinpath?
The main appeal to me of the division operation is that it allows multiple
path elements to be joined on a single line, but the joining method accepts an
arbitrary number of arguments, which helps with that just as much, and doesn't
raise precedence and readability questions.
The above example would be:
Path('home').subpath('a'*5)
An example of retrieving a config file's full name:
Current: os.path.join(HOME_DIR, APP_DIR, CONFIG_FILE)
Division: HOME_DIR / APP_DIR / CONFIG_FILE
Subpath: HOME_DIR.subpath(APP_DIR, CONFIG_FILE)
Cheers,
Nick.
--
Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia
---
http://www.boredomandlaziness.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] / as path join operator
[Nick Coghlan wrote] What if we used subpath as the name instead of joinpath? append? not-a-big-fan-of-joinpath-either-ly yours, Trent -- Trent Mick [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] / as path join operator
Steven Bethard wrote:
My only fear with the / operator is that we'll end up with the same
problems we have for using % in string formatting -- the order of
operations might not be what users expect. Since join is conceptually
an addition-like operator, I would expect:
Path('home') / 'a' * 5
to give me:
home/a
If I understand it right, it would actually give me something like:
home/ahome/ahome/ahome/ahome/a
Both of these examples are rather silly, of course ;) There's two
operators currently used commonly with strings (that I assume Path would
inherit): + and %. Both actually make sense with paths too.
filename_template = '%(USER)s.conf'
p = Path('/conf') / filename_template % os.environ
which means:
p = (Path('/conf') / filename_template) % os.environ
But probably the opposite is intended. Still, it will usually be
harmless. Which is sometimes worse than usually harmful.
+ seems completely innocuous, though:
ext = '.jpg'
name = fields['name']
image = Path('/images') / name + ext
It doesn't really matter what order it happens in there. Assuming
concatenation results in a new Path object, not a str.
--
Ian Bicking | [EMAIL PROTECTED] | http://blog.ianbicking.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] / as path join operator (was: Re: The path module PEP)
On Thu, 2006-01-26 at 12:51 +1300, Tony Meyer wrote:
[John J Lee]
But it's a very readable way to write a common operation. Perhaps one
reason the discrepancy you point out doesn't bother me is that
division is
the least-used of the +-*/ arithmetic operations.
Do you have evidence to back that up? It seems a strange claim.
Outside of doing 'maths-y' work, I would think I'd use + most (but
for strings), then / (for percentages).
I haven't followed the entire thread (I'll try to find time to catch up)
but while I think using __div__ to mean path concatenation is cute, I'm
not sure I'd like to see it all over the place. It does seem awfully
FAST (facinating and stomach turning to use a term from years ago).
What I don't like about os.path.join() having to import os and having to
type all those characters over and over again. What I /like/ about
os.path.join is that you can give it a bunch of path components and have
it return the correctly joined path, e.g. os.path.join('a, 'b', 'c').
That seems more efficient than having to create a bunch of intermediate
objects.
All in all, I'd have to say I'm -0 on __div__ for path concatenation.
-Barry
signature.asc
Description: This is a digitally signed message part
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] / as path join operator
Steven == Steven Bethard [EMAIL PROTECTED] writes:
Steven My only fear with the / operator is that we'll end up with
Steven the same problems we have for using % in string formatting
Steven -- the order of operations might not be what users expect.
Besides STeVe's example, (1) I think it's logical to expect that
Path('home') / 'and/or'
points to a file named and/or in directory home, not to a file
named or in directory home/and.
(2) Note that '/' is also the path separator used by URIs, which RFC
2396 gives different semantics from Unix. Most of my Python usage to
date has been heavily web-oriented, and I'd have little use for /
unless it follows RFC 2396. By that, I mean that I would want the /
operator to treat its rhs as a relative reference, so the result would
be computed by the algorithm in section 5.2 of that RFC. But this is
not correct (realpath) semantics on Unix.
--
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of TsukubaTennodai 1-1-1 Tsukuba 305-8573 JAPAN
Ask not how you can do free software business;
ask what your business can do for free software.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] / as path join operator
On Thu, 2006-01-26 at 11:40 +1000, Nick Coghlan wrote: The main appeal to me of the division operation is that it allows multiple path elements to be joined on a single line, but the joining method accepts an arbitrary number of arguments, which helps with that just as much, and doesn't raise precedence and readability questions. Or the need to employ ugliness like backslashes when you have to split the join across multiple lines. -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] / as path join operator
On Wed, 2006-01-25 at 21:02 -0600, Ian Bicking wrote:
ext = '.jpg'
name = fields['name']
image = Path('/images') / name + ext
Here's a good example of why I ultimately don't like __div__. The last
line seems quite non-obvious to me. It's actually jarring enough that I
have to stop and think about what it means because it /looks/ like
there's math going on.
OTOH, something like:
image = Path('', 'images', name) + ext
or even better
image = Path.join('', 'images', name) + ext
where .join is a staticmethod, seems much clearer.
-Barry
signature.asc
Description: This is a digitally signed message part
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
