On 9 Jul 2014 17:14, "Ethan Furman" wrote:
>
> On 07/09/2014 02:42 PM, Ben Hoyt wrote:
>>>
>>>
>>> Okay, so using that [no platform specific] logic we should head over to
the os module and remove:
>>>
>>>
>>> ctermid, getenv, getegid...
>>>
>>> Okay, I'm tired of typing, but that list is not even
Ben Hoyt writes:
...
> ``scandir()`` yields a ``DirEntry`` object for each file and directory
> in ``path``. Just like ``listdir``, the ``'.'`` and ``'..'``
> pseudo-directories are skipped, and the entries are yielded in
> system-dependent order. Each ``DirEntry`` object has the following
> attri
On 10 July 2014 10:23, Victor Stinner wrote:
> 2014-07-09 17:26 GMT+02:00 Paul Moore :
> > On 9 July 2014 16:05, Victor Stinner wrote:
> >> The PEP says that DirEntry should mimic pathlib.Path, so I think that
> >> DirEntry.is_dir() should work as os.path.isir(): if the entry is a
> >> symbolic
Oh, since I'm proposing to add a new stat() method to DirEntry, we can
optimize it. stat() can reuse lstat() result if the file is not a
symlink. It simplifies is_dir().
New pseudo-code:
---
class DirEntry:
def __init__(self, path, name, lstat=None, d_type=None):
self.name = name
On 07/09/2014 05:15 PM, Victor Stinner wrote:
2014-07-09 17:29 GMT+02:00 Ben Hoyt :
Would this not "break" the tree size script being discussed in the
other thread, as it would follow links and include linked directories
in the "size" of the tree?
The get_tree_size() function in the PEP would
Ben Hoyt writes:
> So here's the ways in which option #2 is now more complicated than option #1:
>
> 1) it has an additional "info" argument, the values of which have to
> be documented ('os', 'type', 'lstat', and what each one means)
> 2) it has an additional "onerror" argument, the signature of
2014-07-09 17:26 GMT+02:00 Paul Moore :
> On 9 July 2014 16:05, Victor Stinner wrote:
>> The PEP says that DirEntry should mimic pathlib.Path, so I think that
>> DirEntry.is_dir() should work as os.path.isir(): if the entry is a
>> symbolic link, you should follow the symlink to get the status of
2014-07-09 17:29 GMT+02:00 Ben Hoyt :
>> Would this not "break" the tree size script being discussed in the
>> other thread, as it would follow links and include linked directories
>> in the "size" of the tree?
The get_tree_size() function in the PEP would use: "if not
entry.is_symlink() and entry
On 07/09/2014 04:22 PM, MRAB wrote:
On 2014-07-09 23:50, Ethan Furman wrote:
Okay, marry the two ideas together:
scandir(path, info=None, onerror=None)
"""
Return a generator that returns one directory entry at a time in a
DirEntry object
Should that be "that yields one
On 2014-07-09 23:50, Ethan Furman wrote:
On 07/09/2014 02:33 PM, Ben Hoyt wrote:
On a system which did not supply is_dir automatically I would write that as:
for entry in os.scandir(path):
if ignore_entry(entry.name):
continue
if os.path.isdir(entry.full_name):
On 07/09/2014 02:33 PM, Ben Hoyt wrote:
On a system which did not supply is_dir automatically I would write that as:
for entry in os.scandir(path):
if ignore_entry(entry.name):
continue
if os.path.isdir(entry.full_name):
# do something interesting
Not har
On 07/09/2014 02:38 PM, Victor Stinner wrote:
2014-07-09 22:44 GMT+02:00 Ethan Furman:
On 07/09/2014 01:24 PM, Victor Stinner wrote:
[...] Python must remain
portable and you should not write code specific to one specific
platform.
Okay, so using that logic we should head over to the os mod
On 07/09/2014 02:42 PM, Ben Hoyt wrote:
Okay, so using that [no platform specific] logic we should head over to the os
module and remove:
ctermid, getenv, getegid...
Okay, I'm tired of typing, but that list is not even half-way through the os
page, and those are all methods or attributes that
2014-07-09 22:44 GMT+02:00 Ethan Furman :
> On 07/09/2014 01:24 PM, Victor Stinner wrote:
>> Sorry, I didn't follow the whole discussion. IMO DirEntry must use
>> methods and you should not expose nor document which infos are already
>> provided by the OS or not. DirEntry should be a best-effort bl
I really don't understand why you *want* a worse, much less cross-platform API?
> Okay, so using that logic we should head over to the os module and remove:
>
> ctermid, getenv, getegid...
>
> Okay, I'm tired of typing, but that list is not even half-way through the os
> page, and those are all me
> On a system which did not supply is_dir automatically I would write that as:
>
> for entry in os.scandir(path): # info defaults to 'os', which is
> basically None in this case
> if ignore_entry(entry.name):
> continue
> if os.path.isdir(entry.full_name):
> # do
On 07/09/2014 01:24 PM, Victor Stinner wrote:
Sorry, I didn't follow the whole discussion. IMO DirEntry must use
methods and you should not expose nor document which infos are already
provided by the OS or not. DirEntry should be a best-effort black-box
object providing an API similar to pathlib
On 07/09/2014 01:57 PM, Paul Moore wrote:
On 9 July 2014 21:24, Victor Stinner wrote:
Example where you may sometimes need is_dir(), but not always
---
for entry in os.scandir(path):
if ignore_entry(entry.name):
# this entry is not interesting, lstat_result is useless here
contin
On 9 July 2014 21:24, Victor Stinner wrote:
> Example where you may sometimes need is_dir(), but not always
> ---
> for entry in os.scandir(path):
> if ignore_entry(entry.name):
> # this entry is not interesting, lstat_result is useless here
> continue
> if entry.is_dir(): # fetch r
2014-07-09 21:59 GMT+02:00 Ben Hoyt :
> Other python-devers, please chime in with your thoughts or votes.
Sorry, I didn't follow the whole discussion. IMO DirEntry must use
methods and you should not expose nor document which infos are already
provided by the OS or not. DirEntry should be a best-e
>> 1) it has an additional "info" argument, the values of which have to
>> be documented ('os', 'type', 'lstat', and what each one means)
>> 2) it has an additional "onerror" argument, the signature of which and
>> fairly complicated return value is non-obvious and has to be
>> documented
>> 3) it
On 07/09/2014 12:03 PM, Ben Hoyt wrote:
So here's the ways in which option #2 is now more complicated than option #1:
1) it has an additional "info" argument, the values of which have to
be documented ('os', 'type', 'lstat', and what each one means)
2) it has an additional "onerror" argument, t
This is just getting way too complex ... further thoughts below.
>> This is an interesting idea, but it's just getting more and more
>> complex, and I'm guessing that being able to change the attributes of
>> DirEntry will make the C implementation more complex.
>>
>> Also, I'm not sure it's very
On 07/09/2014 11:04 AM, Paul Moore wrote:
On 9 July 2014 17:35, Ethan Furman wrote:
More specifically, if we go with choice 1 (no built-in error handling, no
mutable DirEntry), how would I implement choice 2? Would I have to write my
own CustomDirEntry object?
Having built-in error handling
On 07/09/2014 10:10 AM, Paul Moore wrote:
On 9 July 2014 14:22, Ben Hoyt wrote:
One issue with option #2 that I just realized -- does scandir yield
the entry at all if there's a stat error? It can't really, because the
caller will except the .lstat attribute to be set (assuming he asked
for typ
On 9 July 2014 17:35, Ethan Furman wrote:
> More specifically, if we go with choice 1 (no built-in error handling, no
> mutable DirEntry), how would I implement choice 2? Would I have to write my
> own CustomDirEntry object?
Having built-in error handling is, I think, a key point. That's where
#
On 07/09/2014 08:35 AM, Ben Hoyt wrote:
One issue with option #2 that I just realized -- does scandir yield the
entry at all if there's a stat error? It
can't really, because the caller will expect the .lstat attribute to be
set (assuming he asked for type='lstat') but
it won't be. Is effectivel
On 9 July 2014 14:22, Ben Hoyt wrote:
> One issue with option #2 that I just realized -- does scandir yield
> the entry at all if there's a stat error? It can't really, because the
> caller will except the .lstat attribute to be set (assuming he asked
> for type='lstat') but it won't be. Is effect
>> One issue with option #2 that I just realized -- does scandir yield the
>> entry at all if there's a stat error? It
>> can't really, because the caller will expect the .lstat attribute to be
>> set (assuming he asked for type='lstat') but
>>
>> it won't be. Is effectively removing these entries
>> The PEP says that DirEntry should mimic pathlib.Path, so I think that
>> DirEntry.is_dir() should work as os.path.isir(): if the entry is a
>> symbolic link, you should follow the symlink to get the status of the
>> linked file with os.stat().
>
> Would this not "break" the tree size script bein
On 9 July 2014 16:05, Victor Stinner wrote:
> The PEP says that DirEntry should mimic pathlib.Path, so I think that
> DirEntry.is_dir() should work as os.path.isir(): if the entry is a
> symbolic link, you should follow the symlink to get the status of the
> linked file with os.stat().
Would this
2014-07-09 15:12 GMT+02:00 Ben Hoyt :
>> Ok, so it means that your example grouping files per type, files and
>> directories, is also wrong. Or at least, it behaves differently than
>> os.walk(). You should put symbolic links to directories in the "dirs"
>> list too.
>>
>> if entry.is_dir(): # is
On 07/09/2014 06:41 AM, Ethan Furman wrote:
Leave it up to the onerror handler. If it returns None, skip yielding the
entry, otherwise yield whatever it returned
-- which also means the error handler should be able to set fields on the
DirEntry:
def log_err(exc, entry):
logger.warn
On 8 Jul 2014, at 15:52, Ben Hoyt wrote:
Hi folks,
After some very good python-dev feedback on my first version of PEP
471, I've updated the PEP to clarify a few things and added various
"Rejected ideas" subsections. Here's a link to the new version (I've
also copied the full text below):
http
On 07/09/2014 06:22 AM, Ben Hoyt wrote:
One issue with option #2 that I just realized -- does scandir yield the entry
at all if there's a stat error? It
can't really, because the caller will expect the .lstat attribute to be set
(assuming he asked for type='lstat') but
it won't be. Is effectiv
On 07/09/2014 05:48 AM, Ben Hoyt wrote:
So how about tweaking option #2 a tiny bit more to this:
def scandir(path='.', info=None, onerror=None): ...
* if info is None (the default), only the .name and .full_name
attributes are present
* if info is 'type', scandir ensures the is_dir/is_file/is_
On 9 July 2014 14:22, Ben Hoyt wrote:
>> So maybe the onerror function should also receive the DirEntry object
>> - which will only have the name and full_name attributes, but that's
>> all that is needed.
>
> That's an interesting idea -- though enough of a deviation from
> os.walk()'s onerror th
> Option 2:
> def log_err(exc):
> logger.warn("Cannot stat {}".format(exc.filename))
>
> def get_tree_size(path):
> total = 0
> for entry in os.scandir(path, info='lstat', onerror=log_err):
> if entry.is_dir:
> total += get_tree_size(entry.full_name)
> else:
Le 09/07/2014 00:21, Stephen J. Turnbull a écrit :
Steven D'Aprano writes:
> I don't think so. Floating point == represents *numeric* equality,
There is no such thing as floating point == in Python. You can apply
== to two floating point numbers, but == (at the language level)
handles any tw
On 9 July 2014 13:48, Ben Hoyt wrote:
> Okay folks -- please respond: option #1 as per the current PEP 471, or
> option #2 with Ethan's multi-level thing tweaks as per the above?
I'm probably about 50/50 at the moment. What will swing it for me is
likely error handling, so let's try both approach
> Ok, so it means that your example grouping files per type, files and
> directories, is also wrong. Or at least, it behaves differently than
> os.walk(). You should put symbolic links to directories in the "dirs"
> list too.
>
> if entry.is_dir(): # is_dir() checks os.lstat()
> dirs.append(e
2014-07-08 22:09 GMT+02:00 Ben Hoyt :
>>> I think you're misunderstanding is_dir() and is_file(), as these don't
>>> actually call os.stat(). All DirEntry methods either call nothing or
>>> os.lstat() to get the stat info on the entry itself (not the
>>> destination of the symlink).
>>
>>
>> Oh. Ex
> In this case because the names are exactly the same as the os versions which
> /do/ make a system call.
Fair enough.
> So if I'm finally understanding the root problem here:
>
> - listdir returns a list of strings, one for each filename and one for
> each directory, and keeps no other O/S
On 9 July 2014 02:08, Ben Hoyt wrote:
> Comments and votes, please!
+1 on option 1 (current PEP approach) at the moment, but I would like
to see how the error handling would look (suppose the function logs
files that can't be statted, and assumes a size of 0 for them). The
idea of a multi-level e
44 matches
Mail list logo
