> We should mimic os.stat() and os.stat_result: os.stat_result symbol
> exists in the os namespace, but the type constructor is not
> documented. No need for extra protection like not adding the type in
> the os module, or adding a "_" prefix to the name.
Yeah, that works for me.
> By the way, it
2014-07-21 18:48 GMT+02:00 Ben Hoyt :
>> By the way, DirEntry constructor is not documented in the PEP. Should
>> we document it? It might be a way to "invalidate the cache":
>
> I would prefer not to, just to keep things simple. Similar to creating
> os.stat_result() objects ... you can kind of do
On 22 Jul 2014 02:46, "Steve Dower" wrote:
>
> Personally I'd make it a string subclass and put one-shot properties on
it (i.e. call/cache stat() on first access where we don't already know the
answer), which I think is close enough to where it's landed that I'm happy.
(As far as bikeshedding goes
Thanks for an initial look into this, Victor.
> IMO the current os.scandir() API does not fit importlib requirements.
> importlib usually wants fresh data, whereas DirEntry cache cannot be
> invalidated. It's probably possible to cache some os.stat() result in
> importlib, but it looks like it req
Victor Stinner wrote:
> 2014-07-20 18:50 GMT+02:00 Antoine Pitrou :
>> Have you tried modifying importlib's _bootstrap.py to use scandir()
>> instead of listdir() + stat()?
>
> IMO the current os.scandir() API does not fit importlib requirements.
> importlib usually wants fresh data, whereas DirEn
Hi,
2014-07-20 18:50 GMT+02:00 Antoine Pitrou :
> Have you tried modifying importlib's _bootstrap.py to use scandir() instead
> of listdir() + stat()?
IMO the current os.scandir() API does not fit importlib requirements.
importlib usually wants fresh data, whereas DirEntry cache cannot be
invalid
> Even though there is tangible performance improvement from scandir(), it
> would be useful to find out if the API fits well.
Got it -- I see where you're coming from now. I'll take a quick look
(hopefully later this week).
-Ben
___
Python-Dev mailing
Le 20/07/2014 17:34, Ben Hoyt a écrit :
Have you tried modifying importlib's _bootstrap.py to use scandir() instead
of listdir() + stat()?
No, I haven't -- I'm not familiar with that code. What does
_bootstrap.py do -- does it do a lot of listdir calls and stat-ing of
many files?
Quite a bit,
Oh yes. :) The file Antoine is referring to is the implementation of import.
On Sun, Jul 20, 2014, 17:34 Ben Hoyt wrote:
> > Have you tried modifying importlib's _bootstrap.py to use scandir()
> instead
> > of listdir() + stat()?
>
> No, I haven't -- I'm not familiar with that code. What does
>
> Have you tried modifying importlib's _bootstrap.py to use scandir() instead
> of listdir() + stat()?
No, I haven't -- I'm not familiar with that code. What does
_bootstrap.py do -- does it do a lot of listdir calls and stat-ing of
many files?
-Ben
___
Hi,
> Thanks Victor, Nick, Ethan, and others for continued discussion on the
scandir PEP 471 (most recent thread starts at
https://mail.python.org/pipermail/python-dev/2014-July/135377.html).
Have you tried modifying importlib's _bootstrap.py to use scandir()
instead of listdir() + stat()?
I was going to stay out of this one...
On 14Jul2014 10:25, Victor Stinner wrote:
2014-07-14 4:17 GMT+02:00 Nick Coghlan :
Or the ever popular symlink to "." (or a directory higher in the tree).
"." and ".." are explicitly ignored by os.listdir() an os.scandir().
I think os.walk() is a good
On 07/14/2014 11:25 PM, Victor Stinner wrote:
Again: remove any garantee about the cache in the definitions of methods,
instead copy the doc from os.path and os. Add a global remark saying that
most methods don't need any syscall in general, except for symlinks (with
follow_symlinks=True).
I
On 15 July 2014 13:19, Ben Hoyt wrote:
> Hmmm, perhaps. You suggest .full_name implies it's the absolute path,
> which isn't true. I don't mind .path, but it kind of sounds like "the
> Path object associated with this entry". I think "full_name" is fine
> -- it's not "abs_name".
Interesting. I ha
> I'd *keep DirEntry.lstat() method* regardless of existence of
> .stat(*, follow_symlinks=True) method (despite the slight violation of
> DRY principle) for readability. `dir_entry.lstat().st_mode` is more
> consice than `dir_entry.stat(follow_symlinks=False).st_mode` and the
> meaning of lstat is
> Sorry, I don't remember who but someone proposed to add the follow_symlinks
> parameter in scandir() directly. If the parameter is added to methods,
> there is no such issue.
Yeah, I think having the DirEntry methods do different things
depending on how scandir() was called is a really bad idea
> Looks doable. Just make sure the cached entries reflect the
> 'follow_symlinks' setting -- so a symlink could end up with both an lstat
> cached entry and a stat cached entry.
Yes, good point -- basically the functions will use the _stat cache if
follow_symlinks=True, otherwise the _lstat cache
On 14 Jul 2014 22:50, "Ben Hoyt" wrote:
>
> In light of that, I propose I update the PEP to basically follow
> Victor's model of is_X() and stat() following symlinks by default, and
> allowing you to specify follow_symlinks=False if you want something
> other than that.
>
> Victor had one other qu
Le mardi 15 juillet 2014, Ben Hoyt a écrit :
>
>
> Victor had one other question:
>
> > What happens to name and full_name with followlinks=True?
> > Do they contain the name in the directory (name of the symlink)
> > or name of the linked file?
>
> I would say they should contain the name and ful
On 07/14/2014 07:48 PM, Ben Hoyt wrote:
In any case, here's the modified proposal:
scandir(path='.') -> generator of DirEntry objects, which have:
* name: name as per listdir()
* full_name: full path name (not necessarily absolute), equivalent of
os.path.join(path, entry.name)
* is_dir(follow_
> Let's not multiply entities beyond necessity.
>
> There is well-defined *follow_symlinks* parameter
> https://docs.python.org/3/library/os.html#follow-symlinks
> e.g., os.access, os.chown, os.link, os.stat, os.utime and many other
> functions in os module support follow_symlinks parameter, see
>
Nick Coghlan writes:
> On 13 Jul 2014 20:54, "Tim Delaney" wrote:
>>
>> On 14 July 2014 10:33, Ben Hoyt wrote:
>>>
>>>
>>>
>>> If we go with Victor's link-following .is_dir() and .is_file(), then
>>> we probably need to add his suggestion of a follow_symlinks=False
>>> parameter (defaults to Tr
First, just to clarify a couple of points.
> You forgot one of my argument: we must have exactly the same API than
> os.path.is_dir() and pathlib.Path.is_dir(), because it would be very
> confusing (source of bugs) to have a different behaviour.
Actually, I specifically included that argument. It
2014-07-14 6:52 GMT+02:00 Ethan Furman :
> We shoIf you put the option on scandir(), you uld have a flag for that, and
> default it to False:
>
> scandir(path, *, followlinks=False, info=None, onerror=None)
What happens to name and full_name with followlinks=True? Do they
contain the name in th
2014-07-14 4:17 GMT+02:00 Nick Coghlan :
> Or the ever popular symlink to "." (or a directory higher in the tree).
"." and ".." are explicitly ignored by os.listdir() an os.scandir().
> I think os.walk() is a good source of inspiration here: call the flag
> "followlink" and default it to False.
2014-07-14 2:33 GMT+02:00 Ben Hoyt :
> If we go with Victor's link-following .is_dir() and .is_file(), then
> we probably need to add his suggestion of a follow_symlinks=False
> parameter (defaults to True). Either that or you have to say
> "stat.S_ISDIR(entry.lstat().st_mode)" instead, which is a
On 07/13/2014 05:33 PM, Ben Hoyt wrote:
On the recent python-dev thread, Victor especially made some well
thought out suggestions. It seems to me there's general agreement that
the basic API in PEP 471 is good (with Ethan not a fan at first, but
it seems he's on board after further discussion :-
On 14 July 2014 12:17, Nick Coghlan wrote:
>
> I think os.walk() is a good source of inspiration here: call the flag
> "followlink" and default it to False.
>
Actually, that's "followlinks", and I'd forgotten that os.walk() defaulted
to not follow - definitely behaviour to match IMO :)
Tim Delane
On 13 Jul 2014 20:54, "Tim Delaney" wrote:
>
> On 14 July 2014 10:33, Ben Hoyt wrote:
>>
>>
>>
>> If we go with Victor's link-following .is_dir() and .is_file(), then
>> we probably need to add his suggestion of a follow_symlinks=False
>> parameter (defaults to True). Either that or you have to s
On 14 July 2014 10:33, Ben Hoyt wrote:
>
If we go with Victor's link-following .is_dir() and .is_file(), then
> we probably need to add his suggestion of a follow_symlinks=False
> parameter (defaults to True). Either that or you have to say
> "stat.S_ISDIR(entry.lstat().st_mode)" instead, which
Hi folks,
Thanks Victor, Nick, Ethan, and others for continued discussion on the
scandir PEP 471 (most recent thread starts at
https://mail.python.org/pipermail/python-dev/2014-July/135377.html).
Just an aside ... I was reminded again recently why scandir() matters:
a scandir user emailed me the
31 matches
Mail list logo
