Re: [Python-Dev] Suggested addition to PEP 8 for context managers

[Matej Cepl]

On 16.4.2012 18:10, Nam Nguyen wrote:

a_list[pos + 1 : -1]


or other way around

a_list[pos+1:-1]

?

Matěj   
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Suggested addition to PEP 8 for context managers

[Paul Moore]

On 16 April 2012 17:10, Nam Nguyen  wrote:
> PEP 8 suggests no extra spaces after and before square brackets, and
> colons. So code like this is appropriate:
>
> a_list[1:3]
>
> But I find it less readable in the case of:
>
> a_list[pos + 1:-1]
>
> The colon is seemingly lost in the right.
>
> Would it be better to read like below?
>
> a_list[pos + 1 : -1]
>
> Any opinion?

It says no space *before* a colon, not after. So the following should
be OK (and is what I'd use):

a_list[pos + 1: -1]

Paul.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [RFC] PEP 418: Add monotonic time, performance counter and process time functions

[Sümer Cip]

On Sun, Apr 15, 2012 at 6:18 PM, Victor Stinner wrote:

> > Here is a simplified version of the first draft of the PEP 418. The
> > full version can be read online.
> > http://www.python.org/dev/peps/pep-0418/
>
> FYI there is no time.thread_time() function. It would only be
> available on Windows and Linux. It does not use seconds but CPU
> cycles. No module or program of the Python source code need such
> function,


Just FYI: in MACOSx, you can use  thread_info() to get that information.
Also you can get that information in Solaris,too. In yappi profiler I use
all of these approaches together to have an OS independent thread_times()
functionality. Here is the relevant code:
http://bitbucket.org/sumerc/yappi/src/7c7dc11e8728/timing.c

I also think that you are right about Python really not have any use case
for this functionality, ...


-- 
Sümer Cip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

[Antoine Pitrou]

On Mon, 16 Apr 2012 20:41:56 -0400
Brett Cannon  wrote:
> On Mon, Apr 16, 2012 at 20:27, Antoine Pitrou  wrote:
> 
> > On Tue, 17 Apr 2012 01:11:14 +0200
> > Georg Brandl  wrote:
> > >
> > > No, it's not just an existing Python, it is (at least currently) the same
> > > version of Python being built.  Therefore I wrote about the bootstrapping
> > > problems when bytecode changes.
> > >
> > > Depending on Cython is better in that it breaks the bootstrapping cycle,
> > > but on the other hand the C code may need to be regenerated when the C
> > API
> > > changes in an incompatible way.
> >
> > Cython OTOH probably needs Python 2.x, which isn't that great for
> > building Python 3. And requiring Cython for developing is not very
> > contributor-friendly.
> >
> 
> Well, required to regenerate _frozen_importlib, but nothing else. I mean
> making fixes go into importlib directly and get tested that way, not
> through __import__(). So really Cython would only be needed when
> importlib._bootstrap has been changed and you are making a commit.

That's still a large dependency to bring in, while we already have a
working solution.
I'd understand using Cython to develop some new extension module which
requires linking against a C library (and is thus impossible to write
in pure Python). But for importlib that's totally non-necessary.

I guess I'm -1 on it.

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Issue #13959: Re-implement imp.load_source() in imp.py.

[Antoine Pitrou]

On Tue, 17 Apr 2012 04:11:31 +0200
brett.cannon  wrote:
> http://hg.python.org/cpython/rev/3b5b4b4bb43c
> changeset:   76371:3b5b4b4bb43c
> user:Brett Cannon 
> date:Mon Apr 16 22:11:25 2012 -0400
> summary:
>   Issue #13959: Re-implement imp.load_source() in imp.py.
> 
> files:
>   Lib/imp.py  |   29 ++-
>   Python/import.c |  390 
>   2 files changed, 28 insertions(+), 391 deletions(-)

It's nice to see all that C code go away :-)

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

[Eric V. Smith]

On 4/17/2012 5:52 AM, Antoine Pitrou wrote:
> On Mon, 16 Apr 2012 20:41:56 -0400
> Brett Cannon  wrote:
>> On Mon, Apr 16, 2012 at 20:27, Antoine Pitrou  wrote:
>>
>>> On Tue, 17 Apr 2012 01:11:14 +0200
>>> Georg Brandl  wrote:

 No, it's not just an existing Python, it is (at least currently) the same
 version of Python being built.  Therefore I wrote about the bootstrapping
 problems when bytecode changes.

 Depending on Cython is better in that it breaks the bootstrapping cycle,
 but on the other hand the C code may need to be regenerated when the C
>>> API
 changes in an incompatible way.
>>>
>>> Cython OTOH probably needs Python 2.x, which isn't that great for
>>> building Python 3. And requiring Cython for developing is not very
>>> contributor-friendly.
>>>
>>
>> Well, required to regenerate _frozen_importlib, but nothing else. I mean
>> making fixes go into importlib directly and get tested that way, not
>> through __import__(). So really Cython would only be needed when
>> importlib._bootstrap has been changed and you are making a commit.
> 
> That's still a large dependency to bring in, while we already have a
> working solution.
> I'd understand using Cython to develop some new extension module which
> requires linking against a C library (and is thus impossible to write
> in pure Python). But for importlib that's totally non-necessary.
> 
> I guess I'm -1 on it.

I agree. If the problem we're trying to solve is that the generated file
isn't always rebuilt, bringing in a large dependency like Cython seems
like overkill to me.

We basically have a working solution now (thanks, Brett). I think we
should focus on getting it polished. Maybe we can bring in Cython in a
later release, if in the 3.4 timeframe it still seems like we have a
problem to solve. I suspect things will be working fine.

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] issue 9141, finalizers and gc module

[Kristján Valur Jónsson]

Hello there.
For those familiar with the intricacies of the gcmodule.c, I would like to draw 
your attention to http://bugs.python.org/issue9141.

I would like to consult with you to find out more about finalizers/gc in order 
to improve the in-file documentation.

Traditionally, it has not been possible to collect objects that have __del__ 
methods, or more generally, finalizers.  Instead, they and any objects that are 
reachable from them, are put in gc.garbage.

What I want to know is, why is this limitation in place?  Here are two 
possibilities:

1)  "The order of calling finalizers in a cycle is undefined so it is not a 
solvable problem".  But this would allow a single object, with only internal 
cycles to be collected.  Currently this is not the case.

2)  "During collection, the interpreter is in a fragile state (linked lists 
of gc objects with refcount bookkeeping in place) and no unknown code can be 
allowed to run".  This is the reason I personally think is the true reason.

The reason I'm asking is that python has traditionally tested for finalizers by 
checking the tp_del slot of the object's type.  This will be true if the object 
has a __del__ method.  Since generators were added, they use the tp_del slot 
for their own finalizers, but special code was put in place so that the 
generators could tell if the finalizer were "trivial" or not (trivial meaning 
"just doing Py_DECREF()).
This allowed generators to be coollected too, if they were in a common, trivial 
state, but otherwise, they had to be put in gc.garbage().

Yesterday, I stumbled upon the fact that tp_dealloc of iobase objects also 
calls an internal finalizer, one that isn't exposed in any tp_del slot:  It 
will invoke a PyObject_CallMethod(self, "close", "") on itself.  This will 
happen whenever iobase objects are part of a cycle that needs to be cleared.  
This can cause arbitrary code to run.  There are even provisions made for the 
resurrection of the iobase objects based on the action of this close() call.

Clearly, this has the potential to be non-trivial, and therefore, again, I see 
this as an argument for my proposed patched in issue 9141.  But others have 
voiced worries that if we stop collecting iobase objects, that would be a 
regression.

So, I ask you:  What is allowed during tp_clear()?  Is this a hard rule?  What 
is the reason?

Kristján

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

[R. David Murray]

On Tue, 17 Apr 2012 01:11:14 +0200, Georg Brandl  wrote:
> On 16.04.2012 18:15, R. David Murray wrote:
> > I don't see how depending on Cython is better than depending on having
> > an existing Python.
> 
> No, it's not just an existing Python, it is (at least currently) the same
> version of Python being built.  Therefore I wrote about the bootstrapping
> problems when bytecode changes.

Ah, yes, I had missed that subtlety.

--David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Suggested addition to PEP 8 for context managers

[R. David Murray]

On Tue, 17 Apr 2012 08:53:43 +0200, Matej Cepl  wrote:
> On 16.4.2012 18:10, Nam Nguyen wrote:
> > a_list[pos + 1 : -1]
> 
> or other way around
> 
> a_list[pos+1:-1]


That's what I always use.  No spaces inside the brackets for me :)

If the expression gets unreadable that way, factor it out.

--David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic time, performance counter and process time functions

[R. David Murray]

On Tue, 17 Apr 2012 14:48:22 +1000, Cameron Simpson  wrote:
> On 16Apr2012 01:25, Victor Stinner  wrote:
> | I suppose that most people don't care that "resolution" and
> | "precision" are different things.
> 
> If we're using the same definitions we discussed offline, where
> 
>   - resolution is the units the clock call (underneath) works in (for
> example, nanoseconds)
> 
>   - precision is the effective precision of the results, for example
> milliseconds
> 
> I'd say people would care if they knew, and mostly care about
> "precision".

I think what the user cares about is "what is the smallest tick that
this clock result will faithfully represent?".  If the number of bits
returned is larger than the clock accuracy, you want the clock accuracy.
If the number of bits returned is smaller than the clock accuracy,
you want the number of bits.

(Yes, I'm using accuracy in a slightly different sense here...I think
we don't have the right words for this.)

To use other words, what the users cares about are the error bars on
the returned result.

--David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Suggested addition to PEP 8 for context managers

[Jan Kaliszewski]

Paul Moore dixit (2012-04-17, 08:14):

> On 16 April 2012 17:10, Nam Nguyen  wrote:
> > PEP 8 suggests no extra spaces after and before square brackets, and
> > colons. So code like this is appropriate:
> >
> > a_list[1:3]
> >
> > But I find it less readable in the case of:
> >
> > a_list[pos + 1:-1]
> >
> > The colon is seemingly lost in the right.
> >
> > Would it be better to read like below?
> >
> > a_list[pos + 1 : -1]
> >
> > Any opinion?
> 
> It says no space *before* a colon, not after. So the following should
> be OK (and is what I'd use):
> 
> a_list[pos + 1: -1]

I'd prefer either:

a_list[pos+1:-1]

or

a_list[(pos + 1):-1]

Regards.
*j

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic time, performance counter and process time functions

[Chris Angelico]

On Tue, Apr 17, 2012 at 2:48 PM, Cameron Simpson  wrote:
> On 16Apr2012 01:25, Victor Stinner  wrote:
> | I suppose that most people don't care that "resolution" and
> | "precision" are different things.
>
> If we're using the same definitions we discussed offline, where
>
>  - resolution is the units the clock call (underneath) works in (for
>    example, nanoseconds)
>
>  - precision is the effective precision of the results, for example
>    milliseconds
>
> I'd say people would care if they knew, and mostly care about
> "precision".

Meaning that resolution is a matter of format and API, not of clock.
If you take a C clock API that returns a value in nanoseconds and
return it as a Python float, you've changed the resolution. I don't
think resolution matters much, beyond that (for example) nanosecond
resolution allows a clock to be subsequently upgraded as far as
nanosecond precision without breaking existing code, even if currently
it's only providing microsecond precision. But it makes just as much
sense for your resolution to be 2**64ths-of-a-second or
quarters-of-the-internal-CPU-clock-speed as it does for nanoseconds.
As long as it's some fraction of the SI second, every different
resolution is just a fixed ratio away from every other one.

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Suggested addition to PEP 8 for context managers

[Guido van Rossum]

On Tue, Apr 17, 2012 at 12:14 AM, Paul Moore  wrote:
> On 16 April 2012 17:10, Nam Nguyen  wrote:
>> PEP 8 suggests no extra spaces after and before square brackets, and
>> colons. So code like this is appropriate:
>>
>> a_list[1:3]
>>
>> But I find it less readable in the case of:
>>
>> a_list[pos + 1:-1]
>>
>> The colon is seemingly lost in the right.
>>
>> Would it be better to read like below?
>>
>> a_list[pos + 1 : -1]
>>
>> Any opinion?
>
> It says no space *before* a colon, not after. So the following should
> be OK (and is what I'd use):
>
> a_list[pos + 1: -1]

I hope that's now what it says about slices -- that was meant for dict
displays. For slices it should be symmetrical. In this case I would
remove the spaces around the +, but it's okay to add spaces around the
: too. It does look odd to have an operator that binds tighter (the +)
surrounded by spaces while the operator that binds less tight (:) is
not. (And in this context, : should be considered an operator.)

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] issue 9141, finalizers and gc module

[martin]

What I want to know is, why is this limitation in place?  Here are  
two possibilities:


1)  "The order of calling finalizers in a cycle is undefined so  
it is not a solvable problem".  But this would allow a single  
object, with only internal cycles to be collected.  Currently this  
is not the case.


It's similar to this, but not exactly: "A finalizer in a cycle mail
try to refer back to an object that was already cleared, and fail
because of that; this may cause arbitrary failures changing from
run to run".

It's true that a cycle involving only a single object with __del__
could be safely collected. This special case was not implemented.

2)  "During collection, the interpreter is in a fragile state  
(linked lists of gc objects with refcount bookkeeping in place) and  
no unknown code can be allowed to run".  This is the reason I  
personally think is the true reason.


No, that's not the case at all. As Antoine explains in the issue,
there are plenty of ways in which Python code can be run when breaking
a cycle. Not only weakrefs, but also objects released as a consequence
of tp_clear which weren't *in* the cycle (but hung from it).

So, I ask you:  What is allowed during tp_clear()?  Is this a hard  
rule?  What is the reason?


We are all consenting adults. Everything is allowed - you just have to
live with the consequences.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Suggested addition to PEP 8 for context managers

[Barry Warsaw]

On Apr 17, 2012, at 08:25 AM, R. David Murray wrote:

>On Tue, 17 Apr 2012 08:53:43 +0200, Matej Cepl  wrote:
>> On 16.4.2012 18:10, Nam Nguyen wrote:
>> > a_list[pos + 1 : -1]
>> 
>> or other way around
>> 
>> a_list[pos+1:-1]
>
>
>That's what I always use.  No spaces inside the brackets for me :)
>
>If the expression gets unreadable that way, factor it out.

+1

-Barry
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Issue #13959: Re-implement imp.load_source() in imp.py.

[Brett Cannon]

On Tue, Apr 17, 2012 at 05:53, Antoine Pitrou  wrote:

> On Tue, 17 Apr 2012 04:11:31 +0200
> brett.cannon  wrote:
> > http://hg.python.org/cpython/rev/3b5b4b4bb43c
> > changeset:   76371:3b5b4b4bb43c
> > user:Brett Cannon 
> > date:Mon Apr 16 22:11:25 2012 -0400
> > summary:
> >   Issue #13959: Re-implement imp.load_source() in imp.py.
> >
> > files:
> >   Lib/imp.py  |   29 ++-
> >   Python/import.c |  390 
> >   2 files changed, 28 insertions(+), 391 deletions(-)
>
> It's nice to see all that C code go away :-)
>

Oh yes. =) It's definitely acting as motivation to put up with imp's crappy
APIs.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

[Brett Cannon]

On Tue, Apr 17, 2012 at 06:43, Eric V. Smith  wrote:

> On 4/17/2012 5:52 AM, Antoine Pitrou wrote:
> > On Mon, 16 Apr 2012 20:41:56 -0400
> > Brett Cannon  wrote:
> >> On Mon, Apr 16, 2012 at 20:27, Antoine Pitrou 
> wrote:
> >>
> >>> On Tue, 17 Apr 2012 01:11:14 +0200
> >>> Georg Brandl  wrote:
> 
>  No, it's not just an existing Python, it is (at least currently) the
> same
>  version of Python being built.  Therefore I wrote about the
> bootstrapping
>  problems when bytecode changes.
> 
>  Depending on Cython is better in that it breaks the bootstrapping
> cycle,
>  but on the other hand the C code may need to be regenerated when the C
> >>> API
>  changes in an incompatible way.
> >>>
> >>> Cython OTOH probably needs Python 2.x, which isn't that great for
> >>> building Python 3. And requiring Cython for developing is not very
> >>> contributor-friendly.
> >>>
> >>
> >> Well, required to regenerate _frozen_importlib, but nothing else. I mean
> >> making fixes go into importlib directly and get tested that way, not
> >> through __import__(). So really Cython would only be needed when
> >> importlib._bootstrap has been changed and you are making a commit.
> >
> > That's still a large dependency to bring in, while we already have a
> > working solution.
> > I'd understand using Cython to develop some new extension module which
> > requires linking against a C library (and is thus impossible to write
> > in pure Python). But for importlib that's totally non-necessary.
> >
> > I guess I'm -1 on it.
>
> I agree. If the problem we're trying to solve is that the generated file
> isn't always rebuilt, bringing in a large dependency like Cython seems
> like overkill to me.
>

Actually Cython would help with a subtle maintenance burden of maintaining
*any* C code for import. Right now,
Python/import.c:PyImport_ImportModuleLevelObject() is an accelerated C
version of importlib.__import__() through checking sys.modules, after which
it calls into the Python code. Cython would do away with that C
acceleration code (which I have already had to modify once and Antoine
found a couple refleaks in).


>
> We basically have a working solution now (thanks, Brett). I think we
> should focus on getting it polished. Maybe we can bring in Cython in a
> later release, if in the 3.4 timeframe it still seems like we have a
> problem to solve. I suspect things will be working fine.


I don't view this discussion as work/not work but more of work/work better
(possibly; I have severe bias here to cut the C code down to zilch since I
don't want to write anymore of it so I'm definitely not going to make any
final call on this topic).
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] making the import machinery explicit

[Brett Cannon]

The only people to bring up worries about this thread were Eric and Nick
and they both seem fine with making stuff explicit and changing the meaning
of None in sys.path_importer_cache, so I have created
http://bugs.python.org/issue14605 and will plan on implementing the ideas
for it before Python 3.3 goes out.

On Sat, Apr 14, 2012 at 16:03, Brett Cannon  wrote:

> To start off, what I am about to propose was brought up at the PyCon
> language summit and the whole room agreed with what I want to do here, so I
> honestly don't expect much of an argument (famous last words).
>
> In the "ancient" import.c days, a lot of import's stuff was hidden deep in
> the C code and in no way exposed to the user. But with importlib finishing
> PEP 302's phase 2 plans of getting imoprt to be properly refactored to use
> importers, path hooks, etc., this need no longer be the case.
>
> So what I propose to do is stop having import have any kind of implicit
> machinery. This means sys.meta_path gets a path finder that does the heavy
> lifting for import and sys.path_hooks gets a hook which provides a default
> finder. As of right now those two pieces of machinery are entirely implicit
> in importlib and can't be modified, stopped, etc.
>
> If this happens, what changes? First, more of importlib will get publicly
> exposed (e.g. the meta path finder would become public instead of private
> like it is along with everything else that is publicly exposed). Second,
> import itself technically becomes much simpler since it really then is
> about resolving module names, traversing sys.meta_path, and then handling
> fromlist w/ everything else coming from how the path finder and path hook
> work.
>
> What also changes is that sys.meta_path and sys.path_hooks cannot be
> blindly reset w/o blowing out import. I doubt anyone is even touching those
> attributes in the common case, and the few that do can easily just stop
> wiping out those two lists. If people really care we can do a warning in
> 3.3 if they are found to be empty and then fall back to old semantics, but
> I honestly don't see this being an issue as backwards-compatibility would
> just require being more careful of what you delete (which I have been
> warning people to do for years now) which is a minor code change which
> falls in line with what goes along with any new Python version.
>
> And lastly, sticking None in sys.path_importer_cache would no longer mean
> "do the implicit thing" and instead would mean the same as NullImporter
> does now (which also means import can put None into sys.path_importer_cache
> instead of NullImporter): no finder is available for an entry on sys.path
> when None is found. Once again, I don't see anyone explicitly sticking None
> into sys.path_importer_cache, and if they are they can easily stick what
> will be the newly exposed finder in there instead. The more common case
> would be people wiping out all entries of NullImporter so as to have a new
> sys.path_hook entry take effect. That code would instead need to clear out
> None on top of NullImporter as well (in Python 3.2 and earlier this would
> just be a performance loss, not a semantic change). So this too could
> change in Python 3.3 as long as people update their code like they do with
> any other new version of Python.
>
> In summary, I want no more magic "behind the curtain" for Python 3.3 and
> import: sys.meta_path and sys.path_hooks contain what they should and if
> they are emptied then imports will fail and None in sys.path_importer_cache
> means "no finder" instead of "use magical, implicit stuff".
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Require loaders set __package__ and __loader__

[Brett Cannon]

Anyone other than Eric have something to say on this proposal? Obviously
the discussion went tangential before I saw a clear consensus that what I
was proposing was fine with people.

On Sat, Apr 14, 2012 at 16:56, Brett Cannon  wrote:

> An open issue in PEP 302 is whether to require __loader__ attributes on
> modules. The claimed worry is memory consumption, but considering importlib
> and zipimport are already doing this that seems like a red herring.
> Requiring it, though, opens the door to people relying on its existence and
> thus starting to do things like loading assets with
> ``__loader__.get_data(path_to_internal_package_file)`` which allows code to
> not care how modules are stored (e.g. zip file, sqlite database, etc.).
>
> What I would like to do is update the PEP to state that loaders are
> expected to set __loader__. Now importlib will get updated to do that
> implicitly so external code can expect it post-import, but requiring
> loaders to set it would mean that code executed during import can rely on
> it as well.
>
> As for __package__, PEP 366 states that modules should set it but it isn't
> referenced by PEP 302. What I want to do is add a reference and make it
> required like __loader__. Importlib already sets it implicitly post-import,
> but once again it would be nice to do this pre-import.
>
> To help facilitate both new requirements, I would update the
> importlib.util.module_for_loader decorator to set both on a module that
> doesn't have them before passing the module down to the decorated method.
> That way people already using the decorator don't have to worry about
> anything and it is one less detail to have to worry about. I would also
> update the docs on importlib.util.set_package and importlib.util.set_loader
> to suggest people use importlib.util.module_for_loader and only use the
> other two decorators for backwards-compatibility.
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

[Antoine Pitrou]

On Tue, 17 Apr 2012 11:41:32 -0400
Brett Cannon  wrote:
> 
> Actually Cython would help with a subtle maintenance burden of maintaining
> *any* C code for import. Right now,
> Python/import.c:PyImport_ImportModuleLevelObject() is an accelerated C
> version of importlib.__import__() through checking sys.modules, after which
> it calls into the Python code. Cython would do away with that C
> acceleration code (which I have already had to modify once and Antoine
> found a couple refleaks in).

Would it? That's assuming Cython would be smart enough to do the
required optimizations.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] making the import machinery explicit

[Philip Jenvey]

On Apr 14, 2012, at 1:03 PM, Brett Cannon wrote:

> And lastly, sticking None in sys.path_importer_cache would no longer mean "do 
> the implicit thing" and instead would mean the same as NullImporter does now 
> (which also means import can put None into sys.path_importer_cache instead of 
> NullImporter): no finder is available for an entry on sys.path when None is 
> found.

Isn't it more explicit to cache the NullImporter instead of None?

--
Philip Jenvey

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] making the import machinery explicit

[Brett Cannon]

On Tue, Apr 17, 2012 at 13:45, Philip Jenvey  wrote:

>
> On Apr 14, 2012, at 1:03 PM, Brett Cannon wrote:
>
> > And lastly, sticking None in sys.path_importer_cache would no longer
> mean "do the implicit thing" and instead would mean the same as
> NullImporter does now (which also means import can put None into
> sys.path_importer_cache instead of NullImporter): no finder is available
> for an entry on sys.path when None is found.
>
> Isn't it more explicit to cache the NullImporter instead of None?
>

I disagree. NullImporter is just another finder that just so happens to
always fail. None is explicitly not a finder and thus obviously not going
to do anything. Isn't it clearer to say ``sys.path_importer_cache[path] is
None`` than ``isinstance(sys.path_importer_cache[path],
imp.NullImporter)``? I mean we have None to represent something is nothing
which is exactly what I want to convey; None in sys.path_importer_cache
means there is no finder for that path.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] issue 9141, finalizers and gc module

[Kristján Valur Jónsson]

> -Original Message-
> 
> No, that's not the case at all. As Antoine explains in the issue, there are
> plenty of ways in which Python code can be run when breaking a cycle. Not
> only weakrefs, but also objects released as a consequence of tp_clear which
> weren't *in* the cycle (but hung from it).
I see, that makes sense.  The rule is, then that we cannot delete objects with 
finalalizer, that can reach other garbage, simply because doing so may find the 
objects in an unexpected (cleared) state and thus cause weird errors.
(weakrefs are a special case, apparently dealt with separately.  And the 
callback cannot refer back to the referent) . 
 This reasoning belongs in the gcmodule.c, I think.
> 
> > So, I ask you:  What is allowed during tp_clear()?  Is this a hard
> > rule?  What is the reason?
> 
> We are all consenting adults. Everything is allowed - you just have to live 
> with
> the consequences.

Well, we specifically decided that objects with __del__ methods that are part 
of a cycle cannot be run.
The same reasoning was applied to generators, if they are in a certain state.
What makes iobase so special that its 'close' method can be run even if it is 
part of a cycle?
Why not allow it for all objects, then?

At the very least, I think this behaviour (this exception for iobase) merits 
being explicitly documented.

Kristján

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] issue 9141, finalizers and gc module

[Antoine Pitrou]

On Tue, 17 Apr 2012 17:22:57 +
Kristján Valur Jónsson  wrote:
> > 
> > We are all consenting adults. Everything is allowed - you just have to live 
> > with
> > the consequences.
> 
> Well, we specifically decided that objects with __del__ methods that are part 
> of a cycle cannot be run.
> The same reasoning was applied to generators, if they are in a certain state.
> What makes iobase so special that its 'close' method can be run even if it is 
> part of a cycle?

The reason is that making file objects uncollectable when they are part
of a reference cycle would be a PITA and a serious regression for many
applications, I think.

> Why not allow it for all objects, then?

I'm not the author of the original GC design. Perhaps it was
deliberately conservative at the time? I think PyPy has a more tolerant
solution for finalizers in reference cycles, perhaps they can explain it
here.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] importlib is now bootstrapped (and what that means)

[Brett Cannon]

On Tue, Apr 17, 2012 at 13:39, Antoine Pitrou  wrote:

> On Tue, 17 Apr 2012 11:41:32 -0400
> Brett Cannon  wrote:
> >
> > Actually Cython would help with a subtle maintenance burden of
> maintaining
> > *any* C code for import. Right now,
> > Python/import.c:PyImport_ImportModuleLevelObject() is an accelerated C
> > version of importlib.__import__() through checking sys.modules, after
> which
> > it calls into the Python code. Cython would do away with that C
> > acceleration code (which I have already had to modify once and Antoine
> > found a couple refleaks in).
>
> Would it? That's assuming Cython would be smart enough to do the
> required optimizations.
>

Yes, it is an assumption I'm making. I also assume we wouldn't make a
change like this w/o taking the time to run importlib through Cython and
seeing how the performance numbers come out.

-Brett


>
> Regards
>
> Antoine.
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] issue 9141, finalizers and gc module

[Maciej Fijalkowski]

On Tue, Apr 17, 2012 at 8:30 PM, Antoine Pitrou  wrote:

> On Tue, 17 Apr 2012 17:22:57 +
> Kristján Valur Jónsson  wrote:
> > >
> > > We are all consenting adults. Everything is allowed - you just have to
> live with
> > > the consequences.
> >
> > Well, we specifically decided that objects with __del__ methods that are
> part of a cycle cannot be run.
> > The same reasoning was applied to generators, if they are in a certain
> state.
> > What makes iobase so special that its 'close' method can be run even if
> it is part of a cycle?
>
> The reason is that making file objects uncollectable when they are part
> of a reference cycle would be a PITA and a serious regression for many
> applications, I think.
>
> > Why not allow it for all objects, then?
>
> I'm not the author of the original GC design. Perhaps it was
> deliberately conservative at the time? I think PyPy has a more tolerant
> solution for finalizers in reference cycles, perhaps they can explain it
> here.
>
> Regards
>
> Antoine.


PyPy breaks cycles randomly. I think a pretty comprehensive description of
what happens is here:

http://morepypy.blogspot.com/2008/02/python-finalizers-semantics-part-1.html
http://morepypy.blogspot.com/2008/02/python-finalizers-semantics-part-2.html

Cheers,
fijal
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Require loaders set __package__ and __loader__

[Andrew Svetlov]

+1 for initial proposition.

On Tue, Apr 17, 2012 at 6:59 PM, Brett Cannon  wrote:
> Anyone other than Eric have something to say on this proposal? Obviously the
> discussion went tangential before I saw a clear consensus that what I was
> proposing was fine with people.
>
>
> On Sat, Apr 14, 2012 at 16:56, Brett Cannon  wrote:
>>
>> An open issue in PEP 302 is whether to require __loader__ attributes on
>> modules. The claimed worry is memory consumption, but considering importlib
>> and zipimport are already doing this that seems like a red herring.
>> Requiring it, though, opens the door to people relying on its existence and
>> thus starting to do things like loading assets with
>> ``__loader__.get_data(path_to_internal_package_file)`` which allows code to
>> not care how modules are stored (e.g. zip file, sqlite database, etc.).
>>
>> What I would like to do is update the PEP to state that loaders are
>> expected to set __loader__. Now importlib will get updated to do that
>> implicitly so external code can expect it post-import, but requiring loaders
>> to set it would mean that code executed during import can rely on it as
>> well.
>>
>> As for __package__, PEP 366 states that modules should set it but it isn't
>> referenced by PEP 302. What I want to do is add a reference and make it
>> required like __loader__. Importlib already sets it implicitly post-import,
>> but once again it would be nice to do this pre-import.
>>
>> To help facilitate both new requirements, I would update the
>> importlib.util.module_for_loader decorator to set both on a module that
>> doesn't have them before passing the module down to the decorated method.
>> That way people already using the decorator don't have to worry about
>> anything and it is one less detail to have to worry about. I would also
>> update the docs on importlib.util.set_package and importlib.util.set_loader
>> to suggest people use importlib.util.module_for_loader and only use the
>> other two decorators for backwards-compatibility.
>
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com
>



-- 
Thanks,
Andrew Svetlov
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] making the import machinery explicit

[Terry Reedy]

On 4/17/2012 2:01 PM, Brett Cannon wrote:

Isn't it clearer to say
``sys.path_importer_cache[path] is None`` than
``isinstance(sys.path_importer_cache[path], imp.NullImporter)``?


Yes. Great work. Thanks for helping with the Idle breakage.

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Require loaders set __package__ and __loader__

[Nick Coghlan]

+1 here. Previously, it wasn't a reasonable requirement, since CPython
itself didn't comply with it.

--
Sent from my phone, thus the relative brevity :)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic time, performance counter and process time functions

[Victor Stinner]

> I think what the user cares about is "what is the smallest tick that
> this clock result will faithfully represent?".  If the number of bits
> returned is larger than the clock accuracy, you want the clock accuracy.
> If the number of bits returned is smaller than the clock accuracy,
> you want the number of bits.
>
> (Yes, I'm using accuracy in a slightly different sense here...I think
> we don't have the right words for this.)
>
> To use other words, what the users cares about are the error bars on
> the returned result.

Ok ok, resolution / accuracy / precision are confusing (or at least
not well known concepts). So it's better to keep the name:
time.perf_counter() :-)

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic time, performance counter and process time functions

[Cameron Simpson]

On 18Apr2012 00:18, Chris Angelico  wrote:
| On Tue, Apr 17, 2012 at 2:48 PM, Cameron Simpson  wrote:
| > On 16Apr2012 01:25, Victor Stinner  wrote:
| > | I suppose that most people don't care that "resolution" and
| > | "precision" are different things.
| >
| > If we're using the same definitions we discussed offline, where
| >
| >  - resolution is the units the clock call (underneath) works in (for
| >    example, nanoseconds)
| >
| >  - precision is the effective precision of the results, for example
| >    milliseconds
| >
| > I'd say people would care if they knew, and mostly care about
| > "precision".
| 
| Meaning that resolution is a matter of format and API, not of clock.
| If you take a C clock API that returns a value in nanoseconds and
| return it as a Python float, you've changed the resolution. I don't
| think resolution matters much, beyond that (for example) nanosecond
| resolution allows a clock to be subsequently upgraded as far as
| nanosecond precision without breaking existing code, even if currently
| it's only providing microsecond precision.

Yes; as stated, resolution is largely irrelevant to the user; it really
only places an upper bound on the precision. But it _is_ easy to know
from the unlying API doco, so it is easy to annotate the clocks with its
metadata.

Annoyingly, the more useful precision value is often harder to know.
-- 
Cameron Simpson  DoD#743
http://www.cskk.ezoshosting.com/cs/

If anyone disagrees with anything I say, I am quite prepared not only
to retract it, but also to deny under oath that I ever said it.
- Tom Lehrer
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic time, performance counter and process time functions

[Cameron Simpson]

On 17Apr2012 08:35, R. David Murray  wrote:
| On Tue, 17 Apr 2012 14:48:22 +1000, Cameron Simpson  wrote:
| > On 16Apr2012 01:25, Victor Stinner  wrote:
| > | I suppose that most people don't care that "resolution" and
| > | "precision" are different things.
| > 
| > If we're using the same definitions we discussed offline, where
| > 
| >   - resolution is the units the clock call (underneath) works in (for
| > example, nanoseconds)
| > 
| >   - precision is the effective precision of the results, for example
| > milliseconds
| > 
| > I'd say people would care if they knew, and mostly care about
| > "precision".
| 
| I think what the user cares about is "what is the smallest tick that
| this clock result will faithfully represent?".

That is what "precision" is supposed to mean above. I suspect we're all in
agreement here about its purpose.

| To use other words, what the users cares about are the error bars on
| the returned result.

Yes. And your discussion about the hw clock exceeding the API resulution
means we mean "the error bars as they escape from the OS API".

I still think we're all in agreement about the meaning here.
-- 
Cameron Simpson  DoD#743
http://www.cskk.ezoshosting.com/cs/

Often the good of the many is worth more than the good of the few. Saying
"if they have saved one life then they are worthwhile" places the good of the
few above the good of the many and past a certain threshold that's a
reprehensible attitude, for which I have utter contempt.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [RFC] PEP 418: Add monotonic time, performance counter and process time functions

[Victor Stinner]

> Here is a simplified version of the first draft of the PEP 418. The
> full version can be read online.
> http://www.python.org/dev/peps/pep-0418/
>
> The implementation of the PEP can be found in this issue:
> http://bugs.python.org/issue14428

The PEP is now fully ready: I just finished the implementation.

It looks like people, who complained on older versions of the PEP,
don't have new complain. Am I wrong? Everybody agree with the PEP 418?

I created http://hg.python.org/features/pep418/ repository for the
implementation. I tested it on Linux 3.3, FreeBSD 8.2, OpenBSD 5.0 and
Windows Seven. The implementation is now waiting your review!

There is also the toy implementation in pure Python for Python < 3.3:
https://bitbucket.org/haypo/misc/src/tip/python/pep418.py

Antoine asked "Is there a designated dictator for this PEP?". Nobody
answered. Maybe Guido van Rossum?

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [RFC] PEP 418: Add monotonic time, performance counter and process time functions

[Guido van Rossum]

I'll do it. Give me a few days (tomorrow is fully booked with horrible
meetings).

On Tue, Apr 17, 2012 at 5:06 PM, Victor Stinner
 wrote:
>> Here is a simplified version of the first draft of the PEP 418. The
>> full version can be read online.
>> http://www.python.org/dev/peps/pep-0418/
>>
>> The implementation of the PEP can be found in this issue:
>> http://bugs.python.org/issue14428
>
> The PEP is now fully ready: I just finished the implementation.
>
> It looks like people, who complained on older versions of the PEP,
> don't have new complain. Am I wrong? Everybody agree with the PEP 418?
>
> I created http://hg.python.org/features/pep418/ repository for the
> implementation. I tested it on Linux 3.3, FreeBSD 8.2, OpenBSD 5.0 and
> Windows Seven. The implementation is now waiting your review!
>
> There is also the toy implementation in pure Python for Python < 3.3:
> https://bitbucket.org/haypo/misc/src/tip/python/pep418.py
>
> Antoine asked "Is there a designated dictator for this PEP?". Nobody
> answered. Maybe Guido van Rossum?
>
> Victor
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/guido%40python.org



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython (2.7): Clean-up the SQLite introduction.

[Ezio Melotti]

Hi,

On Tue, Apr 17, 2012 at 8:48 PM, raymond.hettinger
 wrote:
> http://hg.python.org/cpython/rev/d229032dc213
> changeset:   76387:d229032dc213
> branch:      2.7
> user:        Raymond Hettinger 
> date:        Tue Apr 17 22:48:06 2012 -0400
> summary:
>  Clean-up the SQLite introduction.
>
> files:
>  Doc/library/sqlite3.rst |  52 ++--
>  1 files changed, 26 insertions(+), 26 deletions(-)
>
>
> diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst
> --- a/Doc/library/sqlite3.rst
> +++ b/Doc/library/sqlite3.rst
> @@ -23,7 +23,7 @@
>  :file:`/tmp/example` file::
>

The filename here should be updated too.

>    import sqlite3
> -   conn = sqlite3.connect('/tmp/example')
> +   conn = sqlite3.connect('example.db')

Best Regards,
Ezio Melotti
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 418: Add monotonic time, performance counter and process time functions

[Stephen J. Turnbull]

On Wed, Apr 18, 2012 at 8:25 AM, Victor Stinner
 wrote:

> Ok ok, resolution / accuracy / precision are confusing (or at least
> not well known concepts).

Maybe not to us, but in fields like astronomy and mechanical
engineering there are commonly accepted definitions:

Resolution: the smallest difference between two physical values that
results in a different measurement by a given instrument.

Precision: the amount of deviation among measurements of the same
physical value by a single instrument.

Accuracy: the amount of deviation of measurements by a given
instrument from true values.

As usual there are issues of average vs. worst case, different
resolution/precision/accuracy over the instrument's range, etc. which
need to be considered in reporting values for these properties.

A typical application to clocks would be the duration of one tick.  If
the clock ticks once per second and time values are reported in
nanoseconds, the /resolution/ is *1 billion* nanoseconds, not *1*
nanosecond./Precision/ corresponds to the standard deviation of
tick durations.  It is not necessarily the case that a precise
instrument will be accurate; if every tick is *exactly* 59/60 seconds,
the clock is infinitely precise but horribly inaccurate for most
purposes (it loses an hour every three days, and you'll miss your
favorite TV show!)  And two /accurate/ clocks should report the same
times and the same durations when measuring the same things.

I don't really care if Python decides to use idiosyncratic
definitions, but the above are easy enough to find (eg
http://en.wikipedia.org/wiki/Accuracy_and_precision).
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com