date:20131008

Le Tue, 8 Oct 2013 08:21:43 +0200,
Antoine Pitrou  a écrit :
> 
> > I think there *is* a potentially worthwhile generalisation here,
> > but I'm far from sure "is-a-dict" is the right data model - for
> > composability reasons, it feels like a "has-a" relationship with an
> > underlying data store may make more sense.
> 
> It doesn't work. Your "underlying mapping" can show too much variation
> for the wrapper/proxy to have sane semantics. For example, how do you
> combine with a defaultdict or a WeakKeyDictionary, knowing that the
> wrapper/proxy has to have its own internal mapping as well?

(for the record, I'm saying that because I did try the composition
route.)

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reduce memory footprint of Python


For the record, if you want to reduce memory footprint of Python on a
x86-64 bit, you can use the "x32" ABI for which Debian/Ubuntu provide
some support.

(install the required base packages and use CFLAGS=-mx32 LDFLAGS=-mx32)

Regards

Antoine.


Le Sun, 6 Oct 2013 17:32:37 +0200,
Victor Stinner  a écrit :
> Hi,
> 
> Slowly, I'm trying to see if it would be possible to reduce the memory
> footprint of Python using the tracemalloc module.
> 
> First, I noticed that linecache can allocate more than 2 MB. What do
> you think of adding a registry of "clear cache" functions? For
> exemple, re.purge() and linecache.clearcache(). gc.collect() clears
> free lists. I don't know if gc.collect() should be related to this new
> registy (clear all caches) or not.
> 
> The dictionary of interned Unicode strings can be large: up to 1.5 MB
> (with +30,000 strings). Just the dictionary, excluding size of
> strings. Is the size normal or not? Using tracemalloc, this dictionary
> is usually to largest memory block.
> 
> unittest doesn't look to release memory (the TestCase class) after the
> execution of a test.
> 
> test_import.test_module_with_large_stack() creates a large Python
> module and import it, but it does not unload it.
> 
> Should I open a separated issue for each idea to track them in the bug
> tracker, or a global issue?
> 
> Victor



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 455: TransformDict

2013-10-08 Thread Nick Coghlan

On 8 Oct 2013 18:36, "Antoine Pitrou"  wrote:
>
> Le Tue, 8 Oct 2013 08:21:43 +0200,
> Antoine Pitrou  a écrit :
> >
> > > I think there *is* a potentially worthwhile generalisation here,
> > > but I'm far from sure "is-a-dict" is the right data model - for
> > > composability reasons, it feels like a "has-a" relationship with an
> > > underlying data store may make more sense.
> >
> > It doesn't work. Your "underlying mapping" can show too much variation
> > for the wrapper/proxy to have sane semantics. For example, how do you
> > combine with a defaultdict or a WeakKeyDictionary, knowing that the
> > wrapper/proxy has to have its own internal mapping as well?
>
> (for the record, I'm saying that because I did try the composition
> route.)

It's OK if the key transforming API has to constrain the behaviour of the
underlying mapping or require an appropriately designed transform function
to handle more esoteric containers. Either would still be better than
categorically *disallowing* composition to the point where you can't even
compose it with OrderedDict. ChainMap doesn't compose sensibly with
arbitrary mappings like defaultdict, but composition is still the right
design choice because it works well with *most* custom mappings.

It's not that I think this is necessarily a *bad* idea (although the
composability problem gives me grave doubts), it's that I think it's not an
*urgent* idea, so why rush to review and include it in the weeks remaining
before the beta, when the option of publishing it as a recipe or a PyPI
module remains available?

Enums eventually made it in because we wanted to *use* them to dramatically
improve error messages from other stdlib modules. What's the comparable end
user payoff for including TransformDict in 3.4 rather than 3.5 (or never)?

Cheers,
Nick.

>
> Regards
>
> Antoine.
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 455: TransformDict

Le Tue, 8 Oct 2013 22:12:02 +1000,
Nick Coghlan  a écrit :
> 
> It's OK if the key transforming API has to constrain the behaviour of
> the underlying mapping or require an appropriately designed transform
> function to handle more esoteric containers. Either would still be
> better than categorically *disallowing* composition to the point
> where you can't even compose it with OrderedDict.

Well, you could ask the same question about OrderedDict, defaultdict or
Weak*Dictionary since neither of them use composition :-)

> ChainMap doesn't
> compose sensibly with arbitrary mappings like defaultdict, but
> composition is still the right design choice because it works well
> with *most* custom mappings.

ChainMap is easy to compose since it doesn't have to keep any
data-driven internal state.

> It's not that I think this is necessarily a *bad* idea (although the
> composability problem gives me grave doubts), it's that I think it's
> not an *urgent* idea, so why rush to review and include it in the
> weeks remaining before the beta, when the option of publishing it as
> a recipe or a PyPI module remains available?

It's just that I disagree we're rushing anything. The feature is fairly
simple, many people have already had a need for something like that.
(and amongst those people absolutely *zero* have said the design of the
feature proposal is inadequate)

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 455: TransformDict

2013-10-08 Thread Nick Coghlan

On 8 Oct 2013 22:31, "Antoine Pitrou"  wrote:
>
> Le Tue, 8 Oct 2013 22:12:02 +1000,
> Nick Coghlan  a écrit :
> >
> > It's OK if the key transforming API has to constrain the behaviour of
> > the underlying mapping or require an appropriately designed transform
> > function to handle more esoteric containers. Either would still be
> > better than categorically *disallowing* composition to the point
> > where you can't even compose it with OrderedDict.
>
> Well, you could ask the same question about OrderedDict, defaultdict or
> Weak*Dictionary since neither of them use composition :-)

We *did* ask the same question about those (except the Weak* variants,
simply due to age).

Each time, the point that each new dict variant would be used to justify
yet *more* variants was a cause for concern.

defaultdict made it through because it's just a convenience API for the
underlying "key missing" protocol, while OrderedDict spent time maturing
outside the standard library first.

>
> > ChainMap doesn't
> > compose sensibly with arbitrary mappings like defaultdict, but
> > composition is still the right design choice because it works well
> > with *most* custom mappings.
>
> ChainMap is easy to compose since it doesn't have to keep any
> data-driven internal state.

Yet composing ChainMap with defaultdict still breaks, because it's a
conceptually incoherent thing to do.

"Composition doesn't work with some mappings" isn't an adequate answer to
the criticism that a composition based design could work with more mappings
than just the builtin dict.

> > It's not that I think this is necessarily a *bad* idea (although the
> > composability problem gives me grave doubts), it's that I think it's
> > not an *urgent* idea, so why rush to review and include it in the
> > weeks remaining before the beta, when the option of publishing it as
> > a recipe or a PyPI module remains available?
>
> It's just that I disagree we're rushing anything. The feature is fairly
> simple, many people have already had a need for something like that.
> (and amongst those people absolutely *zero* have said the design of the
> feature proposal is inadequate)

Except the one who wanted to combine it with OrderedDict. Users won't be
able to combine it with ChainMap either.

The concrete container vs container-wrapper design decision is a
fundamental one and I don't believe the current PEP adequately makes the
case in favour of the former.

Cheers,
Nick.

>
> Regards
>
> Antoine.
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Semi-official read-only Github mirror of the CPython Mercurial repository

2013-10-08 Thread Eli Bendersky

On Mon, Sep 30, 2013 at 6:54 AM, Donald Stufft  wrote:

>
> On Sep 30, 2013, at 9:09 AM, Eli Bendersky  wrote:
>
> Hi all,
>
> https://github.com/python/cpython is now live as a semi-official, *read
> only* Github mirror of the CPython Mercurial repository. Let me know if you
> have any problems/concerns.
>
> I still haven't decided how often to update it (considering either just N
> times a day, or maybe use a Hg hook for batching). Suggestions are welcome.
>
> The methodology I used to create it is via hg-fast-export. I also tried to
> pack and gc the git repo as much as possible before the initial Github push
> - it went down from almost ~2GB to ~200MB (so this is the size of a fresh
> clone right now).
>
> Eli
>
> P.S. thanks Jesse for the keys to https://github.com/python
>
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/donald%40stufft.io
>
>
> Awesome! I find Github way nicer for reading source than hg.python.org's
> web interface, any chance I could convince you to do this for the peps repo
> too? ;)
>

Here you go - https://github.com/python/peps

Both mirrors are currently updated twice per night (23:00 and 04:30 US
Pacific time). I may switch over to a VPS at some point and have more
regular updates.

Eli
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 455: TransformDict

Le Tue, 8 Oct 2013 22:49:28 +1000,
Nick Coghlan  a écrit :
> > Well, you could ask the same question about OrderedDict,
> > defaultdict or Weak*Dictionary since neither of them use
> > composition :-)
> 
> We *did* ask the same question about those (except the Weak* variants,
> simply due to age).
> 
> Each time, the point that each new dict variant would be used to
> justify yet *more* variants was a cause for concern.

Could you explain? I don't understand.

> "Composition doesn't work with some mappings" isn't an adequate
> answer to the criticism that a composition based design could work
> with more mappings than just the builtin dict.

Fair enough.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reduce memory footprint of Python

On Sun, 06 Oct 2013 22:27:52 +0200, mar...@v.loewis.de wrote:
> 
> Quoting Benjamin Peterson :
> 
> >> If you know that your application uses a lot of memory, it is
> >> interesting to sometimes (when the application is idle) try to release
> >> some bytes to not use all the system memory. On embedded devices,
> >> memory is expensive and very limited. Each byte is important :-)
> >
> > How many embedded systems are running Python?
> 
> And of those, how many use the linecache module?

I can give you one data point: a mobile platform that (currently) uses
Python3, and does not use linecache because of how much memory it
consumes.

As far as the other caches go, every MB counts if you don't have swap,
even if you have 2GB of RAM.  And especially if you don't :)

Like Victor, I spent quite a bit of time helping to trim a MB here and a
MB there.  I would have loved to have had his tools for doing that work!
Trimming linecache was one of the early big wins, even though it made
some aspects of development harder.  We also cleared most of the other
caches after startup (of a long running process).

--David

PS: I have always thought it sad that the ready availability of memory,
CPU speed, and disk space tends to result in lazy programs.  I understand
there is an effort/value tradeoff, and I make those tradeoffs myself
all the time...but it still makes me sad.  Then, again, in my early
programming days I spent a fair amount of time writing and using Forth,
and that probably colors my worldview. :)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] On the dangers of giving developers the best resources

On Tue, Oct 8, 2013 at 8:33 AM, R. David Murray wrote:

> PS: I have always thought it sad that the ready availability of memory,
> CPU speed, and disk space tends to result in lazy programs.  I understand
> there is an effort/value tradeoff, and I make those tradeoffs myself
> all the time...but it still makes me sad.  Then, again, in my early
> programming days I spent a fair amount of time writing and using Forth,
> and that probably colors my worldview. :)
>

I never used or cared for Forth, but I have the same worldview. I remember
getting it from David Rosenthal, an early Sun reviewer. He stated that
engineers should be given the smallest desktop computer available, not the
largest, so they would feel their users' pain and optimize appropriately.
Sadly software vendors who are also hardware vendors have incentives going
in the opposite direction -- they want users to feel the pain so they'll
buy a new device.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 455: TransformDict

2013-10-08 Thread Yuriy Taraday

On Fri, Sep 13, 2013 at 10:40 PM, Antoine Pitrou wrote:

>
> Hello,
>
> Following the python-dev discussion, I've written a PEP to recap the
> proposal and the various arguments. It's inlined below, and it will
> probably appear soon at http://www.python.org/dev/peps/pep-0455/, too.
>
> Regards
>
> Antoine.
>

Hello.

Overall I think that's a great idea.
Here are some questions on it though. I'm sorry if some of these have
already been discussed in some other thread.

1. Thread safety.

PEP doesn't mention anything about thread safety while the implementation
proposed in the tracker is (very) not thread-safe. I think, PEP should
mention that this class have no guarantees.

2. Extra dict.

There should be a way to avoid creation of the second dict when there is no
need to store original keys. For example, email.message module doesn't
store original headers as they are not needed. The same applies to web
frameworks parsing HTTP headers or WSGI environment.

I'm sure I had another one. I'll send it once I remember.

-- 

Kind regards, Yuriy.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 455: TransformDict

2013-10-08 Thread MRAB


On 08/10/2013 19:02, Yuriy Taraday wrote:

On Fri, Sep 13, 2013 at 10:40 PM, Antoine Pitrou mailto:solip...@pitrou.net>> wrote:


Hello,

Following the python-dev discussion, I've written a PEP to recap the
proposal and the various arguments. It's inlined below, and it will
probably appear soon at http://www.python.org/dev/peps/pep-0455/, too.

Regards

Antoine.


Hello.

Overall I think that's a great idea.
Here are some questions on it though. I'm sorry if some of these have
already been discussed in some other thread.

1. Thread safety.

PEP doesn't mention anything about thread safety while the
implementation proposed in the tracker is (very) not thread-safe. I
think, PEP should mention that this class have no guarantees.

2. Extra dict.

There should be a way to avoid creation of the second dict when there is
no need to store original keys. For example, email.message module
doesn't store original headers as they are not needed. The same applies
to web frameworks parsing HTTP headers or WSGI environment.


If you don't need the original key, then you might as well just use a
transform function with a dict.


I'm sure I had another one. I'll send it once I remember.



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reduce memory footprint of Python

2013/10/8 R. David Murray :
> PS: I have always thought it sad that the ready availability of memory,
> CPU speed, and disk space tends to result in lazy programs.  I understand
> there is an effort/value tradeoff, and I make those tradeoffs myself
> all the time...but it still makes me sad.  Then, again, in my early
> programming days I spent a fair amount of time writing and using Forth,
> and that probably colors my worldview. :)

In defense of my weak-kneed generation, one of major principles of
programming as I understand it is to be as lazy as possible. (That is,
have the machine do the work. :))



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 455: TransformDict

On Tue, 08 Oct 2013 22:02:43 +0400, Yuriy Taraday  wrote:
> There should be a way to avoid creation of the second dict when there is no
> need to store original keys. For example, email.message module doesn't
> store original headers as they are not needed. The same applies to web
> frameworks parsing HTTP headers or WSGI environment.

This is not true.  email.message *very carefully* preserves the original
header name, case and all [*], that's part of its mandate (faithfully
reproducing the original parsed message).

That said, email.message can't use transformdict, since email.message
needs a list-with-case-insensitive-keyed-lookup, not a dict, because it
also preserves the original *order* of the headers.  And it couldn't use
an orderedtransformdict, either, since it also has to preserve
*duplicate* headers.

--David

[*] currently it loses whitespace information in certain cases, but the
new header parser/folder is supposed to fix that...or will once I fix a
few more corner cases :)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reduce memory footprint of Python

On Tue, 08 Oct 2013 14:43:03 -0400, Benjamin Peterson  
wrote:
> 2013/10/8 R. David Murray :
> > PS: I have always thought it sad that the ready availability of memory,
> > CPU speed, and disk space tends to result in lazy programs.  I understand
> > there is an effort/value tradeoff, and I make those tradeoffs myself
> > all the time...but it still makes me sad.  Then, again, in my early
> > programming days I spent a fair amount of time writing and using Forth,
> > and that probably colors my worldview. :)
> 
> In defense of my weak-kneed generation, one of major principles of
> programming as I understand it is to be as lazy as possible. (That is,
> have the machine do the work. :))

Yeah, that's a commonplace...but the obvious interpretation misses the
point, really, which I'm guessing is why you included the smiley.

It is not so much a matter of being as lazy as possible, as of being as
*smart* as practical[*] up front.  Frontload all the work (and it can be
a *lot* of work, which is why the aphorism is tongue-in-cheek) into the
design and algorithms and tools, then be amazingly productive using them.
That's really part of Python's fundamental design philosophy, I think.

In this context, if we'd been *really* smart-lazy in CPython development,
we'd have kept the memory and startup-time and...well, we probably do
pretty well on CPU actually...smaller, so that when smartphones came
along Python would have been the first high level language used on them,
because it fit.  Then we'd all be able to be *much* lazier now :)

--David

[*] The opposite part of that zen[**] being "if you make the code as smart
as possible, you won't be smart enough to debug it".

[**] Hmm.  I wonder if our "zen" is really a dialectic?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 455: TransformDict

2013-10-08 Thread Yuriy Taraday

On Tue, Oct 8, 2013 at 10:17 PM, MRAB  wrote:

> If you don't need the original key, then you might as well just use a
> transform function with a dict.

As I understood, storing original keys is not the purpose of TransformDict,
allowing hashing on something other then provided key itself is. This part
presents interest for the cases I mentioned.

-- 

Kind regards, Yuriy.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reduce memory footprint of Python

2013/10/8 R. David Murray :
> In this context, if we'd been *really* smart-lazy in CPython development,
> we'd have kept the memory and startup-time and...well, we probably do
> pretty well on CPU actually...smaller, so that when smartphones came
> along Python would have been the first high level language used on them,
> because it fit.  Then we'd all be able to be *much* lazier now :)

Even on desktop, startup time leaves a lot to be desired.



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 455: TransformDict

On Tue, 8 Oct 2013 23:36:05 +0400
Yuriy Taraday  wrote:
> On Tue, Oct 8, 2013 at 10:17 PM, MRAB  wrote:
> 
> > If you don't need the original key, then you might as well just use a
> > transform function with a dict.
> 
> As I understood, storing original keys is not the purpose of TransformDict,
> allowing hashing on something other then provided key itself is. This part
> presents interest for the cases I mentioned.

The purpose is really dual. If we didn't want to retain the original
keys, a dedicated class would be much less justified.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On the dangers of giving developers the best resources

2013-10-08 Thread Tim Delaney

On 9 October 2013 03:35, Guido van Rossum  wrote:

> On Tue, Oct 8, 2013 at 8:33 AM, R. David Murray wrote:
>
>> PS: I have always thought it sad that the ready availability of memory,
>> CPU speed, and disk space tends to result in lazy programs.  I understand
>> there is an effort/value tradeoff, and I make those tradeoffs myself
>> all the time...but it still makes me sad.  Then, again, in my early
>> programming days I spent a fair amount of time writing and using Forth,
>> and that probably colors my worldview. :)
>>
>
> I never used or cared for Forth, but I have the same worldview. I remember
> getting it from David Rosenthal, an early Sun reviewer. He stated that
> engineers should be given the smallest desktop computer available, not the
> largest, so they would feel their users' pain and optimize appropriately.
> Sadly software vendors who are also hardware vendors have incentives going
> in the opposite direction -- they want users to feel the pain so they'll
> buy a new device.
>

I look at it a different way. Developers should be given powerful machines
to speed up the development cycle (especially important when prototyping
and in the code/run unit test cycle), but everything should be tested on
the smallest machine available.

It's also a good idea for each developer to have a resource-constrained
machine for developer testing/profiling/etc. Virtual machines work quite
well for this - you can modify the resource constraints (CPU, memory, etc)
to simulate different scenarios.

I find that this tends to better promote the methodology of "make it right,
then make it fast (small, etc)", which I subscribe to. Optimising too early
leads to all your code being complicated, rather than just the bits that
need it.

Tim Delaney
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On the dangers of giving developers the best resources

Let's agree to disagree then. I see your methodology used all around me
with often problematic results. Maybe devs should have two machines -- one
monster that is *only* usable to develop fast, one small that where they
run their own apps (email, web browser etc.).


On Tue, Oct 8, 2013 at 1:30 PM, Tim Delaney wrote:

> On 9 October 2013 03:35, Guido van Rossum  wrote:
>
>> On Tue, Oct 8, 2013 at 8:33 AM, R. David Murray wrote:
>>
>>> PS: I have always thought it sad that the ready availability of memory,
>>> CPU speed, and disk space tends to result in lazy programs.  I understand
>>> there is an effort/value tradeoff, and I make those tradeoffs myself
>>> all the time...but it still makes me sad.  Then, again, in my early
>>> programming days I spent a fair amount of time writing and using Forth,
>>> and that probably colors my worldview. :)
>>>
>>
>> I never used or cared for Forth, but I have the same worldview. I
>> remember getting it from David Rosenthal, an early Sun reviewer. He stated
>> that engineers should be given the smallest desktop computer available, not
>> the largest, so they would feel their users' pain and optimize
>> appropriately. Sadly software vendors who are also hardware vendors have
>> incentives going in the opposite direction -- they want users to feel the
>> pain so they'll buy a new device.
>>
>
> I look at it a different way. Developers should be given powerful machines
> to speed up the development cycle (especially important when prototyping
> and in the code/run unit test cycle), but everything should be tested on
> the smallest machine available.
>
> It's also a good idea for each developer to have a resource-constrained
> machine for developer testing/profiling/etc. Virtual machines work quite
> well for this - you can modify the resource constraints (CPU, memory, etc)
> to simulate different scenarios.
>
> I find that this tends to better promote the methodology of "make it
> right, then make it fast (small, etc)", which I subscribe to. Optimising
> too early leads to all your code being complicated, rather than just the
> bits that need it.
>
> Tim Delaney
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On the dangers of giving developers the best resources

2013-10-08 Thread Tim Delaney

On 9 October 2013 07:38, Guido van Rossum  wrote:

> Let's agree to disagree then. I see your methodology used all around me
> with often problematic results. Maybe devs should have two machines -- one
> monster that is *only* usable to develop fast, one small that where they
> run their own apps (email, web browser etc.).
>

I've done that before too - it works quite well (especially if you set them
up to use a single keyboard/mouse).

I suspect the main determination of whether a fast machine as the primary
development machine works better depends heavily on the developer and what
their background is. I've also worked in resource-constrained environments,
so I'm always considering the impact of my choices, even when I go for the
less complicated option initially.

I've also been fortunate to mainly work in places where software
development was considered a craft, with pride in what we produced.
However, I think I should probably reconsider my viewpoint in light of my
current employment ... I despair at some of the code I see ...

Tim Delaney
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] inspect() and dir()


Greetings,

Currently, inspect() is dependent on dir().

Now that we can override what dir() returns on a class by class basis, we are seeing the side-effect of (possibly) 
incomplete inspect results, with also leads to (possibly) incomplete help().


I would think we want inspect to be more thorough, and return whatever it finds on the object, as well as whatever an 
abbreviated dir() might return.


Thoughts?

--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On the dangers of giving developers the best resources

2013-10-08 Thread Daniel Holth

Sounds like you are suggesting we get a raspberry pi. All sorts of dumb
waste shows up when you run code on those.
El oct 8, 2013 4:46 p.m., "Guido van Rossum"  escribió:

> Let's agree to disagree then. I see your methodology used all around me
> with often problematic results. Maybe devs should have two machines -- one
> monster that is *only* usable to develop fast, one small that where they
> run their own apps (email, web browser etc.).
>
>
> On Tue, Oct 8, 2013 at 1:30 PM, Tim Delaney 
> wrote:
>
>> On 9 October 2013 03:35, Guido van Rossum  wrote:
>>
>>> On Tue, Oct 8, 2013 at 8:33 AM, R. David Murray 
>>> wrote:
>>>
 PS: I have always thought it sad that the ready availability of memory,
 CPU speed, and disk space tends to result in lazy programs.  I
 understand
 there is an effort/value tradeoff, and I make those tradeoffs myself
 all the time...but it still makes me sad.  Then, again, in my early
 programming days I spent a fair amount of time writing and using Forth,
 and that probably colors my worldview. :)

>>>
>>> I never used or cared for Forth, but I have the same worldview. I
>>> remember getting it from David Rosenthal, an early Sun reviewer. He stated
>>> that engineers should be given the smallest desktop computer available, not
>>> the largest, so they would feel their users' pain and optimize
>>> appropriately. Sadly software vendors who are also hardware vendors have
>>> incentives going in the opposite direction -- they want users to feel the
>>> pain so they'll buy a new device.
>>>
>>
>> I look at it a different way. Developers should be given powerful
>> machines to speed up the development cycle (especially important when
>> prototyping and in the code/run unit test cycle), but everything should be
>> tested on the smallest machine available.
>>
>> It's also a good idea for each developer to have a resource-constrained
>> machine for developer testing/profiling/etc. Virtual machines work quite
>> well for this - you can modify the resource constraints (CPU, memory, etc)
>> to simulate different scenarios.
>>
>> I find that this tends to better promote the methodology of "make it
>> right, then make it fast (small, etc)", which I subscribe to. Optimising
>> too early leads to all your code being complicated, rather than just the
>> bits that need it.
>>
>> Tim Delaney
>>
>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/dholth%40gmail.com
>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reduce memory footprint of Python

2013-10-08 Thread Greg Ewing


R. David Murray wrote:

I can give you one data point: a mobile platform that (currently) uses
Python3, and does not use linecache because of how much memory it
consumes.


Wouldn't a sensible approach be to discard the linecache
when you've finished generating a traceback? You're not
likely to be generating tracebacks often enough to be
worth keeping it from one to the next.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] inspect() and dir()

I'd say that the effect overriding dir() has on help() is intentional. Not
sure about inspect -- it has other uses. However, the last time I used it I
was definitely hunting for stuff to document.


On Tue, Oct 8, 2013 at 2:29 PM, Ethan Furman  wrote:

> Greetings,
>
> Currently, inspect() is dependent on dir().
>
> Now that we can override what dir() returns on a class by class basis, we
> are seeing the side-effect of (possibly) incomplete inspect results, with
> also leads to (possibly) incomplete help().
>
> I would think we want inspect to be more thorough, and return whatever it
> finds on the object, as well as whatever an abbreviated dir() might return.
>
> Thoughts?
>
> --
> ~Ethan~
> __**_
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/**mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/**mailman/options/python-dev/**
> guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On the dangers of giving developers the best resources

It's not actually so much the extreme waste that I'm looking to expose, but
rather the day-to-day annoyances of stuff you use regularly that slows you
down by just a second (or ten), or things that gets slower at each release.

On Tue, Oct 8, 2013 at 2:57 PM, Daniel Holth  wrote:

> Sounds like you are suggesting we get a raspberry pi. All sorts of dumb
> waste shows up when you run code on those.
> El oct 8, 2013 4:46 p.m., "Guido van Rossum"  escribió:
>
>>  Let's agree to disagree then. I see your methodology used all around me
>> with often problematic results. Maybe devs should have two machines -- one
>> monster that is *only* usable to develop fast, one small that where they
>> run their own apps (email, web browser etc.).
>>
>>
>> On Tue, Oct 8, 2013 at 1:30 PM, Tim Delaney 
>> wrote:
>>
>>> On 9 October 2013 03:35, Guido van Rossum  wrote:
>>>
 On Tue, Oct 8, 2013 at 8:33 AM, R. David Murray 
 wrote:

> PS: I have always thought it sad that the ready availability of memory,
> CPU speed, and disk space tends to result in lazy programs.  I
> understand
> there is an effort/value tradeoff, and I make those tradeoffs myself
> all the time...but it still makes me sad.  Then, again, in my early
> programming days I spent a fair amount of time writing and using Forth,
> and that probably colors my worldview. :)
>

 I never used or cared for Forth, but I have the same worldview. I
 remember getting it from David Rosenthal, an early Sun reviewer. He stated
 that engineers should be given the smallest desktop computer available, not
 the largest, so they would feel their users' pain and optimize
 appropriately. Sadly software vendors who are also hardware vendors have
 incentives going in the opposite direction -- they want users to feel the
 pain so they'll buy a new device.

>>>
>>> I look at it a different way. Developers should be given powerful
>>> machines to speed up the development cycle (especially important when
>>> prototyping and in the code/run unit test cycle), but everything should be
>>> tested on the smallest machine available.
>>>
>>> It's also a good idea for each developer to have a resource-constrained
>>> machine for developer testing/profiling/etc. Virtual machines work quite
>>> well for this - you can modify the resource constraints (CPU, memory, etc)
>>> to simulate different scenarios.
>>>
>>> I find that this tends to better promote the methodology of "make it
>>> right, then make it fast (small, etc)", which I subscribe to. Optimising
>>> too early leads to all your code being complicated, rather than just the
>>> bits that need it.
>>>
>>> Tim Delaney
>>>
>>
>>
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/dholth%40gmail.com
>>
>>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On the dangers of giving developers the best resources

2013-10-08 Thread Tim Delaney

On 9 October 2013 09:10, Guido van Rossum  wrote:

> It's not actually so much the extreme waste that I'm looking to expose,
> but rather the day-to-day annoyances of stuff you use regularly that slows
> you down by just a second (or ten), or things that gets slower at each
> release.
>

Veering off-topic (but still related) ...

There's a reason I turn off all animations when I set up a machine for
someone ... I've found turning off the animations is the quickest way to
make a machine feel faster - even better than adding an SSD. The number of
times I've fixed a "slow" machine by this one change ...

I think everyone even remotely involved in the existence of animations in
the OS should be forced to have the slowest animations turned on at all
times, no matter the platform (OSX, Windows, Linux ...). Which comes back
to the idea of developers having slow machines so they feel the pain ...

Tim Delaney
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 457: Syntax For Positional-Only Parameters

2013-10-08 Thread Larry Hastings



I've contributed a new PEP to humanity.  I include the RST for your 
reading pleasure below, but you can also read it online here:


   http://www.python.org/dev/peps/pep-0457/


Discuss,


//arry/

-

PEP: 457
Title: Syntax For Positional-Only Parameters
Version: $Revision$
Last-Modified: $Date$
Author: Larry Hastings 
Discussions-To: Python-Dev 
Status: Draft
Type: Informational
Content-Type: text/x-rst
Created: 08-Oct-2013



Overview


This PEP proposes a syntax for positional-only parameters in Python.
Positional-only parameters are parameters without an externally-usable
name; when a function accepting positional-only parameters is called,
positional arguments are mapped to these parameters based solely on
their position.

=
Rationale
=

Python has always supported positional-only parameters.
Early versions of Python lacked the concept of specifying
parameters by name, so naturally all parameters were
positional-only.  This changed around Python 1.0, when
all parameters suddenly became positional-or-keyword.
But, even in current versions of Python, many CPython
"builtin" functions still only accept positional-only
arguments.

Functions implemented in modern Python can accept
an arbitrary number of positional-only arguments, via the
variadic ``*args`` parameter.  However, there is no Python
syntax to specify accepting a specific number of
positional-only parameters.  Put another way, there are
many builtin functions whose signatures are simply not
expressable with Python syntax.

This PEP proposes a backwards-compatible syntax that should
permit implementing any builtin in pure Python code.

-
Positional-Only Parameter Semantics In Current Python
-

There are many, many examples of builtins that only
accept positional-only parameters.  The resulting
semantics are easily experienced by the Python
programmer--just try calling one, specifying its
arguments by name::

>>> pow(x=5, y=3)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: pow() takes no keyword arguments

In addition, there are some functions with particularly
interesting semantics:

  * ``range()``, which accepts an optional parameter
to the *left* of its required parameter. [#RANGE]_

  * ``dict()``, whose mapping/iterator parameter is optional and
semantically must be positional-only.  Any externally
visible name for this parameter would occlude
that name going into the ``**kwarg`` keyword variadic
parameter dict! [#DICT]_

Obviously one can simulate any of these in pure Python code
by accepting ``(*args, **kwargs)`` and parsing the arguments
by hand.  But this results in a disconnect between the
Python function's signature and what it actually accepts,
not to mention the work of implementing said argument parsing.

==
Motivation
==

This PEP does not propose we implement positional-only
parameters in Python.  The goal of this PEP is simply
to define the syntax, so that:

* Documentation can clearly, unambiguously, and
  consistently express exactly how the arguments
  for a function will be interpreted.

* The syntax is reserved for future use, in case
  the community decides someday to add positional-only
  parameters to the language.

* Argument Clinic can use a variant of the syntax
  as part of its input when defining
  the arguments for built-in functions.

=
The Current State Of Documentation For Positional-Only Parameters
=

The documentation for positional-only parameters is incomplete
and inconsistent:

* Some functions denote optional groups of positional-only arguments
  by enclosing them in nested square brackets. [#BORDER]_

* Some functions denote optional groups of positional-only arguments
  by presenting multiple prototypes with varying numbers of
  arguments. [#SENDFILE]_

* Some functions use *both* of the above approaches. [#RANGE]_ [#ADDCH]_

One more important idea to consider: currently in the documentation
there's no way to tell whether a function takes positional-only
parameters.  ``open()`` accepts keyword arguments, ``ord()`` does
not, but there is no way of telling just by reading the
documentation that this is true.


Syntax And Semantics


From the "ten-thousand foot view", and ignoring ``*args`` and ``**kwargs``
for now, the grammar for a function definition currently looks like this::

def name(positional_or_keyword_parameters, *, keyword_only_parameters):

Building on that perspective, the new syntax for functions would look
like this::

def name(positional_only_parameters, /, 
positional_or_keyword_parameters,

 *, keyword_only_parameters):

All parameters before the ``/`` are

Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters

2013-10-08 Thread Steven D'Aprano

On Wed, Oct 09, 2013 at 01:33:26AM +0200, Larry Hastings wrote:

> This PEP proposes a syntax for positional-only parameters in Python.
> Positional-only parameters are parameters without an externally-usable
> name; when a function accepting positional-only parameters is called,
> positional arguments are mapped to these parameters based solely on
> their position.

Nicely done. A few comments follow:

> Positional-only parameters can be optional, but the mechanism is
> significantly different from positional-or-keyword or keyword-only
> parameters.  Positional-only parameters don't accept default
> values.  Instead, positional-only parameters can be specified
> in optional "groups".  Groups of parameters are surrounded by
> square brackets, like so::
> 
> def addch([y, x,] ch, [attr], /):

I think you need to explain the motivation for this, particularly in 
light of the fact (below) that they will in fact receive a default 
value, namely the proposed "undefined" singleton.

Personally, if I have a function like this:

def mypower(x, y, [n,] /):
if n is undefined:
n = 1
...

I would much prefer to write it like this:

def mypower(x, y, [n=1,] /):
...

The PEP should explain why I cannot, or accept that I should be able to.

>   * For clarity and consistency, the comma for a parameter always
> comes immediately after the parameter name.  It's a syntax error
> to specify a square bracket between the name of a parameter and
> the following comma.  (This is far more readable than putting
> the comma outside the square bracket, particularly for nested
> groups.)

I note that in your example above, you put the comma outside the square 
bracket:

def addch([y, x,] ch, [attr], /):

which seems perfectly readable to me.

I think that a much better explanation for prohibiting the comma outside 
the brackets is logical consistency: if you put the comma outside the 
brackets, and the attr is left out, you get two commas in a row and we 
don't want that.

> It's possible to specify a function prototype where the mapping
> of arguments to parameters is ambiguous.  Consider::
> 
> def range([start,] stop, [range], /):

And you do it again, putting the comma outside of the square brackets 
:-)

> If we decide to implement positional-only parameters in a future
> version of Python, we'd have to do some additional work to preserve
> their semantics.  The problem: how do we inform a parameter that
> no value was passed in for it when the function was called?
> 
> The obvious solution: add a new singleton constant to Python
> that is passed in when a parameter is not mapped to an argument.
> I propose that the value be called called ``undefined``,
> and be a singleton of a special class called ``Undefined``.
> If a positional-only parameter did not receive an argument
> when called, its value would be set to ``undefined``.

I would much prefer Undefined and UndefinedType. That matches other 
singletons like None, NotImplemented, Ellipsis, even True and False.

What (if any) methods and attributes would Undefined have? If it doesn't 
have any, doesn't that make it functionally equivalent to None? Why not 
just use None? The PEP needs to explain why it needs to invent yet 
another singleton that quacks like None.

[Bikeshed: perhaps Missing is more appropriate than Undefined? After 
all, the parameter is defined, only the value is missing.]

> But this raises a further problem.  How do can we tell the
> difference between "this positional-only parameter did not
> receive an argument" and "the caller passed in ``undefined``
> for this parameter"?

Why treat this as a problem to be solved? If somebody wants to go to the 
trouble of writing:

addchr('c', Undefined)

instead of just:

addchr('c')

why not let them?

> It'd be nice to make it illegal to pass ``undefined`` in
> as an argument to a function--to, say, raise an exception.
> But that would slow Python down, and the "consenting adults"
> rule appears applicable here.  So making it illegal should
> probably be strongly discouraged but not outright prevented.

An argument for allowing Undefined (or None, as the case may be): 
sometimes you have a situation like this:

c = get_char()
attr = get_attr()
if attr is Undefined:  # or None
addchr(c)
else:
addchr(c, attr)

That sucks. But this isn't much better:

c = get_char()
attr = get_attr()
if attr is Undefined:  # or None
t = (c,)
else:
t = (c, attr)
addchr(*t)

Being able to pass Undefined explicitly avoids this ugliness.

> However, it should be allowed (and encouraged) for user
> functions to specify ``undefined`` as a default value for
> parameters.

Because I think this is important, I'm going to raise it again: I think 
it is important for the PEP to justify why user functions cannot specify 
arbitrary values as defaults, not just Undefined.

Thanks again for tackling this.

-- 
Steven
___
Python-De

Re: [Python-Dev] On the dangers of giving developers the best resources

2013-10-08 Thread MRAB


On 08/10/2013 23:21, Tim Delaney wrote:

On 9 October 2013 09:10, Guido van Rossum mailto:gu...@python.org>> wrote:

It's not actually so much the extreme waste that I'm looking to
expose, but rather the day-to-day annoyances of stuff you use
regularly that slows you down by just a second (or ten), or things
that gets slower at each release.


Veering off-topic (but still related) ...

There's a reason I turn off all animations when I set up a machine for
someone ... I've found turning off the animations is the quickest way to
make a machine feel faster - even better than adding an SSD. The number
of times I've fixed a "slow" machine by this one change ...

I think everyone even remotely involved in the existence of animations
in the OS should be forced to have the slowest animations turned on at
all times, no matter the platform (OSX, Windows, Linux ...). Which comes
back to the idea of developers having slow machines so they feel the
pain ...


I remember one time when I was using a Mac. Although it was faster than
another machine I was using, the GUI felt sluggish because instead of
windows just appearing and disappearing they expanded and contracted,
which, of course, took time; not much time, true, but enough to become
annoying.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters


On 10/08/2013 04:33 PM, Larry Hastings wrote:


I've contributed a new PEP to humanity.  I include the RST for your reading 
pleasure below, but you can also read it
online here:

http://www.python.org/dev/peps/pep-0457/


I like it!  :)

Feedback below...





==
Notes For A Future Implementor
==


[snip]


The obvious solution: add a new singleton constant to Python
that is passed in when a parameter is not mapped to an argument.
I propose that the value be called called ``undefined``,

^
called is doubled -/


and be a singleton of a special class called ``Undefined``.
If a positional-only parameter did not receive an argument
when called, its value would be set to ``undefined``.


To stick with current practice:

--> type(Ellipsis)


--> type(None)


--> type(NotImplemented)


The instance should be Undefined and its class either undefined or 
UndefinedType.



But this raises a further problem.  How do can we tell the
difference between "this positional-only parameter did not
receive an argument" and "the caller passed in ``undefined``
for this parameter"?

It'd be nice to make it illegal to pass ``undefined`` in
as an argument to a function--to, say, raise an exception.
But that would slow Python down, and the "consenting adults"
rule appears applicable here.  So making it illegal should
probably be strongly discouraged but not outright prevented.


Agreed.



However, it should be allowed (and encouraged) for user
functions to specify ``undefined`` as a default value for
parameters.


Also agreed.




Unresolved Questions


There are three types of parameters in Python:

   1. positional-only parameters,
   2. positional-or-keyword parameters, and
   3. keyword-only parameters.

Python allows functions to have both 2 and 3.  And some
builtins (e.g. range) have both 1 and 3.  Does it make
sense to have functions that have both 1 and 2?  Or
all of the above?


Rather than try to think up the cases where it makes sense, let's just support 
all three.


Thanks for writing this up, Larry!

--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] lzma and 'x' mode open

2013-10-08 Thread Tim Heaney

I love the 'x' mode open in recent versions of Python. I just discovered
that lzma.open doesn't support it. It seems there's an elif that explicitly
checks the modes allowed. I added "x" and "xb" to the choices and now it
seems to work as I would like.


patch.lzma.py
Description: Binary data
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] lzma and 'x' mode open

2013-10-08 Thread Eric V. Smith

Please open a bug report on bugs.python.org so this doesn't get lost. 

--
Eric.

> On Oct 8, 2013, at 8:49 PM, Tim Heaney  wrote:
> 
> I love the 'x' mode open in recent versions of Python. I just discovered that 
> lzma.open doesn't support it. It seems there's an elif that explicitly checks 
> the modes allowed. I added "x" and "xb" to the choices and now it seems to 
> work as I would like.
> 
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters

2013-10-08 Thread Larry Hastings



A very quick reply, more tomorrow.

On 10/09/2013 02:15 AM, Steven D'Aprano wrote:

I note that in your example above, you put the comma outside the square
bracket:

def addch([y, x,] ch, [attr], /):

which seems perfectly readable to me.


It becomes less readable / more tiresome with nested groups.  Which 
square bracket should the comma come after?  Anyway, it seems like you 
agree with the syntactic requirement for other reasons.


(And, I already fixed the two places in the PEP where I had the comma 
outside the square brackets, thanks for pointing it out.)



I would much prefer Undefined and UndefinedType. That matches other
singletons like None, NotImplemented, Ellipsis, even True and False.


You're probably right.



[Bikeshed: perhaps Missing is more appropriate than Undefined? After
all, the parameter is defined, only the value is missing.]


Let the bikeshedding begin!



[Argument for allowing explicitly passing "undefined":
this lets the iterable in foo(*iterable) yield "undefined",
which could be convenient]


That's a good thought.  But I'd be interested in more debate on the 
subject to see if people have other good reasons for/against encouraging 
explicitly using "undefined".




Because I think this is important, I'm going to raise it again: I think
it is important for the PEP to justify why user functions cannot specify
arbitrary values as defaults, not just Undefined.


Primarily because this codifies existing practice.  C bulitins with 
positional-only decide how to assign their arguments based on how many 
there are, and when a parameter doesn't receive an argument it almost 
never gets a default value.  I was trying to preserve these exact 
semantics, which is where the optional groups came from.  And since I 
already had optional groups and "undefined", that seemed sufficient.


I'll also admit, I mainly ruled it out back before the prefer-left 
disambiguation rule, and allowing default values for positional-only led 
to loads of ambiguity.  I hadn't reconsidered the restriction in light 
of the new rule.


But I still think the semantics get weird quickly.  Consider:

   def bar([a=7, b,] c, [d,] /):

Default values would have to grow from the outside in, staying away from 
the required positional-only parameter group.  So on the left side 
they'd be backwards.


And if you call bar() with two arguments, you'd get (a b c), not (c d).  
In fact there'd be no way of specifying
d without providing four arguments.  By that token there'd be no way of 
calling bar() and specifying d without stomping on the default value for a.


I agree I should do a better job of justifying the design in a future 
revision.  I'd have to think long and hard before allowing default 
values for positional-only arguments... and that's a restriction I 
/wouldn't/ relax for Argument Clinic.


Anyway it's late, more tomorrow.


//arry/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] lzma and 'x' mode open

On Tue, 08 Oct 2013 20:49:17 -0400, Tim Heaney  wrote:
> I love the 'x' mode open in recent versions of Python. I just discovered
> that lzma.open doesn't support it. It seems there's an elif that explicitly
> checks the modes allowed. I added "x" and "xb" to the choices and now it
> seems to work as I would like.

Please file an enhancement request at bugs.python.org.

--David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters

On 10/08/2013 05:15 PM, Steven D'Aprano wrote:

On Wed, Oct 09, 2013 at 01:33:26AM +0200, Larry Hastings wrote:

The obvious solution: add a new singleton constant to Python
that is passed in when a parameter is not mapped to an argument.
I propose that the value be called called ``undefined``,
and be a singleton of a special class called ``Undefined``.
If a positional-only parameter did not receive an argument
when called, its value would be set to ``undefined``.

I would much prefer Undefined and UndefinedType. That matches other
singletons like None, NotImplemented, Ellipsis, even True and False.

The type of Ellipsis is ellipsis, and the type of True and False is bool. ;) That aside, I agree that Undefined and
UndefinedType would be better.

What (if any) methods and attributes would Undefined have? If it doesn't
have any, doesn't that make it functionally equivalent to None? Why not
just use None? The PEP needs to explain why it needs to invent yet
another singleton that quacks like None.

I think the key question here is are there places where None is the passed in value, and its meaning is something
besides "no value passed in". I don't know the answer.

[Bikeshed: perhaps Missing is more appropriate than Undefined? After
all, the parameter is defined, only the value is missing.]

Or the value is undefined. ;)

--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters

2013/10/8 Larry Hastings :
> This PEP proposes a backwards-compatible syntax that should
> permit implementing any builtin in pure Python code.

This is rather too strong. You can certainly implement them; you just
have to implement the argument parsing yourself. Python's
call/signature syntax is already extremely expressive, and resolving
call arguments to formal parameters is already a complicated (and
slow) process. Implementing functions with such strange argument
semantics is hardly common enough to justify the whole grouping syntax
proposed in this PEP. -1 to that. I think I can live with "/", but
YANGTNI still.



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] lzma and 'x' mode open

2013-10-08 Thread Tim Heaney

Done.

http://bugs.python.org/issue19201

I guess I should have known that. Sorry to bother python-dev with this.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters

2013-10-08 Thread Joao S. O. Bueno

I am -1 for the optional parameters and grouping stuff -
no need to make complicated stuff easier to do  just because "range" semantics
is strange to start with.
(And one can implement a range-like funciton explictly parsing
the parameters if needed be.)

As for the "/" delimiting positional only parameters - I think it would
be nice to have. -but just that - mandatory positional only parameters.

Apart from that, I could not figure if the PEP covers a situation where the name
of a positional-only parameter is passed in as a kwarg -

def a([b],/,**kw):
   print (b)

does calling "a(b=5)" should raise a TypeError, or put "5" in
"""kw["b"]""" and leave
"b" as "undefined" ?

   js
 -><-



On 8 October 2013 22:31, Benjamin Peterson  wrote:
> 2013/10/8 Larry Hastings :
>> This PEP proposes a backwards-compatible syntax that should
>> permit implementing any builtin in pure Python code.
>
> This is rather too strong. You can certainly implement them; you just
> have to implement the argument parsing yourself. Python's
> call/signature syntax is already extremely expressive, and resolving
> call arguments to formal parameters is already a complicated (and
> slow) process. Implementing functions with such strange argument
> semantics is hardly common enough to justify the whole grouping syntax
> proposed in this PEP. -1 to that. I think I can live with "/", but
> YANGTNI still.
>
>
>
> --
> Regards,
> Benjamin
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/jsbueno%40python.org.br
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] On the dangers of giving developers the best resources

2013-10-08 Thread Nam Nguyen

On Wed, Oct 9, 2013 at 12:22 AM, MRAB  wrote:

> On 08/10/2013 23:21, Tim Delaney wrote:
>
>> On 9 October 2013 09:10, Guido van Rossum > > wrote:
>>
>> It's not actually so much the extreme waste that I'm looking to
>> expose, but rather the day-to-day annoyances of stuff you use
>> regularly that slows you down by just a second (or ten), or things
>> that gets slower at each release.
>>
>>
>> Veering off-topic (but still related) ...
>>
>> There's a reason I turn off all animations when I set up a machine for
>> someone ... I've found turning off the animations is the quickest way to
>> make a machine feel faster - even better than adding an SSD. The number
>> of times I've fixed a "slow" machine by this one change ...
>>
>> I think everyone even remotely involved in the existence of animations
>> in the OS should be forced to have the slowest animations turned on at
>> all times, no matter the platform (OSX, Windows, Linux ...). Which comes
>> back to the idea of developers having slow machines so they feel the
>> pain ...
>>
>>  I remember one time when I was using a Mac. Although it was faster than
> another machine I was using, the GUI felt sluggish because instead of
> windows just appearing and disappearing they expanded and contracted,
> which, of course, took time; not much time, true, but enough to become
> annoying.


Try holding shift and minimizing/restoring Finder in OS X ;).
Nam
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters

2013-10-08 Thread Terry Reedy


On 10/8/2013 9:31 PM, Benjamin Peterson wrote:

2013/10/8 Larry Hastings :

This PEP proposes a backwards-compatible syntax that should
permit implementing any builtin in pure Python code.


This is rather too strong. You can certainly implement them; you just
have to implement the argument parsing yourself. Python's
call/signature syntax is already extremely expressive, and resolving
call arguments to formal parameters is already a complicated (and
slow) process. Implementing functions with such strange argument
semantics is hardly common enough to justify the whole grouping syntax
proposed in this PEP. -1 to that. I think I can live with "/", but
YANGTNI still.


I am for having a way to succintly properly describe the signature of C 
in the manual and docstrings and help output. As it is now, the only 
safe thing to do, without trial and exception, is to assume positional 
only unless one knows otherwise.


But the syntax should only be as complicated as needed for 'most' C 
functions. If there are a few complicated oddballs, perhaps they could 
be simplified, or given a special-case explanation.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters

2013/10/8 Terry Reedy :
> On 10/8/2013 9:31 PM, Benjamin Peterson wrote:
>>
>> 2013/10/8 Larry Hastings :
>>>
>>> This PEP proposes a backwards-compatible syntax that should
>>> permit implementing any builtin in pure Python code.
>>
>>
>> This is rather too strong. You can certainly implement them; you just
>> have to implement the argument parsing yourself. Python's
>> call/signature syntax is already extremely expressive, and resolving
>> call arguments to formal parameters is already a complicated (and
>> slow) process. Implementing functions with such strange argument
>> semantics is hardly common enough to justify the whole grouping syntax
>> proposed in this PEP. -1 to that. I think I can live with "/", but
>> YANGTNI still.
>
>
> I am for having a way to succintly properly describe the signature of C in
> the manual and docstrings and help output. As it is now, the only safe thing
> to do, without trial and exception, is to assume positional only unless one
> knows otherwise.

Having a nice syntax for the docs is quite different from implementing
it in the language.



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters


On 10/08/2013 08:09 PM, Benjamin Peterson wrote:

2013/10/8 Terry Reedy :

On 10/8/2013 9:31 PM, Benjamin Peterson wrote:


2013/10/8 Larry Hastings :


This PEP proposes a backwards-compatible syntax that should
permit implementing any builtin in pure Python code.



This is rather too strong. You can certainly implement them; you just
have to implement the argument parsing yourself. Python's
call/signature syntax is already extremely expressive, and resolving
call arguments to formal parameters is already a complicated (and
slow) process. Implementing functions with such strange argument
semantics is hardly common enough to justify the whole grouping syntax
proposed in this PEP. -1 to that. I think I can live with "/", but
YANGTNI still.



I am for having a way to succintly properly describe the signature of C in
the manual and docstrings and help output. As it is now, the only safe thing
to do, without trial and exception, is to assume positional only unless one
knows otherwise.


Having a nice syntax for the docs is quite different from implementing
it in the language.


It would be nice, however, to have it implemented at some point.

--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters

2013/10/8 Ethan Furman :
> On 10/08/2013 08:09 PM, Benjamin Peterson wrote:
>>
>> 2013/10/8 Terry Reedy :
>>>
>>> On 10/8/2013 9:31 PM, Benjamin Peterson wrote:


 2013/10/8 Larry Hastings :
>
>
> This PEP proposes a backwards-compatible syntax that should
> permit implementing any builtin in pure Python code.



 This is rather too strong. You can certainly implement them; you just
 have to implement the argument parsing yourself. Python's
 call/signature syntax is already extremely expressive, and resolving
 call arguments to formal parameters is already a complicated (and
 slow) process. Implementing functions with such strange argument
 semantics is hardly common enough to justify the whole grouping syntax
 proposed in this PEP. -1 to that. I think I can live with "/", but
 YANGTNI still.
>>>
>>>
>>>
>>> I am for having a way to succintly properly describe the signature of C
>>> in
>>> the manual and docstrings and help output. As it is now, the only safe
>>> thing
>>> to do, without trial and exception, is to assume positional only unless
>>> one
>>> knows otherwise.
>>
>>
>> Having a nice syntax for the docs is quite different from implementing
>> it in the language.
>
>
> It would be nice, however, to have it implemented at some point.

Why? It's quite complex and hardly useful.


-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters

2013-10-08 Thread Georg Brandl

Am 09.10.2013 01:33, schrieb Larry Hastings:
> 
> I've contributed a new PEP to humanity.  I include the RST for your reading
> pleasure below, but you can also read it online here:
> 
> http://www.python.org/dev/peps/pep-0457/

...

>* Documentation can clearly, unambiguously, and
>  consistently express exactly how the arguments
>  for a function will be interpreted.

I don't really like it from the docs perspective.  We have quite a large
amount of C functions with positional-only parameters.  Adding a "/"
to each of those is a) a tedious task and b) probably not helpful for
a lot of people:

For each user who tries to call a C function with keyword args and is
confused, there will be ten or twenty who are confused by the strange
"/" in the first place.  (We get quite a few questions about the *args
and **kwargs syntax already that is infrequently used, which is
otherwise _part of the language_.)

Rather, I would try to make as many C functions as possible "regular",
i.e. use PyArgs_ParseTupleAndKeywords or the Argument Clinic equivalent.
For the few that should be blazingly fast or have irregular signatures,
the documentation can be amended by a single sentence "This function
does not support passing keyword arguments." and that's the end of it.

>* The syntax is reserved for future use, in case
>  the community decides someday to add positional-only
>  parameters to the language.

For this purpose I'm also -1.  Function definition has become almost
too complex already with keyword-only arguments, although they at least
have usefulness other than mimicking the deficiencies of an implementation
detail...  and the "undefined" singleton just smells wrong.  Another way
to spell "None" is just asking for trouble.

>* Argument Clinic can use a variant of the syntax
>  as part of its input when defining
>  the arguments for built-in functions.

For AC it's good enough I guess :)  but that doesn't require a separate
PEP.  Also, the AC DSL can be changed much, much easier than the language
can be if this syntax is accepted.

cheers,
Georg

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] inspect() and dir()


On 10/08/2013 03:07 PM, Guido van Rossum wrote:


I'd say that the effect overriding dir() has on help() is intentional. Not sure 
about inspect -- it has other uses.
However, the last time I used it I was definitely hunting for stuff to document.


If want help to be effected that is easy enough -- there is already an inspect.classify_class_attrs wrapper, so if we 
add some post-processing to remove anything not in dir(obj) we would keep the current behavior in help().


Any other comments/recollections/concerns?

--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 457: Syntax For Positional-Only Parameters