Re: [Python-Dev] ssh://h...@hg.python.org/cpython unstable?

[Terry Reedy]

Since worked ok.

--
Terry Jan Reedy



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython: Issue #14769: test_capi now has SkipitemTest, which cleverly checks

[Larry Hastings]

On 06/22/2012 03:48 PM, MRAB wrote:

On 22/06/2012 21:45, Larry Hastings wrote:

On 06/22/2012 01:29 PM, Terry Reedy wrote:

Of course. And character 32  is also not usable and perhaps not
worth testing.


Au contraire!  I grant you, it's hard to imagine how using it would be a
good idea.  But strictly speaking it is *usable*.  (And what is this
thread about if not wanton pedantry!)

Therefore I'm leaving it in.  Feel free to go behind my back and fix it
if you feel strongly about this.


It _is_ a printable character after all.


Well, if we wanted to get *really* pedantic--and, who am I kidding, of 
course we do, this is python-dev--any byte besides 0 would *work* as a 
format unit, printable or not.  But who among us would dare?



//arry/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython: Issue #14769: test_capi now has SkipitemTest, which cleverly checks

[MRAB]

On 22/06/2012 21:45, Larry Hastings wrote:

On 06/22/2012 01:29 PM, Terry Reedy wrote:

Of course. And character 32  is also not usable and perhaps not
worth testing.


Au contraire!  I grant you, it's hard to imagine how using it would be a
good idea.  But strictly speaking it is *usable*.  (And what is this
thread about if not wanton pedantry!)

Therefore I'm leaving it in.  Feel free to go behind my back and fix it
if you feel strongly about this.


It _is_ a printable character after all.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A Desperate Plea For Introspection (aka: BDFAP Needed)

[Yury Selivanov]

On 2012-06-22, at 5:26 PM, Guido van Rossum wrote:

> I am accepting the PEP. Congrats Yuri! (And others who feel they deserve it. 
> :-)

Great!  Larry will merge the implementation then.

Larry, Brett and I worked on the PEP together (~200 emails in private 
discussions),
so everybody deserves ;)

-
Yury
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A Desperate Plea For Introspection (aka: BDFAP Needed)

[Guido van Rossum]

I am accepting the PEP. Congrats Yuri! (And others who feel they deserve it. :-)

On Fri, Jun 22, 2012 at 2:04 PM, Yury Selivanov  wrote:
> On 2012-06-22, at 4:58 PM, Ethan Furman wrote:
>> That looks strange to me -- I suggest putting brackets around each one, like:
>>
>>   replace(*, [name=,] [kind=,] [default=,] 
>> [annotation=]) -> Parameter
>
> Isn't it too much?  The PEP clearly indicates '=' is just
> a notation for an optional parameter.
>
> If it's that much of an issue - We can use '=_void' instead, as it is
> in the implementation, and describe how it works.  But that's just noise,
> that will distract the reader from the PEP.
>
>> which is still a bit noisy.  At the risk of raising the ire of those who 
>> don't condone the use of '...' :)  how about:
>>
>>   replace(*, [name=...,] [kind=...,] [default=...,] [annotation=...]
>>            ) -> Parameter
>
> That may confuse someone, as ... - Ellipsis is a legitimate object
> to be used as a default value or annotation:
>
>     def foo(bar:...=...)->...:
>         pass
>
> I'd keep it simple ;)

Please leave  in.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[Donald Stufft]

On Friday, June 22, 2012 at 4:55 PM, Terry Reedy wrote:
> 
> Every time windows users download and install a binary, they are taking 
> a chance. I try to use a bit more sense than some people, but I know it 
> is not risk free. There *is* a third party site that builds installers, 
> but should I trust it? I would prefer that (except perhaps for known and 
> trusted authors) PyPI compile binaries, perhaps after running code 
> through a security checker, followed by running it through one or more 
> virus checkers.
> 
I think you overestimate the abilities of "security checkers" and antivirus. 
Installing
from PyPI is a risk, wether you use source or binaries. There is currently not
a very good security story for installing python packages from PyPI (not all of 
this
falls on PyPI), but even if we get to a point there is, PyPI can never be as
safe as installing from RPM's or DEB and somewhat mores in the case of 
binaries. You
_have_ to make a case by case choice if you trust the authors/maintainers of a 
particular package. 

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A Desperate Plea For Introspection (aka: BDFAP Needed)

[Yury Selivanov]

On 2012-06-22, at 4:58 PM, Ethan Furman wrote:
> That looks strange to me -- I suggest putting brackets around each one, like:
> 
>   replace(*, [name=,] [kind=,] [default=,] 
> [annotation=]) -> Parameter

Isn't it too much?  The PEP clearly indicates '=' is just
a notation for an optional parameter.

If it's that much of an issue - We can use '=_void' instead, as it is 
in the implementation, and describe how it works.  But that's just noise,
that will distract the reader from the PEP.

> which is still a bit noisy.  At the risk of raising the ire of those who 
> don't condone the use of '...' :)  how about:
> 
>   replace(*, [name=...,] [kind=...,] [default=...,] [annotation=...]
>) -> Parameter

That may confuse someone, as ... - Ellipsis is a legitimate object
to be used as a default value or annotation:

 def foo(bar:...=...)->...:
 pass

I'd keep it simple ;)

-
Yury
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A Desperate Plea For Introspection (aka: BDFAP Needed)

[Ethan Furman]

Yury Selivanov wrote:

On 2012-06-22, at 3:47 PM, Ethan Furman wrote:


Guido van Rossum wrote:

On Fri, Jun 22, 2012 at 12:24 PM, Yury Selivanov
 wrote:

On 2012-06-22, at 3:18 PM, Guido van Rossum wrote:


On Fri, Jun 22, 2012 at 12:10 PM, Yury Selivanov
 wrote:

Yes, I meant optional.  Would 'name(arg1, *, [arg2])' be better?

Hardly, because that's not valid syntax. I'd write name(arg1, *,
arg2=).

Like

  replace(*, name=, kind=, default=,
annotation=) -> Parameter

or

  replace(*, name=, kind=, default=,
annotation=) -> Parameter

Either one's an improvement, but you'll have to explain at the top of
the PEP what you intend this notation to mean. I'd go with 
since the key thing here seems to be that various keywords, when not
specified, mean that nothing changes. OTOH in some places you can
probably write "foo=Signature.empty" (etc.).

Parameter names that follow '*' in the signature are not optional (unless that 
has changed since 3.2).  In other words, the above signature requires that 
name, kind, default, and annotation be specified by name *and* be given values 
when replace is called)


I know.  Those are optional keyword-only arguments.

In the code:

def replace(self, *, name=_void, kind=_void, annotation=_void,
default=_void):

We just need some clear convention for the PEP - and the 
mark should work.


That looks strange to me -- I suggest putting brackets around each one, 
like:


   replace(*, [name=,] [kind=,] 
[default=,] [annotation=]) -> Parameter


which is still a bit noisy.  At the risk of raising the ire of those who 
don't condone the use of '...' :)  how about:


   replace(*, [name=...,] [kind=...,] [default=...,] [annotation=...]
) -> Parameter

~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[Terry Reedy]

On 6/22/2012 6:09 AM, Vinay Sajip wrote:


Easy enough on Posix platforms, perhaps, but what about Windows?


Every time windows users download and install a binary, they are taking 
a chance. I try to use a bit more sense than some people, but I know it 
is not risk free. There *is* a third party site that builds installers, 
but should I trust it? I would prefer that (except perhaps for known and 
trusted authors) PyPI compile binaries, perhaps after running code 
through a security checker, followed by running it through one or more 
virus checkers.



One can't expect a C compiler to be installed everywhere.


Just having 'a C compiler' is not enough to compile on Windows.

--
Terry Jan Reedy



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A Desperate Plea For Introspection (aka: BDFAP Needed)

[Yury Selivanov]

On 2012-06-22, at 3:25 PM, Christian Heimes wrote:

> Am 22.06.2012 21:10, schrieb Yury Selivanov:
>> I think that if a function lacks an annotation, that should be reflected
>> in the same way for its signature.
>> 
>> Currently:
>> 
>>if hasattr(signature, 'return_annotation'):
>> 
>> If we use Signature.empty:
>> 
>>if signature.return_annotation is not signature.empty:
>> 
>> So (in my humble opinion) it doesn't simplify things too much.
>> And also you can use 'try .. except AttributeError .. else' blocks,
>> which make code even more readable.
> 
> The second form has two benefits:
> 
> * you get a sensible error message when you mistype the name of the
> attribute. hasattr(signature, 'return_annotatoin') is clearly an error,
> hard to notice with the naked eye and passes silently.
> 
> * modern Python IDEs have code completion. "signature.re is not
> signature.em" safes key strokes.

Agree on both.

This change also cut 20 lines from the implementation.
So I guess it is a good decision after all ;)

-
Yury
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython: Issue #14769: test_capi now has SkipitemTest, which cleverly checks

[Larry Hastings]

On 06/22/2012 01:29 PM, Terry Reedy wrote:
Of course. And character 32  is also not usable and perhaps not 
worth testing.


Au contraire!  I grant you, it's hard to imagine how using it would be a 
good idea.  But strictly speaking it is *usable*.  (And what is this 
thread about if not wanton pedantry!)


Therefore I'm leaving it in.  Feel free to go behind my back and fix it 
if you feel strongly about this.



//arry/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A Desperate Plea For Introspection (aka: BDFAP Needed)

[Yury Selivanov]

On 2012-06-22, at 4:20 PM, Terry Reedy wrote:

> On 6/22/2012 3:24 PM, Yury Selivanov wrote:
>> On 2012-06-22, at 3:18 PM, Guido van Rossum wrote:
> 
>>> Hardly, because that's not valid syntax. I'd write name(arg1, *,
>>> arg2=).
>> 
>> Like
>> 
>> replace(*, name=, kind=, default=,
>>   annotation=) -> Parameter
>> 
>> or
>> 
>> replace(*, name=, kind=, default=,
>>   annotation=) -> Parameter
> 
> I do not undertand the 'or'. I hope you mean default argument expressions in 
> the standard manner rather than '' or ''.

It's now in the updated PEP.  We use '' notation
just to specify that the parameter is optional, i.e. can
(or in some cases should) be omitted.

-
Yury
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] removing packaging

[Antoine Pitrou]

On Tue, 19 Jun 2012 17:46:30 -0400
Éric Araujo  wrote:
> 
>With beta coming, a way to deal with that unfortunate situation needs 
> to be found.  We could (a) grant an exception to packaging to allow 
> changes after beta1; (b) keep packaging as it is now under a provisional 
> status, with due warnings that many things are expected to change; (c) 
> remove the unstable parts and deliver a subset that works (proposed by 
> Tarek to the Pyramid author on distutils-sig); (d) not release packaging 
> as part of Python 3.3 (I think that was also suggested on distutils-sig 
> last month).
> 
>I don’t think (a) would give us enough time; we really want a few 
> months (and releases) to hash out the API (most notably with the pip and 
> buildout developers) and clean the bdist situation.  Likewise (c) would 
> require developer (my) time that is currently in short supply.  (b) also 
> requires time and would make development harder, not to mention probable 
> user pain.  This leaves (d), after long reflection, as my preferred 
> choice, even though I disliked the idea at first (and I fully expect 
> Tarek to feel the same way).

So if you want to remove packaging from 3.3, it should be before the
beta which will be out in a few days.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[David Cournapeau]

On Fri, Jun 22, 2012 at 9:11 PM, PJ Eby  wrote:
> On Fri, Jun 22, 2012 at 5:22 AM, Dag Sverre Seljebotn
>  wrote:
>>
>> On 06/22/2012 10:40 AM, Paul Moore wrote:
>>>
>>> On 22 June 2012 06:05, Nick Coghlan  wrote:

 distutils really only plays at the SRPM level - there is no defined OS
 neutral RPM equivalent. That's why I brought up the bdist_simple
 discussion earlier in the thread - if we can agree on a standard
 bdist_simple format, then we can more cleanly decouple the "build"
 step from the "install" step.
>>>
>>>
>>> That was essentially the key insight I was trying to communicate in my
>>> "think about the end users" comment. Thanks, Nick!
>>
>>
>> The subtlety here is that there's no way to know before building the
>> package what files should be installed. (For simple extensions, and perhaps
>> documentation, you could get away with ad-hoc rules or special support for
>> Sphinx and what-not, but there's no general solution that works in all
>> cases.)
>>
>> What Bento does is have one metadata file for the source-package, and
>> another metadata file (manifest) for the built-package. The latter is
>> normally generated by the build process (but follows a standard
>> nevertheless). Then that manifest is used for installation (through several
>> available methods).
>
>
> This is the right thing to do, IMO.
>
> Also, I think rather than bikeshedding the One Serialization To Rule Them
> All, it should only be the *built* manifest that is standardized for tool
> consumption, and leave source descriptions to end-user tools.  setup.cfg,
> bento.info, or whatever...  that part should NOT be the first thing
> designed, and should not be the part that's frozen in a spec, since it
> otherwise locks out the ability to enhance that format.

agreed. I may not have been very clear before, but the bento.info
format is really peripherical to what bento is about (it just happens
that  what would become bento was started as a 2 hours proof of
concept for another packaging discussion 3 years ago :) ).

As for the build manifest, I have a few, very out-dated notes there:

http://cournape.github.com/Bento/html/hacking.html#build-manifest-and-building-installers

I will try to update them this WE. I do have code to install, produce
eggs, msi, .exe and .mpkg from this format. The API is kind of
crappy/inconsistent, but the features are there, and there are even
some tests around it. I don't think it would be very difficult to hack
distutils2 to produce this build manifest.

David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython: Issue #14769: test_capi now has SkipitemTest, which cleverly checks

[Terry Reedy]

On 6/22/2012 2:21 PM, MRAB wrote:

On 22/06/2012 17:39, Terry Reedy wrote:

On 6/22/2012 6:57 AM, larry.hastings wrote:

http://hg.python.org/cpython/rev/ace45d23628a
changeset:   77567:ace45d23628a
user:Larry Hastings 
date:Fri Jun 22 03:56:29 2012 -0700
summary:
   Issue #14769: test_capi now has SkipitemTest, which cleverly checks
for "parity" between PyArg_ParseTuple() and the Python/getargs.c static
function skipitem() for all possible "format units".


You sensibly only test printable ascii chars, which are in the
contiguous range 32 to 127 inclusive. So it makes no sense to claim
otherwise and then deny the wrong claim, or to enlarge the range and
then shrink it again.


ASCII character 127 is a control character, not a printable character.


Of course. And character 32  is also not usable and perhaps not 
worth testing.



+This function brute-force tests all** ASCII characters (1 to
127
+inclusive) as format units, checking to see that



There are 128 ASCII characters (0 to 127 inclusive).


With a few exceptions**, test all printable ASCII characters (32 to 127
inclusive) as...


+
+  ** Okay, it actually skips some ASCII characters.  Some
characters


** Some characters ...

+ have special funny semantics, and it would be difficult to
+ accomodate them here.



+for i in range(1, 128):


for i in range(32, 128):


so,
for i in range(32, 127):
or
for i in range(33, 127):


+if (not c.isprintable()) or (c in '()e|$'):


if c in '()e|$':




--
Terry Jan Reedy



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A Desperate Plea For Introspection (aka: BDFAP Needed)

[Terry Reedy]

On 6/22/2012 3:24 PM, Yury Selivanov wrote:

On 2012-06-22, at 3:18 PM, Guido van Rossum wrote:



Hardly, because that's not valid syntax. I'd write name(arg1, *,
arg2=).


Like

 replace(*, name=, kind=, default=,
   annotation=) -> Parameter

or

 replace(*, name=, kind=, default=,
   annotation=) -> Parameter


I do not undertand the 'or'. I hope you mean default argument 
expressions in the standard manner rather than '' or ''.


--
Terry Jan Reedy



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A Desperate Plea For Introspection (aka: BDFAP Needed)

[Yury Selivanov]

On 2012-06-22, at 3:18 PM, Guido van Rossum wrote:

> On Fri, Jun 22, 2012 at 12:10 PM, Yury Selivanov
>  wrote:
>> Guido,
>> 
>> On 2012-06-22, at 2:52 PM, Guido van Rossum wrote:
>>> Of these, only (1) is a blocker for PEP acceptance -- I'd either like
>>> to see this defended vigorously (maybe it was discussed? then please
>>> quote, I can't go back and read the threads) or changed.
>>> 
>>> Otherwise it looks great!

OK, we've updated the PEP: http://www.python.org/dev/peps/pep-0362/
( and the implementation: http://bugs.python.org/issue15008 )

Please take a look.

-
Yury

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[PJ Eby]

On Fri, Jun 22, 2012 at 5:22 AM, Dag Sverre Seljebotn <
d.s.seljeb...@astro.uio.no> wrote:

> On 06/22/2012 10:40 AM, Paul Moore wrote:
>
>> On 22 June 2012 06:05, Nick Coghlan  wrote:
>>
>>> distutils really only plays at the SRPM level - there is no defined OS
>>> neutral RPM equivalent. That's why I brought up the bdist_simple
>>> discussion earlier in the thread - if we can agree on a standard
>>> bdist_simple format, then we can more cleanly decouple the "build"
>>> step from the "install" step.
>>>
>>
>> That was essentially the key insight I was trying to communicate in my
>> "think about the end users" comment. Thanks, Nick!
>>
>
> The subtlety here is that there's no way to know before building the
> package what files should be installed. (For simple extensions, and perhaps
> documentation, you could get away with ad-hoc rules or special support for
> Sphinx and what-not, but there's no general solution that works in all
> cases.)
>
> What Bento does is have one metadata file for the source-package, and
> another metadata file (manifest) for the built-package. The latter is
> normally generated by the build process (but follows a standard
> nevertheless). Then that manifest is used for installation (through several
> available methods).
>

This is the right thing to do, IMO.

Also, I think rather than bikeshedding the One Serialization To Rule Them
All, it should only be the *built* manifest that is standardized for tool
consumption, and leave source descriptions to end-user tools.  setup.cfg,
bento.info, or whatever...  that part should NOT be the first thing
designed, and should not be the part that's frozen in a spec, since it
otherwise locks out the ability to enhance that format.

There's also been a huge amount of previous discussion regarding setup.cfg,
which anyone proposing to alter it should probably read.  setup.cfg allows
hooks to external systems, so IIUC,  you should be able to write a
setup.cfg file that contains little besides your publication metadata
(name, version, dependencies) and a hook to invoke whatever build tools you
want, as long as you're willing to write a Python hook.

This means that bikeshedding the build process is totally beside the
point.  If people want to use distutils, bento, SCons, ...  it really
doesn't matter, as long as they're willing to write a hook.  This is a
killer app for "packaging", as it frees up the stdlib from having to do
every bloody thing itself and create One Build Process To Rule Them All.

I didn't invent setup.cfg or write the "packaging" code, but I support this
design approach wholeheartedly.  I have only the smallest of quibbles and
questions with it, but they aren't showstoppers.  I've already had some
discussion about these points on Distutils-SIG, and I think that should be
continued.

If there *is* to be any major discussion about switching directions in
packaging, the place to start should be *use cases* rather than file
formats.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] ssh://h...@hg.python.org/cpython unstable?

[Terry Reedy]

90% of the way through recloning cpython on Win7, I got
Putty Error: Network error: software called connection abort
Tortoise hg said
abort: stream ended unexpectedly (got 53602 bytes, expected 55236)

Two retries give same Putty error almost immdiately, with hg message
no suitable response from remote hg

Third retry started download, but it failed again after 10 minutes.
similar message from hg.
Is
ssh://h...@hg.python.org/cpython unstable (intermittantly down)?
or is something strange happening on my system?
(I have previously downloaded multi-gigabyte files, though not today and 
with perhaps more error tolerant downloaders.)


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Feature Freeze

[Terry Reedy]

On 6/22/2012 3:00 PM, Ethan Furman wrote:


can somebody review (and commit! :) issues:

http://bugs.python.org/issue14954


About weakref, no response yet.
Beyond my knowledge.


http://bugs.python.org/issue14617


About __hash__, a short response from Éric Araujo
I might look at this.

--
Terry Jan Reedy




___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A reference leak with PyType_FromSpec

[Antoine Pitrou]

On Fri, 22 Jun 2012 21:23:10 +0200
Antoine Pitrou  wrote:
> 
> Hi,
> 
> As mentioned in the commit message for 96513d71e650, creating a type
> using PyType_FromSpec seems to leak references when the type is
> instantiated. This happens with SSLError:

The patch in http://bugs.python.org/issue15142 seems to fix it.
Feedback welcome from typeobject experts :)

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A Desperate Plea For Introspection (aka: BDFAP Needed)

[Yury Selivanov]

On 2012-06-22, at 3:47 PM, Ethan Furman wrote:

> Guido van Rossum wrote:
>> On Fri, Jun 22, 2012 at 12:24 PM, Yury Selivanov
>>  wrote:
>>> On 2012-06-22, at 3:18 PM, Guido van Rossum wrote:
>>> 
 On Fri, Jun 22, 2012 at 12:10 PM, Yury Selivanov
  wrote:
> Yes, I meant optional.  Would 'name(arg1, *, [arg2])' be better?
 Hardly, because that's not valid syntax. I'd write name(arg1, *,
 arg2=).
>>> Like
>>> 
>>>   replace(*, name=, kind=, default=,
>>> annotation=) -> Parameter
>>> 
>>> or
>>> 
>>>   replace(*, name=, kind=, default=,
>>> annotation=) -> Parameter
>> Either one's an improvement, but you'll have to explain at the top of
>> the PEP what you intend this notation to mean. I'd go with 
>> since the key thing here seems to be that various keywords, when not
>> specified, mean that nothing changes. OTOH in some places you can
>> probably write "foo=Signature.empty" (etc.).
> 
> Parameter names that follow '*' in the signature are not optional (unless 
> that has changed since 3.2).  In other words, the above signature requires 
> that name, kind, default, and annotation be specified by name *and* be given 
> values when replace is called)

I know.  Those are optional keyword-only arguments.

In the code:

def replace(self, *, name=_void, kind=_void, annotation=_void,
default=_void):

We just need some clear convention for the PEP - and the 
mark should work.

-
Yury
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A Desperate Plea For Introspection (aka: BDFAP Needed)

[Christian Heimes]

Am 22.06.2012 21:32, schrieb Larry Hastings:
> 
> On 06/22/2012 12:21 PM, Christian Heimes wrote:
>> The PEP is already complex enough and went to several incarnations. It
>> was a wise decision to focus on the features that could be implemented
>> before the first beta is released. Kudos for pulling it off, Larry!
> 
> Guys, guys!  I have done next-to-nothing on this PEP.  Yury has
> absolutely been the one driving it since it rose from the dead this
> year.  All the kudos should be addressed to him.

No kudos for you then and all kodus to Yury. Sorry for the
misconception, Yury! :)

Larry, you shouldn't underrate your role for the PEP on the Python dev
list. You did a fine job managing the 'political' side over the past
days and weeks.

Christian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A Desperate Plea For Introspection (aka: BDFAP Needed)

[Ethan Furman]

Guido van Rossum wrote:

On Fri, Jun 22, 2012 at 12:24 PM, Yury Selivanov
 wrote:

On 2012-06-22, at 3:18 PM, Guido van Rossum wrote:


On Fri, Jun 22, 2012 at 12:10 PM, Yury Selivanov
 wrote:

Yes, I meant optional.  Would 'name(arg1, *, [arg2])' be better?

Hardly, because that's not valid syntax. I'd write name(arg1, *,
arg2=).

Like

   replace(*, name=, kind=, default=,
 annotation=) -> Parameter

or

   replace(*, name=, kind=, default=,
 annotation=) -> Parameter


Either one's an improvement, but you'll have to explain at the top of
the PEP what you intend this notation to mean. I'd go with 
since the key thing here seems to be that various keywords, when not
specified, mean that nothing changes. OTOH in some places you can
probably write "foo=Signature.empty" (etc.).


Parameter names that follow '*' in the signature are not optional 
(unless that has changed since 3.2).  In other words, the above 
signature requires that name, kind, default, and annotation be specified 
by name *and* be given values when replace is called)


~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython: Issue #14769: test_capi now has SkipitemTest, which cleverly checks

[Larry Hastings]

On 06/22/2012 11:21 AM, MRAB wrote:

On 22/06/2012 17:39, Terry Reedy wrote:

You sensibly only test printable ascii chars, which are in the
contiguous range 32 to 127 inclusive. So it makes no sense to claim
otherwise and then deny the wrong claim, or to enlarge the range and
then shrink it again.

ASCII character 127 is a control character, not a printable character.

+This function brute-force tests all** ASCII characters (1 
to 127

+inclusive) as format units, checking to see that



There are 128 ASCII characters (0 to 127 inclusive).


Okay, message received.  I'll test from 32 to 126 inclusive.  I'm going 
to be obnoxious and code those values straight in--which I concede will 
be a maintenance nightmare should the ASCII standard change.



//arry/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A Desperate Plea For Introspection (aka: BDFAP Needed)

[Larry Hastings]

On 06/22/2012 12:21 PM, Christian Heimes wrote:

The PEP is already complex enough and went to several incarnations. It
was a wise decision to focus on the features that could be implemented
before the first beta is released. Kudos for pulling it off, Larry!


Guys, guys!  I have done next-to-nothing on this PEP.  Yury has 
absolutely been the one driving it since it rose from the dead this 
year.  All the kudos should be addressed to him.


The only feature I contributed is the one everyone savagely *hated* (the 
"is_implemented" attribute).  That was taken out and shot last week.  ;-)



Signatures for builtin functions should be handled by a new PEP. We need
a way to extract or define the signatures (perhaps parse the C code and
parse PyArg_* signatures) and a secure way to store the signature
(perhaps implement the signature class in C?). That's a LOT of work.


I'm growing something for 3.4; I hope to preview it in the next month or 
two.


Does it really need a new PEP, though?  It's an implementation detail 
for CPython; it shouldn't affect Python-level interfaces one jot.  It's 
essentially a replacement for PyArg_ParseTupleAndKeywords()... I didn't 
think that merited a PEP.


Cheers,


//arry/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A Desperate Plea For Introspection (aka: BDFAP Needed)

[Guido van Rossum]

On Fri, Jun 22, 2012 at 12:24 PM, Yury Selivanov
 wrote:
> On 2012-06-22, at 3:18 PM, Guido van Rossum wrote:
>
>> On Fri, Jun 22, 2012 at 12:10 PM, Yury Selivanov
>>  wrote:
>>> Guido,
>>>
>>> On 2012-06-22, at 2:52 PM, Guido van Rossum wrote:
> ...
>>> 'empty' will also work.  When python-dev collectively decided to
>>> go with missing attributes, 'empty' didn't yet exist (we added
>>> it with 'replace()' methods).
>>>
>>> If you think that using 'empty' is better, we can add that to the PEP.
>>
>> Yes, please do.
>
> OK
>
 (2) Could use an example on how to remove and add parameters using 
 replace().
>>>
>>> You have to build a new list of parameters and then pass it to 'replace()'.
>>> Example (from the actual signature() function implementation):
>>>
>>>    if isinstance(obj, types.MethodType):
>>>        # In this case we skip the first parameter of the underlying
>>>        # function (usually `self` or `cls`).
>>>        sig = signature(obj.__func__)
>>>        return sig.replace(parameters=tuple(sig.parameters.values())[1:])
>>>
 (3) You are using name(arg1, *, arg2) a lot. I think in most cases you
 mean for arg2 to be an optional keyword arg, but this notation doesn't
 convey that it is optional. Can you clarify?
>>>
>>> Yes, I meant optional.  Would 'name(arg1, *, [arg2])' be better?
>>
>> Hardly, because that's not valid syntax. I'd write name(arg1, *,
>> arg2=).
>
> Like
>
>    replace(*, name=, kind=, default=,
>                                  annotation=) -> Parameter
>
> or
>
>    replace(*, name=, kind=, default=,
>                                  annotation=) -> Parameter

Either one's an improvement, but you'll have to explain at the top of
the PEP what you intend this notation to mean. I'd go with 
since the key thing here seems to be that various keywords, when not
specified, mean that nothing changes. OTOH in some places you can
probably write "foo=Signature.empty" (etc.).

 (4) "If the object is a method" -- shouldn't that be "bound method"?
 (Unbound methods are undetectable.) Or is there some wider definition
 of method? What does it do for static or class methods?
>>>
>>> Yes, it should be "If the object is a bound method".  We'll fix this
>>> shortly.
>>
>> Great.
>>
>>> classmethod as a descriptor returns a BoundMethod (bound to the class),
>>> staticmethod returns the original unmodified function, so both of
>>> them are supported automatically.
>>
>> Oh, great. IIRC it wasn't always like that. Maybe just add this to the
>> PEP as a note?
>
> OK.  I'll clarify that.
>
> -
> Yury



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] A reference leak with PyType_FromSpec

[Antoine Pitrou]

Hi,

As mentioned in the commit message for 96513d71e650, creating a type
using PyType_FromSpec seems to leak references when the type is
instantiated. This happens with SSLError:

>>> e = ssl.SSLError()
>>> sys.getrefcount(ssl.SSLError)  
35
>>> e = ssl.SSLError()
>>> sys.getrefcount(ssl.SSLError)  
36
>>> e = ssl.SSLError()
>>> sys.getrefcount(ssl.SSLError)  
37

(the SSLError definition is quite simple; it only uses the
Py_tp_base, Py_tp_doc and Py_tp_str slots)

The SSLError subclasses, e.g. SSLWantReadError, which are created using
PyErr_NewExceptionWithDoc, are not affected:

>>> e = ssl.SSLWantReadError()
>>> sys.getrefcount(ssl.SSLWantReadError)
8
>>> e = ssl.SSLWantReadError()
>>> sys.getrefcount(ssl.SSLWantReadError)
8
>>> e = ssl.SSLWantReadError()
>>> sys.getrefcount(ssl.SSLWantReadError)
8

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A Desperate Plea For Introspection (aka: BDFAP Needed)

[Christian Heimes]

Am 22.06.2012 21:10, schrieb Yury Selivanov:
> I think that if a function lacks an annotation, that should be reflected
> in the same way for its signature.
> 
> Currently:
> 
> if hasattr(signature, 'return_annotation'):
> 
> If we use Signature.empty:
> 
> if signature.return_annotation is not signature.empty:
> 
> So (in my humble opinion) it doesn't simplify things too much.
> And also you can use 'try .. except AttributeError .. else' blocks,
> which make code even more readable.

The second form has two benefits:

 * you get a sensible error message when you mistype the name of the
attribute. hasattr(signature, 'return_annotatoin') is clearly an error,
hard to notice with the naked eye and passes silently.

* modern Python IDEs have code completion. "signature.re is not
signature.em" safes key strokes.

Christian

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A Desperate Plea For Introspection (aka: BDFAP Needed)

[Guido van Rossum]

On Fri, Jun 22, 2012 at 12:21 PM, Christian Heimes  wrote:
> Am 22.06.2012 20:52, schrieb Guido van Rossum:
>> (5) Too bad there's no proposal for adding signatures to builtin
>> functions/methods, but understood.
>
> Larry et al. did an experiment with a mutable __signature__ attribute to
> PyCFunction. He immediately backed out and removed the attribute as I
> explained that it breaks isolation between subinterpreter instances.

Good point. Maybe the PEP could explain this (remember that a good PEP
also mentioned some rejected ideas and the reason why they were
rejected).

> The PEP is already complex enough and went to several incarnations. It
> was a wise decision to focus on the features that could be implemented
> before the first beta is released. Kudos for pulling it off, Larry!

Indeed, limiting the scope in this way was very wise.

> Signatures for builtin functions should be handled by a new PEP. We need
> a way to extract or define the signatures (perhaps parse the C code and
> parse PyArg_* signatures) and a secure way to store the signature
> (perhaps implement the signature class in C?). That's a LOT of work.

Agreed it's an open problem. I just hope someone will tackle it next.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A Desperate Plea For Introspection (aka: BDFAP Needed)

[Yury Selivanov]

On 2012-06-22, at 3:18 PM, Guido van Rossum wrote:

> On Fri, Jun 22, 2012 at 12:10 PM, Yury Selivanov
>  wrote:
>> Guido,
>> 
>> On 2012-06-22, at 2:52 PM, Guido van Rossum wrote:
...
>> 'empty' will also work.  When python-dev collectively decided to
>> go with missing attributes, 'empty' didn't yet exist (we added
>> it with 'replace()' methods).
>> 
>> If you think that using 'empty' is better, we can add that to the PEP.
> 
> Yes, please do.

OK

>>> (2) Could use an example on how to remove and add parameters using 
>>> replace().
>> 
>> You have to build a new list of parameters and then pass it to 'replace()'.
>> Example (from the actual signature() function implementation):
>> 
>>if isinstance(obj, types.MethodType):
>># In this case we skip the first parameter of the underlying
>># function (usually `self` or `cls`).
>>sig = signature(obj.__func__)
>>return sig.replace(parameters=tuple(sig.parameters.values())[1:])
>> 
>>> (3) You are using name(arg1, *, arg2) a lot. I think in most cases you
>>> mean for arg2 to be an optional keyword arg, but this notation doesn't
>>> convey that it is optional. Can you clarify?
>> 
>> Yes, I meant optional.  Would 'name(arg1, *, [arg2])' be better?
> 
> Hardly, because that's not valid syntax. I'd write name(arg1, *,
> arg2=).

Like 

replace(*, name=, kind=, default=, 
  annotation=) -> Parameter

or

replace(*, name=, kind=, default=, 
  annotation=) -> Parameter

>>> (4) "If the object is a method" -- shouldn't that be "bound method"?
>>> (Unbound methods are undetectable.) Or is there some wider definition
>>> of method? What does it do for static or class methods?
>> 
>> Yes, it should be "If the object is a bound method".  We'll fix this
>> shortly.
> 
> Great.
> 
>> classmethod as a descriptor returns a BoundMethod (bound to the class),
>> staticmethod returns the original unmodified function, so both of
>> them are supported automatically.
> 
> Oh, great. IIRC it wasn't always like that. Maybe just add this to the
> PEP as a note?

OK.  I'll clarify that.

-
Yury
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A Desperate Plea For Introspection (aka: BDFAP Needed)

[Christian Heimes]

Am 22.06.2012 20:52, schrieb Guido van Rossum:
> (5) Too bad there's no proposal for adding signatures to builtin
> functions/methods, but understood.

Larry et al. did an experiment with a mutable __signature__ attribute to
PyCFunction. He immediately backed out and removed the attribute as I
explained that it breaks isolation between subinterpreter instances.

The PEP is already complex enough and went to several incarnations. It
was a wise decision to focus on the features that could be implemented
before the first beta is released. Kudos for pulling it off, Larry!

Signatures for builtin functions should be handled by a new PEP. We need
a way to extract or define the signatures (perhaps parse the C code and
parse PyArg_* signatures) and a secure way to store the signature
(perhaps implement the signature class in C?). That's a LOT of work.

Christian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A Desperate Plea For Introspection (aka: BDFAP Needed)

[Guido van Rossum]

On Fri, Jun 22, 2012 at 12:10 PM, Yury Selivanov
 wrote:
> Guido,
>
> On 2012-06-22, at 2:52 PM, Guido van Rossum wrote:
>
>> This looks great, much better than the version I reviewed half a year
>> ago! Thanks you and others (especially Yuri) for all your efforts in
>> guiding the discussion and implementing as the discussion went along;
>> also thanks to Nick for participating to intensely.
>>
>> Quick review notes:
>>
>> (1) I don't like APIs that require you to use hasattr(), which you use
>> for return_annotation. Since you have Signature.empty to remove it,
>> why not set it to Signature.empty when not set? Ditto for other values
>> that are currently not set when they don't apply (Parameter.empty I
>> think.)
>
> I think that if a function lacks an annotation, that should be reflected
> in the same way for its signature.
>
> Currently:
>
>    if hasattr(signature, 'return_annotation'):
>
> If we use Signature.empty:
>
>    if signature.return_annotation is not signature.empty:
>
> So (in my humble opinion) it doesn't simplify things too much.

But it is much more of a pain if someone just wants to pass the value
on or print it. Instead of

  print(sig.return_annotation)

you'd have to write

  if hasattr(sig, 'return_annotation'):
print(sig.return_annotation)
  else:
print('empty')

In general hasattr() (or getattr() with a 3rd arg) has a code smell;
making it a required part of an API truly stinks IMO.

> And also you can use 'try .. except AttributeError .. else' blocks,
> which make code even more readable.

No, try/except blocks make code *less* readable.

> All in all, I don't have a very strong opinion in this topic.

But I do. :-)

> 'empty' will also work.  When python-dev collectively decided to
> go with missing attributes, 'empty' didn't yet exist (we added
> it with 'replace()' methods).
>
> If you think that using 'empty' is better, we can add that to the PEP.

Yes, please do.

>> (2) Could use an example on how to remove and add parameters using replace().
>
> You have to build a new list of parameters and then pass it to 'replace()'.
> Example (from the actual signature() function implementation):
>
>    if isinstance(obj, types.MethodType):
>        # In this case we skip the first parameter of the underlying
>        # function (usually `self` or `cls`).
>        sig = signature(obj.__func__)
>        return sig.replace(parameters=tuple(sig.parameters.values())[1:])
>
>> (3) You are using name(arg1, *, arg2) a lot. I think in most cases you
>> mean for arg2 to be an optional keyword arg, but this notation doesn't
>> convey that it is optional. Can you clarify?
>
> Yes, I meant optional.  Would 'name(arg1, *, [arg2])' be better?

Hardly, because that's not valid syntax. I'd write name(arg1, *,
arg2=).

>> (4) "If the object is a method" -- shouldn't that be "bound method"?
>> (Unbound methods are undetectable.) Or is there some wider definition
>> of method? What does it do for static or class methods?
>
> Yes, it should be "If the object is a bound method".  We'll fix this
> shortly.

Great.

> classmethod as a descriptor returns a BoundMethod (bound to the class),
> staticmethod returns the original unmodified function, so both of
> them are supported automatically.

Oh, great. IIRC it wasn't always like that. Maybe just add this to the
PEP as a note?

>> (5) Too bad there's no proposal for adding signatures to builtin
>> functions/methods, but understood.
>>
>> Of these, only (1) is a blocker for PEP acceptance -- I'd either like
>> to see this defended vigorously (maybe it was discussed? then please
>> quote, I can't go back and read the threads) or changed.
>>
>> Otherwise it looks great!
>>
>> --Guido
>
> Thanks!

Thank *you* for your hard work!


-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A Desperate Plea For Introspection (aka: BDFAP Needed)

[Yury Selivanov]

Guido,

On 2012-06-22, at 2:52 PM, Guido van Rossum wrote:

> This looks great, much better than the version I reviewed half a year
> ago! Thanks you and others (especially Yuri) for all your efforts in
> guiding the discussion and implementing as the discussion went along;
> also thanks to Nick for participating to intensely.
> 
> Quick review notes:
> 
> (1) I don't like APIs that require you to use hasattr(), which you use
> for return_annotation. Since you have Signature.empty to remove it,
> why not set it to Signature.empty when not set? Ditto for other values
> that are currently not set when they don't apply (Parameter.empty I
> think.)

I think that if a function lacks an annotation, that should be reflected
in the same way for its signature.

Currently:

if hasattr(signature, 'return_annotation'):

If we use Signature.empty:

if signature.return_annotation is not signature.empty:

So (in my humble opinion) it doesn't simplify things too much.
And also you can use 'try .. except AttributeError .. else' blocks,
which make code even more readable.

All in all, I don't have a very strong opinion in this topic.
'empty' will also work.  When python-dev collectively decided to 
go with missing attributes, 'empty' didn't yet exist (we added 
it with 'replace()' methods).

If you think that using 'empty' is better, we can add that to the PEP.

> (2) Could use an example on how to remove and add parameters using replace().

You have to build a new list of parameters and then pass it to 'replace()'.
Example (from the actual signature() function implementation):

if isinstance(obj, types.MethodType):
# In this case we skip the first parameter of the underlying
# function (usually `self` or `cls`).
sig = signature(obj.__func__)
return sig.replace(parameters=tuple(sig.parameters.values())[1:])

> (3) You are using name(arg1, *, arg2) a lot. I think in most cases you
> mean for arg2 to be an optional keyword arg, but this notation doesn't
> convey that it is optional. Can you clarify?

Yes, I meant optional.  Would 'name(arg1, *, [arg2])' be better?

> (4) "If the object is a method" -- shouldn't that be "bound method"?
> (Unbound methods are undetectable.) Or is there some wider definition
> of method? What does it do for static or class methods?

Yes, it should be "If the object is a bound method".  We'll fix this
shortly.

classmethod as a descriptor returns a BoundMethod (bound to the class),
staticmethod returns the original unmodified function, so both of
them are supported automatically.

> (5) Too bad there's no proposal for adding signatures to builtin
> functions/methods, but understood.
> 
> Of these, only (1) is a blocker for PEP acceptance -- I'd either like
> to see this defended vigorously (maybe it was discussed? then please
> quote, I can't go back and read the threads) or changed.
> 
> Otherwise it looks great!
> 
> --Guido

Thanks!

-
Yury
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Feature Freeze

[martin]

Zitat von Ethan Furman :


Does the feature freeze affect documentation enhancements?


No. Incorrect/missing documentation is always a bug (unless there
is a debate whether something is an implementation detail or a
language feature); bugs can be fixed at any time, and documentation
bugs in particular in an ongoing manner even after the release.

Thanks!  (And if not necessary before the feature freeze, sorry for  
the noise.)


Pushing issues is certainly on-topic for the list.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Feature Freeze

[Ethan Furman]

Does the feature freeze affect documentation enhancements?  If it does, 
can somebody review (and commit! :) issues:


http://bugs.python.org/issue14954

http://bugs.python.org/issue14617

Thanks!  (And if not necessary before the feature freeze, sorry for the 
noise.)


~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A Desperate Plea For Introspection (aka: BDFAP Needed)

[Guido van Rossum]

This looks great, much better than the version I reviewed half a year
ago! Thanks you and others (especially Yuri) for all your efforts in
guiding the discussion and implementing as the discussion went along;
also thanks to Nick for participating to intensely.

Quick review notes:

(1) I don't like APIs that require you to use hasattr(), which you use
for return_annotation. Since you have Signature.empty to remove it,
why not set it to Signature.empty when not set? Ditto for other values
that are currently not set when they don't apply (Parameter.empty I
think.)

(2) Could use an example on how to remove and add parameters using replace().

(3) You are using name(arg1, *, arg2) a lot. I think in most cases you
mean for arg2 to be an optional keyword arg, but this notation doesn't
convey that it is optional. Can you clarify?

(4) "If the object is a method" -- shouldn't that be "bound method"?
(Unbound methods are undetectable.) Or is there some wider definition
of method? What does it do for static or class methods?

(5) Too bad there's no proposal for adding signatures to builtin
functions/methods, but understood.

Of these, only (1) is a blocker for PEP acceptance -- I'd either like
to see this defended vigorously (maybe it was discussed? then please
quote, I can't go back and read the threads) or changed.

Otherwise it looks great!

--Guido

On Fri, Jun 22, 2012 at 11:36 AM, Guido van Rossum  wrote:
> I'll review it right now.
>
> On Fri, Jun 22, 2012 at 11:31 AM, Larry Hastings  wrote:
>>
>>
>> Here's PEP 362:
>>
>> http://www.python.org/dev/peps/pep-0362/
>>
>> It adds easy introspection abilities to Python callables.  After a whirlwind
>> of activity over the past several weeks we think it's ready.
>>
>> All it needs now is an official pronouncement from some seasoned veteran of
>> the Python community.  But that's where it's hit an impasse.  Nick Coghlan
>> has recused himself because he was heavily involved in shaping it.  And
>> obviously the authors (including myself, in a small way) are ineligible.
>> Nobody else has stepped forward.  Yet the feature freeze for 3.3 fast
>> approaches--ominous, unstoppable, like a big round boulder headed straight
>> for Indiana Jones.  Time is running out.
>>
>> If you're BDFAP material, why not spend an enjoyable hour today perusing the
>> fine work of these capable folks?  Then naturally all you'd need do is haul
>> out your rubber stamp.  Mere moments later you'd be on your way, whistling a
>> happy tune, a new spring in your step, knowing in your heart you'd made the
>> world a better place.
>>
>> The reference implementation for CPython trunk is here:
>>
>> https://bitbucket.org/1st1/cpython/changesets/tip/branch(%22pep362%22)
>>
>> And here's the bug tracker issue:
>>
>> http://bugs.python.org/issue15008
>>
>>
>> So shines a good deed in a weary world,
>>
>>
>> /arry
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>
>
>
> --
> --Guido van Rossum (python.org/~guido)



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A Desperate Plea For Introspection (aka: BDFAP Needed)

[Guido van Rossum]

I'll review it right now.

On Fri, Jun 22, 2012 at 11:31 AM, Larry Hastings  wrote:
>
>
> Here's PEP 362:
>
> http://www.python.org/dev/peps/pep-0362/
>
> It adds easy introspection abilities to Python callables.  After a whirlwind
> of activity over the past several weeks we think it's ready.
>
> All it needs now is an official pronouncement from some seasoned veteran of
> the Python community.  But that's where it's hit an impasse.  Nick Coghlan
> has recused himself because he was heavily involved in shaping it.  And
> obviously the authors (including myself, in a small way) are ineligible.
> Nobody else has stepped forward.  Yet the feature freeze for 3.3 fast
> approaches--ominous, unstoppable, like a big round boulder headed straight
> for Indiana Jones.  Time is running out.
>
> If you're BDFAP material, why not spend an enjoyable hour today perusing the
> fine work of these capable folks?  Then naturally all you'd need do is haul
> out your rubber stamp.  Mere moments later you'd be on your way, whistling a
> happy tune, a new spring in your step, knowing in your heart you'd made the
> world a better place.
>
> The reference implementation for CPython trunk is here:
>
> https://bitbucket.org/1st1/cpython/changesets/tip/branch(%22pep362%22)
>
> And here's the bug tracker issue:
>
> http://bugs.python.org/issue15008
>
>
> So shines a good deed in a weary world,
>
>
> /arry
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] A Desperate Plea For Introspection (aka: BDFAP Needed)

[Larry Hastings]

Here's PEP 362:

   http://www.python.org/dev/peps/pep-0362/

It adds easy introspection abilities to Python callables.  After a 
whirlwind of activity over the past several weeks we think it's ready.


All it needs now is an official pronouncement from some seasoned veteran 
of the Python community.  But that's where it's hit an impasse.  Nick 
Coghlan has recused himself because he was heavily involved in shaping 
it.  And obviously the authors (including myself, in a small way) are 
ineligible.  Nobody else has stepped forward.  Yet the feature freeze 
for 3.3 fast approaches--ominous, unstoppable, like a big round boulder 
headed straight for Indiana Jones.  Time is running out.


If you're BDFAP material, why not spend an enjoyable hour today perusing 
the fine work of these capable folks?  Then naturally all you'd need do 
is haul out your rubber stamp.  Mere moments later you'd be on your way, 
whistling a happy tune, a new spring in your step, knowing in your heart 
you'd made the world a better place.


The reference implementation for CPython trunk is here:

   https://bitbucket.org/1st1/cpython/changesets/tip/branch(%22pep362%22)
   

And here's the bug tracker issue:

   http://bugs.python.org/issue15008


So shines a good deed in a weary world,


//arry/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython: Issue #14769: test_capi now has SkipitemTest, which cleverly checks

[MRAB]

On 22/06/2012 17:39, Terry Reedy wrote:

On 6/22/2012 6:57 AM, larry.hastings wrote:

http://hg.python.org/cpython/rev/ace45d23628a
changeset:   77567:ace45d23628a
user:Larry Hastings 
date:Fri Jun 22 03:56:29 2012 -0700
summary:
   Issue #14769: test_capi now has SkipitemTest, which cleverly checks
for "parity" between PyArg_ParseTuple() and the Python/getargs.c static
function skipitem() for all possible "format units".


You sensibly only test printable ascii chars, which are in the
contiguous range 32 to 127 inclusive. So it makes no sense to claim
otherwise and then deny the wrong claim, or to enlarge the range and
then shrink it again.


ASCII character 127 is a control character, not a printable character.


+This function brute-force tests all** ASCII characters (1 to 127
+inclusive) as format units, checking to see that



There are 128 ASCII characters (0 to 127 inclusive).


With a few exceptions**, test all printable ASCII characters (32 to 127
inclusive) as...


+
+  ** Okay, it actually skips some ASCII characters.  Some characters


** Some characters ...

+ have special funny semantics, and it would be difficult to
+ accomodate them here.



+for i in range(1, 128):


for i in range(32, 128):



+if (not c.isprintable()) or (c in '()e|$'):


if c in '()e|$':


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython: Issue #14769: test_capi now has SkipitemTest, which cleverly checks

[Terry Reedy]

On 6/22/2012 6:57 AM, larry.hastings wrote:

http://hg.python.org/cpython/rev/ace45d23628a
changeset:   77567:ace45d23628a
user:Larry Hastings 
date:Fri Jun 22 03:56:29 2012 -0700
summary:
   Issue #14769: test_capi now has SkipitemTest, which cleverly checks
for "parity" between PyArg_ParseTuple() and the Python/getargs.c static
function skipitem() for all possible "format units".


You sensibly only test printable ascii chars, which are in the 
contiguous range 32 to 127 inclusive. So it makes no sense to claim 
otherwise and then deny the wrong claim, or to enlarge the range and 
then shrink it again.



+This function brute-force tests all** ASCII characters (1 to 127
+inclusive) as format units, checking to see that


With a few exceptions**, test all printable ASCII characters (32 to 127 
inclusive) as...



+
+  ** Okay, it actually skips some ASCII characters.  Some characters


** Some characters ...

+ have special funny semantics, and it would be difficult to
+ accomodate them here.



+for i in range(1, 128):


for i in range(32, 128):



+if (not c.isprintable()) or (c in '()e|$'):


if c in '()e|$':

tjr

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Signed packages

[Donald Stufft]

Not at the moment, but I could gather them up and make them public later today. 
They 
are very rough draft at the moment.


On Friday, June 22, 2012 at 1:09 PM, Alexandre Zani wrote:

> On Fri, Jun 22, 2012 at 9:56 AM, Donald Stufft  (mailto:donald.stu...@gmail.com)> wrote:
> > On Friday, June 22, 2012 at 12:54 PM, Alexandre Zani wrote:
> > 
> > 
> > Key distribution is the real issue though. If there isn't a key
> > distribution infrastructure in place, we might as well not bother with
> > signatures. PyPI could issue x509 certs to packagers. You wouldn't be
> > able to verify that the name given is accurate, but you would be able
> > to verify that all packages with the same listed author are actually
> > by that author.
> > 
> > I've been sketching out ideas for key distribution, but it's very much
> > a chicken and egg problem, very few people sign their packages (because
> > nothing uses it currently), and nobody is motivated to work on
> > infrastructure
> > or tooling because no one signs their packages.
> > 
> 
> 
> Are those ideas available publicly? I would love to chip in. 

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Signed packages

[Alexandre Zani]

On Fri, Jun 22, 2012 at 9:56 AM, Donald Stufft  wrote:
> On Friday, June 22, 2012 at 12:54 PM, Alexandre Zani wrote:
>
>
> Key distribution is the real issue though. If there isn't a key
> distribution infrastructure in place, we might as well not bother with
> signatures. PyPI could issue x509 certs to packagers. You wouldn't be
> able to verify that the name given is accurate, but you would be able
> to verify that all packages with the same listed author are actually
> by that author.
>
> I've been sketching out ideas for key distribution, but it's very much
> a chicken and egg problem, very few people sign their packages (because
> nothing uses it currently), and nobody is motivated to work on
> infrastructure
> or tooling because no one signs their packages.

Are those ideas available publicly? I would love to chip in.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Signed packages

[Donald Stufft]

On Friday, June 22, 2012 at 12:54 PM, Alexandre Zani wrote:
> 
> Key distribution is the real issue though. If there isn't a key
> distribution infrastructure in place, we might as well not bother with
> signatures. PyPI could issue x509 certs to packagers. You wouldn't be
> able to verify that the name given is accurate, but you would be able
> to verify that all packages with the same listed author are actually
> by that author.
> 
> 

I've been sketching out ideas for key distribution, but it's very much
a chicken and egg problem, very few people sign their packages (because
nothing uses it currently), and nobody is motivated to work on infrastructure
or tooling because no one signs their packages. 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Signed packages

[Alexandre Zani]

On Fri, Jun 22, 2012 at 9:35 AM, Donald Stufft  wrote:
> Ideally authors will be signing their packages (using gpg keys). Of course
> how to distribute keys is an exercise left to the reader.

Key distribution is the real issue though. If there isn't a key
distribution infrastructure in place, we might as well not bother with
signatures. PyPI could issue x509 certs to packagers. You wouldn't be
able to verify that the name given is accurate, but you would be able
to verify that all packages with the same listed author are actually
by that author.

>
> On Friday, June 22, 2012 at 11:48 AM, Vinay Sajip wrote:
>
>  v.loewis.de> writes:
>
>
> See above. Also notice that such signing is already implemented, as part
> of PEP 381.
>
>
> BTW, I notice that the certificate for https://pypi.python.org/ expired a
> week
> ago ...
>
> Regards,
>
> Vinay Sajip
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/donald.stufft%40gmail.com
>
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/alexandre.zani%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Signed packages

[Donald Stufft]

Ideally authors will be signing their packages (using gpg keys). Of course 
how to distribute keys is an exercise left to the reader.


On Friday, June 22, 2012 at 11:48 AM, Vinay Sajip wrote:

>  v.loewis.de (http://v.loewis.de)> writes:
> 
> > 
> > See above. Also notice that such signing is already implemented, as part
> > of PEP 381.
> > 
> 
> 
> BTW, I notice that the certificate for https://pypi.python.org/ expired a week
> ago ...
> 
> Regards,
> 
> Vinay Sajip
> 
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org (mailto:Python-Dev@python.org)
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/donald.stufft%40gmail.com
> 
> 


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[Jesse Noller]

More fuel; fire:

http://lucumr.pocoo.org/2012/6/22/hate-hate-hate-everywhere/



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Summary of Python tracker Issues

[Python tracker]

ACTIVITY SUMMARY (2012-06-15 - 2012-06-22)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open3482 (+10)
  closed 23435 (+51)
  total  26917 (+61)

Open issues with patches: 1460 


Issues opened (46)
==

#15079: pickle: Possibly misplaced test
http://bugs.python.org/issue15079  opened by mstefanro

#15080: Cookie library doesn't parse date properly
http://bugs.python.org/issue15080  opened by jgillick

#15082: [httplib] httplib.BadStatusLine on any HTTPS connection in cer
http://bugs.python.org/issue15082  opened by jtr51

#15083: Rewrite ElementTree tests in a cleaner and safer way
http://bugs.python.org/issue15083  opened by eli.bendersky

#15088: PyGen_NeedsFinalizing is public, but undocumented
http://bugs.python.org/issue15088  opened by ncoghlan

#15090: Add etag support to urllib.request.urlretrieve()
http://bugs.python.org/issue15090  opened by rhettinger

#15091: ImportError when package is symlinked on Unix
http://bugs.python.org/issue15091  opened by jason.coombs

#15092: Using enum PyUnicode_Kind
http://bugs.python.org/issue15092  opened by storchaka

#15093: ntpath.isdir returns False for directory symlinks
http://bugs.python.org/issue15093  opened by jason.coombs

#15094: Incorrectly placed #endif in _tkinter.c.
http://bugs.python.org/issue15094  opened by storchaka

#15095: test_imaplib problem - intermittent skips and LOGINDISABLED no
http://bugs.python.org/issue15095  opened by ncoghlan

#15097: Improving wording on the thread-safeness of import
http://bugs.python.org/issue15097  opened by valhallasw

#15099: exec of function doesn't call __getitem__ or __missing__ on un
http://bugs.python.org/issue15099  opened by johnf

#15100: Race conditions in shutil.copy, shutil.copy2 and shutil.copyfi
http://bugs.python.org/issue15100  opened by radoslaw.zarzynski

#15101: multiprocessing pool finalizer can fail if triggered in backgr
http://bugs.python.org/issue15101  opened by sbt

#15102: Fix 64-bit building for buildbot scripts
http://bugs.python.org/issue15102  opened by jkloth

#15104: Unclear language in __main__ description
http://bugs.python.org/issue15104  opened by techtonik

#15105: curses: wrong indentation
http://bugs.python.org/issue15105  opened by vjp

#15106: Potential Bug in errors.c
http://bugs.python.org/issue15106  opened by Ken.Cheung

#15108: ERROR: SystemError: ./../Objects/tupleobject.c:118: bad argume
http://bugs.python.org/issue15108  opened by pxd

#15109: sqlite3.Connection.iterdump() dies with encoding exception
http://bugs.python.org/issue15109  opened by ekontsevoy

#15110: strange Tracebacks with importlib
http://bugs.python.org/issue15110  opened by amaury.forgeotdarc

#15111: Wrong ImportError message with importlib
http://bugs.python.org/issue15111  opened by amaury.forgeotdarc

#15112: argparse: nargs='*' positional argument doesn't accept any ite
http://bugs.python.org/issue15112  opened by waltermundt

#15114: Deprecate strict mode of HTMLParser
http://bugs.python.org/issue15114  opened by ezio.melotti

#15115: Duplicated Content-Transfer-Encoding header when applying emai
http://bugs.python.org/issue15115  opened by cancel

#15116: remove out-of-date Mac application scripting documentation
http://bugs.python.org/issue15116  opened by hhas

#15117: Please document top-level sqlite3 module variables
http://bugs.python.org/issue15117  opened by wchlm

#15118: uname and other os functions should return a struct sequence i
http://bugs.python.org/issue15118  opened by larry

#15119: ctypes mixed-types bitfield layout nonsensical; doesn't match 
http://bugs.python.org/issue15119  opened by mark.dickinson

#15121: devguide doesn't document all bug tracker components
http://bugs.python.org/issue15121  opened by petri.lehtinen

#15122: Add an option to always rewrite single-file mailboxes in-place
http://bugs.python.org/issue15122  opened by petri.lehtinen

#15124: _thread.LockType: Optimize lock deletion, acquisition of uncon
http://bugs.python.org/issue15124  opened by kristjan.jonsson

#15125: argparse: positional arguments containing - in name not handle
http://bugs.python.org/issue15125  opened by nstiurca

#15127: Supressing warnings with -w  "whether gcc supports ParseTuple"
http://bugs.python.org/issue15127  opened by samueljohn

#15128: inspect raises exception when frames are misleading about sour
http://bugs.python.org/issue15128  opened by acapnotic

#15130: remove redundant paragraph in socket howto
http://bugs.python.org/issue15130  opened by tshepang

#15131: Document py/pyw launchers
http://bugs.python.org/issue15131  opened by brian.curtin

#15132: Let unittest.TestProgram()'s defaultTest argument be a list
http://bugs.python.org/issue15132  opened by cjerdonek

#15133: tkinter.BooleanVar.get() docstring is wrong
http://bugs.python.org/issue15133  opened by mark

#15134: urllib.request.thishost() fails

Re: [Python-Dev] Signed packages

[Vinay Sajip]

  v.loewis.de> writes:

> 
> See above. Also notice that such signing is already implemented, as part
> of PEP 381.
> 

BTW, I notice that the certificate for https://pypi.python.org/ expired a week
ago ...

Regards,

Vinay Sajip


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[Vinay Sajip]

Paul Moore  gmail.com> writes:

> As a user, I guess not that much. I may be misremembering bad
> experiences with different things. We've had annoyances with
> self-signed jars, and websites. It's generally more about annoying
> "can't confirm this should be trusted, please verify" messages which
> people end up just saying "yes" to (and so ruining any value from the
> check).

Like those pesky EULAs ;-)

> But you say "I got a code signing certificate". How? When I dabbled
> with signing, the only option I could find that didn't involve paying
> and/or having a registered domain of my own was a self-signed
> certificate, which from a UI point of view seems of little use "Paul
> Moore says you should trust him. Do you? Yes/No"...

I got mine from Certum (certum.pl) - they offer (or at least did offer, last
year) free code signing certificates for Open Source developers (you have to
have "Open Source Developer" in what's being certified). See:

http://www.certum.eu/certum/cert,offer_en_open_source_cs.xml

> If signed binaries is the way we go, then we should be aware that we
> exclude people who don't have certificates from uploading to PyPI.

I don't think that any exclusion would occur. It just means that there's a
mechanism for people who are picky about such things to have a slightly larger
comfort zone.

> Maybe that's OK, but without some sort of check I don't know how many
> current developers that would exclude, let alone how many potential
> developers would be put off.

I don't think any packager need be excluded. It would be up to individual
packagers and package consumers as to whether they sign packages / stick to only
using signed packages. For almost everyone, life should go on as before.

> A Python-supported build farm, which signed code on behalf of
> developers, might alleviate this. But then we need to protect against
> malicious code being submitted to the build farm, etc.

There is IMO neither the will nor the resource to do any sort of policing.
Caveat emptor (or caveat user, rather). Let's not forget, all of this software
is without warranty of any kind.
 
> Fair enough. I don't object to offering the option to verify
> signatures (I think I said something like that in an earlier message).
> I do have concerns about making signed code mandatory. (Not least over
> whether it'd let me install my own unsigned code!)

Any workable mechanism would need to be optional (the user doing the installing
would be the decider as to whether to go ahead and install, with signature, or
lack thereof, in mind).

Regards,

Vinay Sajip

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Signed packages

[martin]

Zitat von Antoine Pitrou :


On Fri, 22 Jun 2012 12:27:19 +0100
Paul Moore  wrote:


Signed binaries may be a solution. My experience with signed binaries
has not been exactly positive, but it's an option. Presumably PyPI
would be the trusted authority? Would PyPI and the downloaders need to
use SSL? Would developers need to have signing keys to use PyPI? And
more to the point, do the people designing the packaging solutions
have experience with this sort of stuff (I sure don't :-))?


The ones signing the binaries would have to be the packagers, not PyPI.


It depends. PyPI already signs all binaries (essentially) as part of the
mirror protocol. What this proves is that the mirror has not modified
the data compared to the copy of PyPI. If PyPI can be trusted not to modify
the binaries, then this also proves that the binaries are the same as
originally uploaded.

What this doesn't prove is that the upload was really made by the declared
author of the package (which could be prevented by signing the packages
by the original author); it also doesn't prove that the binaries are free
of malicous code (which no amount of signing can prove).


PyPI-signing of packages would not achieve anything, since PyPI cannot
vouch for the quality and non-maliciousness of uploaded files.


That's just not true. It can prove that the files have not been modified
by mirrors, caches, and the like, of which there are plenty in practice.


It would only serve as a replacement for SSL downloads.


See above. Also notice that such signing is already implemented, as part
of PEP 381.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Signed packages

[Antoine Pitrou]

On Fri, 22 Jun 2012 12:27:19 +0100
Paul Moore  wrote:
> 
> Signed binaries may be a solution. My experience with signed binaries
> has not been exactly positive, but it's an option. Presumably PyPI
> would be the trusted authority? Would PyPI and the downloaders need to
> use SSL? Would developers need to have signing keys to use PyPI? And
> more to the point, do the people designing the packaging solutions
> have experience with this sort of stuff (I sure don't :-))?

The ones signing the binaries would have to be the packagers, not PyPI.

Also, if packages are signed, you arguably don't need to use SSL when
downloading them (but SSL can still be useful for other purposes e.g.
navigating in the catalog).

PyPI-signing of packages would not achieve anything, since PyPI cannot
vouch for the quality and non-maliciousness of uploaded files. It would
only serve as a replacement for SSL downloads.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[Paul Moore]

On 22 June 2012 13:09, Vinay Sajip  wrote:
> Paul Moore  gmail.com> writes:
>
>> Signed binaries may be a solution. My experience with signed binaries
>> has not been exactly positive, but it's an option. Presumably PyPI
>> would be the trusted authority? Would PyPI and the downloaders need to
>> use SSL? Would developers need to have signing keys to use PyPI? And
>> more to the point, do the people designing the packaging solutions
>> have experience with this sort of stuff (I sure don't )?
>
> I'm curious - what problems have you had with signed binaries?

As a user, I guess not that much. I may be misremembering bad
experiences with different things. We've had annoyances with
self-signed jars, and websites. It's generally more about annoying
"can't confirm this should be trusted, please verify" messages which
people end up just saying "yes" to (and so ruining any value from the
check).

But I don't know how often I have used them, to the extent that the
only time I'm aware of them is when they don't work silently (e.g., I
get a prompt asking if I want to trust this publisher - this is
essentially a failure, as I always say "yes" simply because I have no
idea how I would go about deciding that I do trust them, beyond what
I've already done in locating and downloading the software from them!)

> I dipped my toes
> in this particular pool with the Python launcher installers - I got a code
> signing certificate and signed my MSIs with it. The process was fairly 
> painless.

OK, that's a good example, I didn't even realise those installers were
signed, making it an excellent example of how easy it can be when it
works.

But you say "I got a code signing certificate". How? When I dabbled
with signing, the only option I could find that didn't involve paying
and/or having a registered domain of my own was a self-signed
certificate, which from a UI point of view seems of little use "Paul
Moore says you should trust him. Do you? Yes/No"...

If signed binaries is the way we go, then we should be aware that we
exclude people who don't have certificates from uploading to PyPI.
Maybe that's OK, but without some sort of check I don't know how many
current developers that would exclude, let alone how many potential
developers would be put off.

A Python-supported build farm, which signed code on behalf of
developers, might alleviate this. But then we need to protect against
malicious code being submitted to the build farm, etc.

> As far as I know, all signing does is to indicate that the binary package 
> hasn't
> been tampered with and allows the downloader to decide whether they trust the
> signer not to have allowed backdoors, etc. I don't see that it mandates use of
> SSL, or even signing, by anyone. At least some people will require that an
> installer be invokable with an option that causes it to bail if any part of
> what's being installed can't be verified (for some value of "verified").

Fair enough. I don't object to offering the option to verify
signatures (I think I said something like that in an earlier message).
I do have concerns about making signed code mandatory. (Not least over
whether it'd let me install my own unsigned code!)

Paul
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[David Cournapeau]

On Fri, Jun 22, 2012 at 2:24 PM, Paul Moore  wrote:

>
> I suppose if you're saying that "pip install lxml" should download and
> install for me Visual Studio, libxml2 sources and any dependencies,
> and run all the builds, then you're right. But I assume you're not. So
> why should I need to install Visual Studio just to *use* lxml?
>
> On the other hand, I concede that there are some grey areas between
> the 2 extremes. I don't know enough to do a proper review of the
> various cases. But I do think that there's a risk that the discussion,
> because it is necessarily driven by developers, forgets that "end
> users" really don't have some tools that a developer would consider
> "trivial" to have.

Binary installers are important: if you think lxml is hard on windows,
think about what it means to build fortran libraries and link them
with visual studio for scipy :) That's one of the reason virtualenv +
pip is not that useful for numpy/scipy end users. Bento has code to
build basic binary installers in all the formats supported by
distutils except for RPM, and the code is by design mostly independ of
the rest. I would be happy to clean up that code to make it more
reusable (most of it is extracted from distutils/setuptools anyway).

But it should be completely orthogonal to the issue of package
description: if there is one thing that distutils got horribly wrong,
that's tying everything altogether. The uncoupling is the key, because
otherwise, one keep discussing all the issues together, which is part
of what makes the discussion so hard. Different people have different
needs.

David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[Paul Moore]

On 22 June 2012 13:39, Stephen J. Turnbull  wrote:
> Nick Coghlan writes:
>  > On Fri, Jun 22, 2012 at 4:25 PM, Stephen J. Turnbull  
> wrote:
>  > > Paul Moore writes:
>  > >
>  > >  > End users should not need packaging tools on their machines.
>  > >
>  > > I think this desideratum is close to obsolete these days, with webapps
>  > > in "the cloud" downloading resources (including, but not limited to,
>  > > code) on an as-needed basis.
>  >
>  > There's still a lot more to the software world than what happens on
>  > the public internet.
>
> That's taking just one extreme out of context.  The other extreme I
> mentioned is a whole (virtual) Python environment to go with your app.
>
> And I don't really see a middle ground, unless you're delivering a
> non-standard stdlib anyway, with all the stuff that end users don't
> need stripped out of it.  They'll get the debugger and the profiler
> with Python; should we excise them from the stdlib just because end
> users don't need them?  How about packaging diagnostic tools,
> especially in the early days of the new module?
>
> I agreed that end users should not need to download the packaging
> tools separately or in advance.  But that's rather different from
> having a *requirement* that the tools not be included, or that
> installers should have no dependencies on the toolset outside of a
> minimal and opaque runtime module.

I suppose if you're saying that "pip install lxml" should download and
install for me Visual Studio, libxml2 sources and any dependencies,
and run all the builds, then you're right. But I assume you're not. So
why should I need to install Visual Studio just to *use* lxml?

On the other hand, I concede that there are some grey areas between
the 2 extremes. I don't know enough to do a proper review of the
various cases. But I do think that there's a risk that the discussion,
because it is necessarily driven by developers, forgets that "end
users" really don't have some tools that a developer would consider
"trivial" to have.

Paul.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[Alex Clark]

Hi,

On 6/22/12 1:05 AM, Nick Coghlan wrote:

On Fri, Jun 22, 2012 at 10:01 AM, Donald Stufft  wrote:

The idea i'm hoping for is to stop worrying about one implementation over
another and
hoping to create a common format that all the tools can agree upon and
create/install.


Right, and this is where it encouraged me to see in the Bento docs
that David had cribbed from RPM in this regard (although I don't
believe he has cribbed *enough*).

A packaging system really needs to cope with two very different levels
of packaging:
1. Source distributions (e.g. SRPMs). To get from this to useful
software requires developer tools.
2. "Binary" distributions (e.g. RPMs). To get from this to useful
software mainly requires a "file copy" utility (well, that and an
archive decompressor).

An SRPM is *just* a SPEC file and source tarball. That's it. To get
from that to an installed product, you have a bunch of additional
"BuildRequires" dependencies, along with %build and %install scripts
and a %files definition that define what will be packaged up and
included in the binary RPM. The exact nature of the metadata format
doesn't really matter, what matters is that it's a documented standard
that multiple tools can read.

An RPM includes files that actually get installed on the target
system. An RPM can be arch specific (if they include built binary
bits) or "noarch" if they're platform neutral.

distutils really only plays at the SRPM level - there is no defined OS
neutral RPM equivalent. That's why I brought up the bdist_simple
discussion earlier in the thread - if we can agree on a standard
bdist_simple format, then we can more cleanly decouple the "build"
step from the "install" step.

I think one of the key things to learn from the SPEC file format is
the configuration language it used for the various build phases: sh
(technically, any shell on the system, but almost everyone just uses
the default system shell)

This is why you can integrate whatever build system you like with it:
so long as you can invoke the build from the shell, then you can use
it to make your RPM.

Now, there's an obvious problem with this: it's completely useless
from a *cross-platform* building point of view. Isn't it a shame
there's no language we could use that would let us invoke build
systems in a cross platform way? Oh, wait...

So here's some sheer pie-in-the-sky speculation. If people like
elements of this idea enough to run with it, great. If not... oh well:

- I believe the "egg" term has way too much negative baggage (courtesy
of easy_install), and find the full term Distribution to be too easily
confused with "Linux distribution". However, "Python dist" is
unambiguous (since the more typical abbreviation for an aggregate
distribution is "distro"). Thus, I attempt to systematically refer to
the objects used to distribute Python software from developers to
users as "dists". In practice, this terminology is already used in
many places (distutils, sdist, bdist_msi, bdist_rpm, the .dist-info
format in PEP 376 etc). Thus, Python software is distributed as dists
(either sdists or bdists), which may in turn be converted to distro
packages (e.g. SRPMs and RPMs) for deployment to particular
environments.



+0.5. There is definitely a problem with the term "egg", but I don't 
think negative baggage is it.


Rather, I think "egg" is just plain too confusing, and perhaps too 
"cutsie", too. A blurb from the internet[1]:



"An egg is a bundle that contains all the package data. In the ideal 
case, an egg is a zip-compressed file with all the necessary package 
files. But in some cases, setuptools decides (or is told by switches) 
that a package should not be zip-compressed. In those cases, an egg is 
simply an uncompressed subdirectory, but with the same contents. The 
single file version is handy for transporting, and saves a little bit of 
disk space, but an egg directory is functionally and organizationally 
identical."



Compared to the definitions of package and distribution I posted earlier 
in this thread, the confusion is:


- A package is one or more modules inside another module, a distribution 
is a compressed archive of those modules, but an egg is either or both.


- The blurb author uses the term "package data" presumably to refer to 
package modules, package data (i.e. resources like templates, etc), and 
package metadata.


So to avoid this confusion I've personally stopped using the term "egg" 
in favor of "package". (Outside a computer context, everyone knows a 
package is something "with stuff in it") But as Donald said, what we are 
all talking about is technically called a "distribution". ("Honey, a 
distribution arrived for you in the mail today!" :-))


I love that Nick is thinking "outside the box" re: terminology, but I'm 
not 100% convinced the new term should be "dist". Rather I propose:


- Change the definition of package to: a module (or modules) plus 
package data and package metadata inside another module.


- Re

Re: [Python-Dev] Status of packaging in 3.3

[Stephen J. Turnbull]

Nick Coghlan writes:
 > On Fri, Jun 22, 2012 at 4:25 PM, Stephen J. Turnbull  
 > wrote:
 > > Paul Moore writes:
 > >
 > >  > End users should not need packaging tools on their machines.
 > >
 > > I think this desideratum is close to obsolete these days, with webapps
 > > in "the cloud" downloading resources (including, but not limited to,
 > > code) on an as-needed basis.
 > 
 > There's still a lot more to the software world than what happens on
 > the public internet.

That's taking just one extreme out of context.  The other extreme I
mentioned is a whole (virtual) Python environment to go with your app.

And I don't really see a middle ground, unless you're delivering a
non-standard stdlib anyway, with all the stuff that end users don't
need stripped out of it.  They'll get the debugger and the profiler
with Python; should we excise them from the stdlib just because end
users don't need them?  How about packaging diagnostic tools,
especially in the early days of the new module?

I agreed that end users should not need to download the packaging
tools separately or in advance.  But that's rather different from
having a *requirement* that the tools not be included, or that
installers should have no dependencies on the toolset outside of a
minimal and opaque runtime module.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[Tim Golden]

On 22/06/2012 13:14, Barry Warsaw wrote:
> On Jun 22, 2012, at 12:27 PM, Paul Moore wrote:
> 
>> And what I am trying to say is that no matter how much effort gets put
>> into trying to make build from source easy, it'll pretty much always
>> not be even remotely trivial on Windows.
> 
> It seems to me that a "Windows build service" is something the Python
> infrastructure could support.  This would be analogous to the types of binary
> build services Linux distros provide, e.g. the normal Ubuntu workflow of
> uploading a source package to a build daemon, which churns away for a while,
> and results in platform-specific binary packages which can be directly
> installed on an end-user system.

The devil would be in the details. As Paul Moore pointed out earlier,
building *any* extension which relies on some 3rd-party library on
Windows (mysql, libxml, sdl, whatever) can be an exercise in iterative
frustration as you discover build requirements on build requirements.
This isn't just down to Python: try building TortoiseSvn by yourself,
for example.

That's not say that this is insurmountable. Christopher Gohlke has
for a long while maintained an unofficial binary store at his site:

  http://www.lfd.uci.edu/~gohlke/pythonlibs/

but I've no idea how much work he's had to put in to get all the
dependencies built.

Someone who just turned up with a new build:
"Here's a Python interface for ToastRack -- the new card-printing
service" would need a way to provide the proposed build infrastructure
with what was needed to build the library behind the Python extension.

Little fleas have smaller fleas... and so on.

TJG
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[Barry Warsaw]

On Jun 22, 2012, at 07:49 AM, Vinay Sajip wrote:

>The format-neutral alternative I used for logging configuration was a
>dictionary schema - JSON, YAML and Python code can all be mapped to
>that. Perhaps the relevant APIs can work at the dict layer.

I don't much care whether it's ini, json, or yaml, but I do think it needs to
be declarative and language neutral.  I don't want to lock up all that
metadata into Python data structures.  There are valid use cases for being
able to access the data from outside of Python.

And please give some thought to test declarations.  We need a standard way to
declare how a package's tests should be run, and what the test dependencies
are, so that we can improve the quality of all distro/binary packages by
running the test suite at build time.  Having to guess whether it's `python
setup.py test` or `python -m unittest discover` or whether nose or py.test is
required, etc. etc. is no good.

-Barry
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[Barry Warsaw]

On Jun 22, 2012, at 12:27 PM, Paul Moore wrote:

>And what I am trying to say is that no matter how much effort gets put
>into trying to make build from source easy, it'll pretty much always
>not be even remotely trivial on Windows.

It seems to me that a "Windows build service" is something the Python
infrastructure could support.  This would be analogous to the types of binary
build services Linux distros provide, e.g. the normal Ubuntu workflow of
uploading a source package to a build daemon, which churns away for a while,
and results in platform-specific binary packages which can be directly
installed on an end-user system.

-Barry
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[Vinay Sajip]

Paul Moore  gmail.com> writes:

> Signed binaries may be a solution. My experience with signed binaries
> has not been exactly positive, but it's an option. Presumably PyPI
> would be the trusted authority? Would PyPI and the downloaders need to
> use SSL? Would developers need to have signing keys to use PyPI? And
> more to the point, do the people designing the packaging solutions
> have experience with this sort of stuff (I sure don't )?

I'm curious - what problems have you had with signed binaries? I dipped my toes
in this particular pool with the Python launcher installers - I got a code
signing certificate and signed my MSIs with it. The process was fairly painless.
As far as I know, all signing does is to indicate that the binary package hasn't
been tampered with and allows the downloader to decide whether they trust the
signer not to have allowed backdoors, etc. I don't see that it mandates use of
SSL, or even signing, by anyone. At least some people will require that an
installer be invokable with an option that causes it to bail if any part of
what's being installed can't be verified (for some value of "verified").

Regards,

Vinay Sajip



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[Paul Moore]

On 22 June 2012 11:28, Dag Sverre Seljebotn  wrote:
> And I'm saying that would encourage a culture that's very dangerous from a
> security perspective. Even if many uses binaries, it is important to
> encourage a culture where it is always trivial (well, as trivial as we can
> possibly make it, in the case of Windows) to build from source for those who
> wish to.

And what I am trying to say is that no matter how much effort gets put
into trying to make build from source easy, it'll pretty much always
not be even remotely trivial on Windows. There has been a lot of work
done to try to achieve this, but as far as I've seen, it's always
failed. One external dependency, and you're in a mess. Unless you're
proposing some means of Python's packaging solution encapsulating URLs
for binary libraries of external packages which will be automatically
downloaded - and then all the security holes open again.

You have to remember that not only do many Windows users not have a
compiler, but also getting a compiler is non-trivial (not hard, just
download and install VS Express, but still a pain to do just to get
(say) lxml installed). And there is no standard location for external
libraries in Windows, so you also need the end user to specify where
everything is (or guess, or mandate a directory structure).

The only easy-to-use solution that has ever really worked on Windows
in my experience is downloadable binaries. Blame whoever you like,
point out that it's not good practice if you must, but don't provide
binaries and you lose a major part of your user base. (You may choose
not to care about losing that group, that's a different question).

Signed binaries may be a solution. My experience with signed binaries
has not been exactly positive, but it's an option. Presumably PyPI
would be the trusted authority? Would PyPI and the downloaders need to
use SSL? Would developers need to have signing keys to use PyPI? And
more to the point, do the people designing the packaging solutions
have experience with this sort of stuff (I sure don't :-))?

Paul.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[Donald Stufft]

On Friday, June 22, 2012 at 6:20 AM, David Cournapeau wrote:
> If by manifest you mean the build manifest, then that's not desirable:
> the manifest contains the explicit filenames, and those are
> platform/environment specific. You don't want this to be user-facing.
> 
It appears I misunderstood the files that bento uses then ;) It is late (well 
early now) 
and I have not used bento extensively.

What I suggest mirrors RPM's similarly except the build step (when there is 
indeed a
build step) is handled by a python script included in the package by the author 
of said
package.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[Donald Stufft]

On Friday, June 22, 2012 at 5:52 AM, Dag Sverre Seljebotn wrote:

> 
> The reason PyPI isn't one big security risk is that packages are built 
> from source, and so you can have some confidence that backdoors would be 
> noticed and highlighted by somebody.
> 
> Having a common standards for binary installation phase would be great 
> sure, but security-minded users would still need to build from source in 
> every case (or trust a 3rt party build farm that builds from source). 
> The reason you can trust RPMs at all is because they're built from SRPMs.
> 
> Dag 
The reason you trust RPM's is not because they are built from SRPM's,
but because you trust the people running the repositories. In the case of PyPI
you can't make a global call to implicitly trust all packages because there is
no gatekeeper as in an RPM system, so it falls to the individual to decide
for him or herself which authors they trust and which authors they do not trust.

But this proposal alludes to both source dists and built dists, either which may
be published and installed from.

In the case of a source dist the package format would include all the metadata
of the package. Included in that is a python script that knows how to build this
particular package (if special steps are required). This script could simply 
call
out to an already existing build system, or if simple enough work on it's own.
Source dists would also obviously contain the source.

In the case of a binary dist the package format would include all the metadata
of the package, plus the binary files.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[Dag Sverre Seljebotn]

On 06/22/2012 12:20 PM, David Cournapeau wrote:

On Fri, Jun 22, 2012 at 10:38 AM, Donald Stufft  wrote:

On Friday, June 22, 2012 at 5:22 AM, Dag Sverre Seljebotn wrote:


What Bento does is have one metadata file for the source-package, and
another metadata file (manifest) for the built-package. The latter is
normally generated by the build process (but follows a standard
nevertheless). Then that manifest is used for installation (through
several available methods).

 From what I understand, this dist.(yml|json|ini) would be replacing the
mainfest not the bento.info then. When bento builds a package compatible
with the proposed format it would instead of generating it's own manifest
it would generate the dist.(yml|json|ini).


If by manifest you mean the build manifest, then that's not desirable:
the manifest contains the explicit filenames, and those are
platform/environment specific. You don't want this to be user-facing.

The way it should work is:
- package description (dist.yaml, setup.cfg, bento.info, whatever)
- use this as input to the build process
- build process produces a build manifest that is platform
specific. It should be extremely simple, no conditional or anything,
and should ideally be fed to both python and non-python programs.
- build manifest is then the sole input to the process building
installers (besides the actual build tree, of course).

Conceptually, after the build, you can do :

manifest = BuildManifest.from_file("build_manifest.json")
manifest.update_path(path_configuration) # This is needed so as to
allow path scheme to be changed depending on installer format
for category, source, target on manifest.iter_files():
  # simple case is copying source to target,  potentially using the
category label for category specific stuff.

This was enough for me to do straight install, eggs, .exe and .msi
windows installers and .mpkg from that with a relatively simple API.
Bonus point, if you include this file inside the installers, you can
actually losslessly convert from one to the other.


I think Donald's suggestion can be phrased as this: During build, copy 
the dist metadata (name, version, dependencies...) to the build manifest 
as well. Then allow to upload only the built versions for different 
platforms to PyPI etc. and allow relative anarchy to reign in how you 
create the built dists.


And I'm saying that would encourage a culture that's very dangerous from 
a security perspective. Even if many uses binaries, it is important to 
encourage a culture where it is always trivial (well, as trivial as we 
can possibly make it, in the case of Windows) to build from source for 
those who wish to.


Making the user-facing entry point of the dist metadata be in the source 
package rather than the binary package seems like a necessary (but not 
sufficient) condition for such a culture.


Dag
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[David Cournapeau]

On Fri, Jun 22, 2012 at 10:38 AM, Donald Stufft  wrote:
> On Friday, June 22, 2012 at 5:22 AM, Dag Sverre Seljebotn wrote:
>
>
> What Bento does is have one metadata file for the source-package, and
> another metadata file (manifest) for the built-package. The latter is
> normally generated by the build process (but follows a standard
> nevertheless). Then that manifest is used for installation (through
> several available methods).
>
> From what I understand, this dist.(yml|json|ini) would be replacing the
> mainfest not the bento.info then. When bento builds a package compatible
> with the proposed format it would instead of generating it's own manifest
> it would generate the dist.(yml|json|ini).

If by manifest you mean the build manifest, then that's not desirable:
the manifest contains the explicit filenames, and those are
platform/environment specific. You don't want this to be user-facing.

The way it should work is:
   - package description (dist.yaml, setup.cfg, bento.info, whatever)
   - use this as input to the build process
   - build process produces a build manifest that is platform
specific. It should be extremely simple, no conditional or anything,
and should ideally be fed to both python and non-python programs.
   - build manifest is then the sole input to the process building
installers (besides the actual build tree, of course).

Conceptually, after the build, you can do :

manifest = BuildManifest.from_file("build_manifest.json")
manifest.update_path(path_configuration) # This is needed so as to
allow path scheme to be changed depending on installer format
for category, source, target on manifest.iter_files():
 # simple case is copying source to target,  potentially using the
category label for category specific stuff.

This was enough for me to do straight install, eggs, .exe and .msi
windows installers and .mpkg from that with a relatively simple API.
Bonus point, if you include this file inside the installers, you can
actually losslessly convert from one to the other.

David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[Vinay Sajip]

Dag Sverre Seljebotn  astro.uio.no> writes:

> Well, but I think you need to care about the whole process here.
> 
> Focusing only on the "end-user case" and binary installers has the flip 
> side that smuggling in a back door is incredibly easy in compiled 
> binaries. You simply upload a binary that doesn't match the source.
> 
> The reason PyPI isn't one big security risk is that packages are built 
> from source, and so you can have some confidence that backdoors would be 
> noticed and highlighted by somebody.
> 
> Having a common standards for binary installation phase would be great 
> sure, but security-minded users would still need to build from source in 
> every case (or trust a 3rt party build farm that builds from source). 
> The reason you can trust RPMs at all is because they're built from SRPMs.

Easy enough on Posix platforms, perhaps, but what about Windows? One can't
expect a C compiler to be installed everywhere. Perhaps security against
backdoors could also be provided through other mechanisms, such as signing of
binary installers.

Regards,

Vinay Sajip





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[Antoine Pitrou]

On Fri, 22 Jun 2012 15:05:08 +1000
Nick Coghlan  wrote:
> 
> So here's some sheer pie-in-the-sky speculation. If people like
> elements of this idea enough to run with it, great. If not... oh well:

Could this kind of discussion perhaps go on python-ideas?

Thanks

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Checking if unsigned int less then zero.

[Peter Otten]

Dmitriy Tochansky wrote:

> Playing with cpython source, I found some strange strings in
> socketmodule.c:
> 
> ---
>  if (flowinfo < 0 || flowinfo > 0xf) {
>  PyErr_SetString(
>  PyExc_OverflowError,
>  "getsockaddrarg: flowinfo must be 0-1048575.");
>  return 0;
>  }
> ---
> 
> ---
>  if (flowinfo < 0 || flowinfo > 0xf) {
>  PyErr_SetString(PyExc_OverflowError,
>  "getsockaddrarg: flowinfo must be 0-1048575.");
>  return NULL;
>  }
> ---
> 
> The flowinfo variable declared few strings above as unsgined int. Is
> there any practical sense in this check? Seems like gcc just removes
> this check. I think any compiler will generate code that checks as
> unsigned, for example in x86 its JAE/JGE. May be this code is for "bad"
> compilers or exotic arch?

I think you are right, the < 0 check is redundant. The developers probably 
forgot to remove it when

http://bugs.python.org/issue9975

was fixed.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[Dag Sverre Seljebotn]

On 06/22/2012 11:38 AM, Donald Stufft wrote:

On Friday, June 22, 2012 at 5:22 AM, Dag Sverre Seljebotn wrote:


What Bento does is have one metadata file for the source-package, and
another metadata file (manifest) for the built-package. The latter is
normally generated by the build process (but follows a standard
nevertheless). Then that manifest is used for installation (through
several available methods).

 From what I understand, this dist.(yml|json|ini) would be replacing the
mainfest not the bento.info then. When bento builds a package compatible
with the proposed format it would instead of generating it's own manifest
it would generate the dist.(yml|json|ini).


Well, but I think you need to care about the whole process here.

Focusing only on the "end-user case" and binary installers has the flip 
side that smuggling in a back door is incredibly easy in compiled 
binaries. You simply upload a binary that doesn't match the source.


The reason PyPI isn't one big security risk is that packages are built 
from source, and so you can have some confidence that backdoors would be 
noticed and highlighted by somebody.


Having a common standards for binary installation phase would be great 
sure, but security-minded users would still need to build from source in 
every case (or trust a 3rt party build farm that builds from source). 
The reason you can trust RPMs at all is because they're built from SRPMs.


Dag
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[Donald Stufft]

On Friday, June 22, 2012 at 5:22 AM, Dag Sverre Seljebotn wrote:
> 
> What Bento does is have one metadata file for the source-package, and
> another metadata file (manifest) for the built-package. The latter is
> normally generated by the build process (but follows a standard
> nevertheless). Then that manifest is used for installation (through
> several available methods).
> 
> 

>From what I understand, this dist.(yml|json|ini) would be replacing the
mainfest not the bento.info then. When bento builds a package compatible
with the proposed format it would instead of generating it's own manifest
it would generate the dist.(yml|json|ini). 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[Dag Sverre Seljebotn]

On 06/22/2012 10:40 AM, Paul Moore wrote:

On 22 June 2012 06:05, Nick Coghlan  wrote:

distutils really only plays at the SRPM level - there is no defined OS
neutral RPM equivalent. That's why I brought up the bdist_simple
discussion earlier in the thread - if we can agree on a standard
bdist_simple format, then we can more cleanly decouple the "build"
step from the "install" step.


That was essentially the key insight I was trying to communicate in my
"think about the end users" comment. Thanks, Nick!


The subtlety here is that there's no way to know before building the 
package what files should be installed. (For simple extensions, and 
perhaps documentation, you could get away with ad-hoc rules or special 
support for Sphinx and what-not, but there's no general solution that 
works in all cases.)


What Bento does is have one metadata file for the source-package, and 
another metadata file (manifest) for the built-package. The latter is 
normally generated by the build process (but follows a standard 
nevertheless). Then that manifest is used for installation (through 
several available methods).


Dag
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[Vinay Sajip]

David Cournapeau  gmail.com> writes:

> I agree having yet another format is a bit crazy, and am actually considering
changing bento.info to be a yaml. I initially did got toward a cabal-like syntax
instead for the following reasons:
>   - lack of conditional (a must IMO, it is even more useful for cross
-platform stuff than it is for RPM only)

Conditionals could perhaps be handled in different ways, e.g.

1. Markers as used in distutils2/packaging (where the condition is platform or
version related)
2. A scheme to resolve variables, such as is used in PEP 391 (dictionary-based
configuration for logging).

If conditionals are much more involved than this, there's a possibility of
introducing too much program logic - the setup.py situation

>   - regarding hooks: I think it is simpler to have a single file which
contains all the hooks, if only to allow for easy communication between hooks
and code reuse between hooks. I don't see any drawback to using only one file ?

I was assuming that the dist.yaml file would just have callable references here;
I suppose having (sizable) Python fragments in dist.yaml might become unwieldy.

Regards,

Vinay Sajip


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Checking if unsigned int less then zero.

[Dmitriy Tochansky]

Hello!
Playing with cpython source, I found some strange strings in 
socketmodule.c:


---
if (flowinfo < 0 || flowinfo > 0xf) {
PyErr_SetString(
PyExc_OverflowError,
"getsockaddrarg: flowinfo must be 0-1048575.");
return 0;
}
---

---
if (flowinfo < 0 || flowinfo > 0xf) {
PyErr_SetString(PyExc_OverflowError,
"getsockaddrarg: flowinfo must be 0-1048575.");
return NULL;
}
---

The flowinfo variable declared few strings above as unsgined int. Is 
there any practical sense in this check? Seems like gcc just removes 
this check. I think any compiler will generate code that checks as 
unsigned, for example in x86 its JAE/JGE. May be this code is for "bad" 
compilers or exotic arch?


--
Dmitriy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[Paul Moore]

On 22 June 2012 06:05, Nick Coghlan  wrote:
> distutils really only plays at the SRPM level - there is no defined OS
> neutral RPM equivalent. That's why I brought up the bdist_simple
> discussion earlier in the thread - if we can agree on a standard
> bdist_simple format, then we can more cleanly decouple the "build"
> step from the "install" step.

That was essentially the key insight I was trying to communicate in my
"think about the end users" comment. Thanks, Nick!

Comments on the rest of your email to follow (if needed) when I've
digested it...
Paul
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[David Cournapeau]

On Fri, Jun 22, 2012 at 6:05 AM, Nick Coghlan  wrote:

> On Fri, Jun 22, 2012 at 10:01 AM, Donald Stufft 
> wrote:
> > The idea i'm hoping for is to stop worrying about one implementation over
> > another and
> > hoping to create a common format that all the tools can agree upon and
> > create/install.
>
> Right, and this is where it encouraged me to see in the Bento docs
> that David had cribbed from RPM in this regard (although I don't
> believe he has cribbed *enough*).
>
> A packaging system really needs to cope with two very different levels
> of packaging:
> 1. Source distributions (e.g. SRPMs). To get from this to useful
> software requires developer tools.
> 2. "Binary" distributions (e.g. RPMs). To get from this to useful
> software mainly requires a "file copy" utility (well, that and an
> archive decompressor).
>
> An SRPM is *just* a SPEC file and source tarball. That's it. To get
> from that to an installed product, you have a bunch of additional
> "BuildRequires" dependencies, along with %build and %install scripts
> and a %files definition that define what will be packaged up and
> included in the binary RPM. The exact nature of the metadata format
> doesn't really matter, what matters is that it's a documented standard
> that multiple tools can read.
>
> An RPM includes files that actually get installed on the target
> system. An RPM can be arch specific (if they include built binary
> bits) or "noarch" if they're platform neutral.
>
> distutils really only plays at the SRPM level - there is no defined OS
> neutral RPM equivalent. That's why I brought up the bdist_simple
> discussion earlier in the thread - if we can agree on a standard
> bdist_simple format, then we can more cleanly decouple the "build"
> step from the "install" step.
>
> I think one of the key things to learn from the SPEC file format is
> the configuration language it used for the various build phases: sh
> (technically, any shell on the system, but almost everyone just uses
> the default system shell)
>
> This is why you can integrate whatever build system you like with it:
> so long as you can invoke the build from the shell, then you can use
> it to make your RPM.
>
> Now, there's an obvious problem with this: it's completely useless
> from a *cross-platform* building point of view. Isn't it a shame
> there's no language we could use that would let us invoke build
> systems in a cross platform way? Oh, wait...
>
> So here's some sheer pie-in-the-sky speculation. If people like
> elements of this idea enough to run with it, great. If not... oh well:
>
> - I believe the "egg" term has way too much negative baggage (courtesy
> of easy_install), and find the full term Distribution to be too easily
> confused with "Linux distribution". However, "Python dist" is
> unambiguous (since the more typical abbreviation for an aggregate
> distribution is "distro"). Thus, I attempt to systematically refer to
> the objects used to distribute Python software from developers to
> users as "dists". In practice, this terminology is already used in
> many places (distutils, sdist, bdist_msi, bdist_rpm, the .dist-info
> format in PEP 376 etc). Thus, Python software is distributed as dists
> (either sdists or bdists), which may in turn be converted to distro
> packages (e.g. SRPMs and RPMs) for deployment to particular
> environments.
>
> - I reject setup.cfg, as I believe ini-style configuration files are
> not appropriate for a metadata format that needs to include file
> listings and code fragments
>
> - I reject bento.info, as I think if we accept
> yet-another-custom-configuration-file-format into the standard library
> instead of just using YAML, we're even crazier than is already
> apparent
>

I agree having yet another format is a bit crazy, and am actually
considering changing bento.info to be a yaml. I initially did got toward a
cabal-like syntax instead for the following reasons:
  - lack of conditional (a must IMO, it is even more useful for cross
-platform stuff than it is for RPM only)
  - yaml becomes quite a bit verbose for some cases

I find JSON to be inappropriate because beyond the above issues, it does
not support comments, and it is significantly more verbose. That being
said, that's just syntax and what matters more is the features we allow:
  - I like the idea of categorizing like you did better than how it works
in bento, but I think one need to be able to create its own category as
well. A category is just a mapping from a name to an install directory (see
http://cournape.github.com/Bento/html/tutorial.html#installed-data-files-datafiles-section,
but we could find another syntax of course).
  - I don't find the distinction between source and build very useful in
the-yet-to-be-implemented description. Or maybe that's just a naming issue,
and it is just the same distinction as extra files vs installed files I
made in bento ? See next point
  - regarding build, I don't think we want to force people to implement
target

Re: [Python-Dev] Status of packaging in 3.3

[Donald Stufft]

I think json probably makes the most sense, it's already part of the stdlib for 
2.6+
and while it has some issues with human editablity, there's no reason why this 
json
file couldn't be auto generated from another data structure by the "package 
creation tool"
that exists outside of the stdlib (or inside, but outside the scope of this 
proposal).

Which is really part of what I like a lot about this proposal, how you come 
about the final
product doesn't matter, distutils, bento, yet-uncreated-tool, manually crafting 
tar balls and files,
you could describe your data in yaml, python, or going towards more magical 
ends of things,
it could be automatically generated from your filesystem. It doesn't matter, 
all that matters is
you create your final archive with the agreed upon structure and the agreed 
upon dist.(yml|json|ini)
and any compliant installer should be able to install it.


On Friday, June 22, 2012 at 3:56 AM, Vinay Sajip wrote:

> Nick Coghlan  gmail.com (http://gmail.com)> writes:
> 
> > ini-style is often good enough, and failing that there's json. Or, you
> > just depend on PyYAML :)
> > 
> 
> 
> Except when PyYAML is packaged and distributed using dist.yaml :-)
> 
> Regards,
> 
> Vinay Sajip
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org (mailto:Python-Dev@python.org)
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/donald.stufft%40gmail.com
> 
> 


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[Vinay Sajip]

Nick Coghlan  gmail.com> writes:

> ini-style is often good enough, and failing that there's json. Or, you
> just depend on PyYAML :)

Except when PyYAML is packaged and distributed using dist.yaml :-)

Regards,

Vinay Sajip

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[Vinay Sajip]

Nick Coghlan  gmail.com> writes:

> 
> On Fri, Jun 22, 2012 at 4:42 PM, Tarek Ziadé  ziade.org> wrote:
> > On 6/22/12 7:05 AM, Nick Coghlan wrote:
> > I don't understand what's the problem is with ini-style files, as they are
> > suitable for multi-line variables etc. (see zc.buildout)
> >
> > yaml vs ini vs xxx seems to be an implementation detail, and my take on this
> > is that we have ConfigParser in the stdlib
> 
> You can't do more than one layer of nested data structures cleanly
> with an ini-style solution, and some aspects of packaging are just
> crying out for metadata that nests more deeply than that. The
> setup.cfg format for specifying installation layouts doesn't even come
> *close* to being intuitively readable - using a format with better
> nesting support has some hope of fixing that, since filesystem layouts
> are naturally hierarchical.
> 
> A JSON based format would also be acceptable to me from a functional
> point of view, although in that case, asking people to edit it
> directly would be cruel - you would want to transform it to YAML in
> order to actually read it or write it.

The format-neutral alternative I used for logging configuration was a dictionary
schema - JSON, YAML and Python code can all be mapped to that. Perhaps the
relevant APIs can work at the dict layer.

I agree that YAML is the human-friendliest "one obvious" format for review/edit,
though.

+1 to the overall approach suggested, it makes a lot of sense. Simple is better
than complex, and all that :-)

Regards,

Vinay Sajip


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[Nick Coghlan]

On Fri, Jun 22, 2012 at 5:24 PM, Tarek Ziadé  wrote:
> On 6/22/12 9:11 AM, Nick Coghlan wrote:
>>
>> On Fri, Jun 22, 2012 at 4:42 PM, Tarek Ziadé  wrote:
>>>
>>> On 6/22/12 7:05 AM, Nick Coghlan wrote:
>>> I don't understand what's the problem is with ini-style files, as they
>>> are
>>> suitable for multi-line variables etc. (see zc.buildout)
>>>
>>> yaml vs ini vs xxx seems to be an implementation detail, and my take on
>>> this
>>> is that we have ConfigParser in the stdlib
>>
>> You can't do more than one layer of nested data structures cleanly
>> with an ini-style solution, and some aspects of packaging are just
>> crying out for metadata that nests more deeply than that. The
>> setup.cfg format for specifying installation layouts doesn't even come
>> *close* to being intuitively readable - using a format with better
>> nesting support has some hope of fixing that, since filesystem layouts
>> are naturally hierarchical.
>>
>> A JSON based format would also be acceptable to me from a functional
>> point of view, although in that case, asking people to edit it
>> directly would be cruel - you would want to transform it to YAML in
>> order to actually read it or write it.
>
>
> I still think this is an implementation detail, and that ini can work here,
> as they have proven to work with buildout and look very clean to me.

Yeah, and I later realised that RPM also uses a flat format. I think
nested is potentially cleaner, but that's the kind of thing a PEP can
thrash out.

> I am now wondering why we don't have a yaml module in the stdlib btw :)

ini-style is often good enough, and failing that there's json. Or, you
just depend on PyYAML :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[Tarek Ziadé]

On 6/22/12 9:11 AM, Nick Coghlan wrote:

On Fri, Jun 22, 2012 at 4:42 PM, Tarek Ziadé  wrote:

On 6/22/12 7:05 AM, Nick Coghlan wrote:
I don't understand what's the problem is with ini-style files, as they are
suitable for multi-line variables etc. (see zc.buildout)

yaml vs ini vs xxx seems to be an implementation detail, and my take on this
is that we have ConfigParser in the stdlib

You can't do more than one layer of nested data structures cleanly
with an ini-style solution, and some aspects of packaging are just
crying out for metadata that nests more deeply than that. The
setup.cfg format for specifying installation layouts doesn't even come
*close* to being intuitively readable - using a format with better
nesting support has some hope of fixing that, since filesystem layouts
are naturally hierarchical.

A JSON based format would also be acceptable to me from a functional
point of view, although in that case, asking people to edit it
directly would be cruel - you would want to transform it to YAML in
order to actually read it or write it.


I still think this is an implementation detail, and that ini can work here,
as they have proven to work with buildout and look very clean to me.

But I guess that's not important -- looking forward for you changes 
proposals on packaging.


I am now wondering why we don't have a yaml module in the stdlib btw :)



Cheers,
Nick.



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[Nick Coghlan]

On Fri, Jun 22, 2012 at 4:42 PM, Tarek Ziadé  wrote:
> On 6/22/12 7:05 AM, Nick Coghlan wrote:
> I don't understand what's the problem is with ini-style files, as they are
> suitable for multi-line variables etc. (see zc.buildout)
>
> yaml vs ini vs xxx seems to be an implementation detail, and my take on this
> is that we have ConfigParser in the stdlib

You can't do more than one layer of nested data structures cleanly
with an ini-style solution, and some aspects of packaging are just
crying out for metadata that nests more deeply than that. The
setup.cfg format for specifying installation layouts doesn't even come
*close* to being intuitively readable - using a format with better
nesting support has some hope of fixing that, since filesystem layouts
are naturally hierarchical.

A JSON based format would also be acceptable to me from a functional
point of view, although in that case, asking people to edit it
directly would be cruel - you would want to transform it to YAML in
order to actually read it or write it.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com