Re: [Python-Dev] Why does _PyUnicode_FromId return a new reference?

2011-11-06 Thread Martin v. Löwis 
Am 05.11.2011 23:26, schrieb Antoine Pitrou:
> 
> Given it returns an eternal object, and it's almost always used
> temporarily (for attribute lookup, string joining, etc.), it would seem
> more practical for it to return a borrowed reference.

For purity reasons: all PyUnicode_From* functions return new references
(most of them return actually new objects most of the time); having
PyUnicode_FromId return a borrowed reference would break uniformity.
I personally it difficult to remember which functions return borrowed
references, and wish there were fewer of them in the API (with
PyArg_ParseTuple being the notable exception where borrowed references
are a good idea).

Now, practicality beats purity, so the real answer is: I just didn't
consider that it might return a borrowed reference.

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Packaging and binary distributions

2011-11-06 Thread Martin v. Löwis 
> I agree in principle, but one thing you get with setup.cfg which seems harder 
> to
> achieve with MSI is the use of Python to do things at installation time. For
> example, with setup.cfg hooks, you can use ctypes to make Windows API calls at
> installation time to decide where to put things. While this same flexibility
> exists in the MSI format (with custom actions and so forth) it's not as 
> readily
> accessible to someone who wants to use Python to code this type of 
> installation
> logic.

Again, that's a bdist_msi implementation issue. It could generate custom
actions that run the "proper" setup.cfg hooks (I presume - I have no
idea what a setup.cfg hook actually is).

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 382 specification and implementation complete

2011-11-06 Thread Martin v. Löwis 
I had announced this to import-sig already; now python-dev.

I have now written an implementation of PEP 382, and fixed some details
of the PEP in the process. The implementation is available at

http://hg.python.org/features/pep-382-2/

With this PEP, a Python package P can consist of either a P/__init__.py
directory and module, or multiple P.pyp directories, or both. Using a
directory suffix resulted from the following requirements/observations:
- people apparently prefer an approach where a directory has to be
  declared as a Python package (several people commented this after
  my PyConDE talk, expressing dislike of how Java packages are
  "unflagged" directories)
- people also commented that any declaration should indicate that this
  is about Python, hence the choice of .pyp as the directory suffix.
- in choosing between a file marker inside of the directory (e.g.
  zope-interfaces.pyp) and a directory suffix, the directory suffix
  wins for simplicity reasons. A file marker would have to have a
  name which wouldn't matter except that it needs to be unique - which
  is a confusing requirement that people likely would fail to meet.

In the new form, the PEP was much easier to implement than in the first
version (plus I understand import.c better now).

This implementation now features .pyp directories, zipimporter support,
documentation and test cases.

As the next step, I'd like to advance this to ask for pronouncement.

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] None as slice params to list.index() and tuple.index()

2011-11-06 Thread Petri Lehtinen 
list.index() and list.tuple() don't currently accept None as slice
parameters, as reported in http://bugs.python.org/issue13340. For
example:

>>> [1, 2, 3].index(2, None, None)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: slice indices must be integers or None or have an __index__ method

The error message of list.index() and tuple.index() (as a consequence
of using _PyEval_SliceIndex() for parsing arguments) indicates that
None is a valid value.

Currently, find(), rfind(), index(), rindex(), count(), startswith()
and endswith() of str, bytes and bytearray accept None. Should
list.index() and tuple.index() accept it, too?

I'm bringing this up because I already committed a fix that enables
the None arguments, but Raymond pointed out that maybe they shouldn't
accept None at all. I also committed the fix to 3.2 and 2.7 based on
discussion in http://bugs.python.org/issue11828#msg133532, in which
Raymond says that for string functions, this can be considered a bug
because the exception's error message makes a promise that None is
accepted.

Petri
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 382 specification and implementation complete

2011-11-06 Thread Nick Coghlan 
On Sun, Nov 6, 2011 at 6:10 PM, "Martin v. Löwis"  wrote:
> I had announced this to import-sig already; now python-dev.
>
> I have now written an implementation of PEP 382, and fixed some details
> of the PEP in the process. The implementation is available at
>
> http://hg.python.org/features/pep-382-2/
>
> With this PEP, a Python package P can consist of either a P/__init__.py
> directory and module, or multiple P.pyp directories, or both. Using a
> directory suffix resulted from the following requirements/observations:
> - people apparently prefer an approach where a directory has to be
>  declared as a Python package (several people commented this after
>  my PyConDE talk, expressing dislike of how Java packages are
>  "unflagged" directories)
> - people also commented that any declaration should indicate that this
>  is about Python, hence the choice of .pyp as the directory suffix.
> - in choosing between a file marker inside of the directory (e.g.
>  zope-interfaces.pyp) and a directory suffix, the directory suffix
>  wins for simplicity reasons. A file marker would have to have a
>  name which wouldn't matter except that it needs to be unique - which
>  is a confusing requirement that people likely would fail to meet.

I finally got around to doing the search Barry and I promised for the
previously raised objections to the directory suffix approach.

They were in the "Rejected Alternatives" section of PJE's proposed
redraft of PEP 382 (before he decided to create PEP 402 as a competing
proposal):

=
*  Another approach considered during revisions to this PEP was to
   simply rename package directories to add a suffix like ``.ns``
   or ``-ns``, to indicate their namespaced nature.  This would effect
   a small performance improvement for the initial import of a
   namespace package, avoid the need to create empty ``*.ns`` files,
   and even make it clearer that the directory involved is a namespace
   portion.

   The downsides, however, are also plentiful.  If a package starts
   its life as a normal package, it must be renamed when it becomes
   a namespace, with the implied consequences for revision control
   tools.

   Further, there is an immense body of existing code (including the
   distutils and many other packaging tools) that expect a package
   directory's name to be the same as the package name.  And porting
   existing Python 2.x namespace packages to Python 3 would require
   widespread directory renaming as well.

   In short, this approach would require a vastly larger number of
   changes to both the standard library and third-party code, for
   a tiny potential performance improvement and a small increase in
   clarity.  It was therefore rejected on "practicality vs. purity"
   grounds.
=

I think this was based on the assumption that *existing* namespace
package approaches would break under the new scheme. Since that is not
the case, I suspect those previous objections were overstated (and all
packaging related code manages to cope well enough with modules where
the file name doesn't match the package name)

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] genious hack in python

2011-11-06 Thread Martin Goudreau 
Write Oleg,

If there is a better way to implemant this? sure. But the idea is still good.




___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 382 specification and implementation complete

2011-11-06 Thread PJ Eby 
On Sun, Nov 6, 2011 at 7:29 AM, Nick Coghlan  wrote:

> I think this was based on the assumption that *existing* namespace
> package approaches would break under the new scheme. Since that is not
> the case, I suspect those previous objections were overstated (and all
> packaging related code manages to cope well enough with modules where
> the file name doesn't match the package name)
>

I was actually referring to all the code that does things like split
package names on '.' and then use os.path.join, or that makes assumptions
which are the moral equivalent of that.  PEP 402's version of namespace
packages should break less of that sort of code than adding a directory
name extension.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyDict_Get/SetItem and dict subclasses

2011-11-06 Thread Antoine Pitrou 

Le 05/11/2011 17:34, Éric Araujo a écrit :

Hi Victor,


PyDict_GetItem() and PyDict_SetItem() don't call __getitem__ and __setitem__
for dict subclasses. Is there a reason for that?


http://bugs.python.org/issue10977 “Currently, the concrete object C API
bypasses any methods defined on subclasses of builtin types.”


I think that's the correct behaviour. If you expect to get an arbitrary 
mapping, just use the abstract API. You should use PyDict_GetItem when 
you know the object is exactly a dict (generally because you have 
created it yourself, or you know at least where and how it was created).


Regards

Antoine.

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why does _PyUnicode_FromId return a new reference?

2011-11-06 Thread Antoine Pitrou 

Le 06/11/2011 08:08, "Martin v. Löwis" a écrit :

Am 05.11.2011 23:26, schrieb Antoine Pitrou:


Given it returns an eternal object, and it's almost always used
temporarily (for attribute lookup, string joining, etc.), it would seem
more practical for it to return a borrowed reference.


For purity reasons: all PyUnicode_From* functions return new references
(most of them return actually new objects most of the time); having
PyUnicode_FromId return a borrowed reference would break uniformity.
I personally it difficult to remember which functions return borrowed
references, and wish there were fewer of them in the API


I agree with this general sentiment. For PyUnicode_FromId, though, I 
think it makes sense to return a borrowed reference.


Regards

Antoine.

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] None as slice params to list.index() and tuple.index()

2011-11-06 Thread Antoine Pitrou 
On Sun, 6 Nov 2011 09:49:27 +0200
Petri Lehtinen  wrote:
> list.index() and list.tuple() don't currently accept None as slice
> parameters, as reported in http://bugs.python.org/issue13340. For
> example:
> 
> >>> [1, 2, 3].index(2, None, None)
> Traceback (most recent call last):
>   File "", line 1, in 
> TypeError: slice indices must be integers or None or have an __index__ method
> 
> The error message of list.index() and tuple.index() (as a consequence
> of using _PyEval_SliceIndex() for parsing arguments) indicates that
> None is a valid value.
> 
> Currently, find(), rfind(), index(), rindex(), count(), startswith()
> and endswith() of str, bytes and bytearray accept None. Should
> list.index() and tuple.index() accept it, too?

Either that or fix the error message. I can't find much benefit in
accepting None, that said (nor in refusing it).

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] None as slice params to list.index() and tuple.index()

2011-11-06 Thread Robert Collins 
On Mon, Nov 7, 2011 at 8:16 AM, Antoine Pitrou  wrote:
> Either that or fix the error message. I can't find much benefit in
> accepting None, that said (nor in refusing it).

Its very convenient when working with slices to not have to special
case the end points. +1 on accepting None, FWIW.

-Rob
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython: Fix #13327. Remove the need for an explicit None as the second argument to

2011-11-06 Thread Benjamin Peterson 
2011/11/6 brian.curtin :
> -
> -    if (!PyArg_ParseTuple(args, "O&O:utime",
> +    PyObject* arg = NULL;

You could set arg = Py_None here.
> +
> +    if (!PyArg_ParseTuple(args, "O&|O:utime",
>                           PyUnicode_FSConverter, &opath, &arg))
>         return NULL;
>     path = PyBytes_AsString(opath);
> -    if (arg == Py_None) {
> +    if (!arg || (arg == Py_None)) {

And then not have to change this.



-- 
Regards,
Benjamin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython: Fix #13327. Remove the need for an explicit None as the second argument to

2011-11-06 Thread Brian Curtin 
On Sun, Nov 6, 2011 at 13:46, Benjamin Peterson  wrote:
> 2011/11/6 brian.curtin :
>> -
>> -    if (!PyArg_ParseTuple(args, "O&O:utime",
>> +    PyObject* arg = NULL;
>
> You could set arg = Py_None here.
>> +
>> +    if (!PyArg_ParseTuple(args, "O&|O:utime",
>>                           PyUnicode_FSConverter, &opath, &arg))
>>         return NULL;
>>     path = PyBytes_AsString(opath);
>> -    if (arg == Py_None) {
>> +    if (!arg || (arg == Py_None)) {
>
> And then not have to change this.

Ah, good point. I'm going to be making this same change to the other
functions in utime family, so I'll look at updating this one and
change the others accordingly.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] None as slice params to list.index() and tuple.index()

2011-11-06 Thread Raymond Hettinger 

On Nov 6, 2011, at 12:49 AM, Petri Lehtinen wrote:

> Currently, find(), rfind(), index(), rindex(), count(), startswith()
> and endswith() of str, bytes and bytearray accept None. Should
> list.index() and tuple.index() accept it, too?

The string methods accept None as a historical artifact
of being in string.py where optional arguments defaulted to None.
That doesn't imply that you should change every other API that
accepts a start argument.

The list.index() API is ancient and stable.  There has been little or
no demonstrated need for its start argument to be None.

Also, the list API does not exist in isolation.  It shows up in
strings, the sequence ABC, and every API that aspires to
be list-like. 

Overall, I'm -1 on this change and find it to be gratuitous.
We have *way* to many micro API changes of dubious benefit.

Also, the change should not have been applied to Py2.7 and Py3.2.
We don't backport API changes.   That would just make Jython
and IronPython become non-compliant in mid-stream.


Raymond


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 382 specification and implementation complete

2011-11-06 Thread Martin v. Löwis 
> I think this was based on the assumption that *existing* namespace
> package approaches would break under the new scheme. Since that is not
> the case, I suspect those previous objections were overstated (and all
> packaging related code manages to cope well enough with modules where
> the file name doesn't match the package name)

I just drafted a message rebutting Phillip's objection, when I then
found that you disagree as well :-)

One elaboration:

>   The downsides, however, are also plentiful.  If a package starts
>   its life as a normal package, it must be renamed when it becomes
>   a namespace, with the implied consequences for revision control
>   tools.

If a package starts out as a regular (P/__init__.py) package
(possibly with __init__.py being empty), then making it a namespace
package actually requires no change at all - that single __init__.py
could continue to exist.

Developers may decide to still delete __init__.py and rename the
package to .pyp (with VCS consequences), but that would be their
choice deciding whether purity beats practicality in their case
(where I think most developers actually would rename the directory
once they can drop support for pre-3.3 releases).

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 382 specification and implementation complete

2011-11-06 Thread Martin v. Löwis 
Am 06.11.2011 16:38, schrieb PJ Eby:
> On Sun, Nov 6, 2011 at 7:29 AM, Nick Coghlan  > wrote:
> 
> I think this was based on the assumption that *existing* namespace
> package approaches would break under the new scheme. Since that is not
> the case, I suspect those previous objections were overstated (and all
> packaging related code manages to cope well enough with modules where
> the file name doesn't match the package name)
> 
> 
> I was actually referring to all the code that does things like split
> package names on '.' and then use os.path.join, or that makes
> assumptions which are the moral equivalent of that.  PEP 402's version
> of namespace packages should break less of that sort of code than adding
> a directory name extension.

I think tools emulating the import mechanism will break no matter what
change is made: the whole point of changing it is that it does something
new that didn't work before.

I think adjusting the tools will be straight-forward: they already need
to recognize that an imported name could come either from a file (with
various extensions), or a directory with special properties. So
extending this should be "easy". Also, the number of tools that emulate
the Python import algorithm is rather small.

Tools that merely inspect __path__ after importing a package will
continue to work just fine even under PEP 382.

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyDict_Get/SetItem and dict subclasses

2011-11-06 Thread Martin v. Löwis 
Am 06.11.2011 17:39, schrieb Antoine Pitrou:
> Le 05/11/2011 17:34, Éric Araujo a écrit :
>> Hi Victor,
>>
>>> PyDict_GetItem() and PyDict_SetItem() don't call __getitem__ and
>>> __setitem__
>>> for dict subclasses. Is there a reason for that?
>>
>> http://bugs.python.org/issue10977 “Currently, the concrete object C API
>> bypasses any methods defined on subclasses of builtin types.”
> 
> I think that's the correct behaviour. If you expect to get an arbitrary
> mapping, just use the abstract API. You should use PyDict_GetItem when
> you know the object is exactly a dict (generally because you have
> created it yourself, or you know at least where and how it was created).

If anybody has spare time at their hands, they should go through the
code base and eliminate all uses of concrete API where it's not certain
that the object really is of the base class (unless I missed that
somebody already did, and that any remaining occurrences would be just
minor bugs).

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyDict_Get/SetItem and dict subclasses

2011-11-06 Thread Raymond Hettinger 

On Nov 6, 2011, at 1:26 PM, Martin v. Löwis wrote:

> Am 06.11.2011 17:39, schrieb Antoine Pitrou:
>> Le 05/11/2011 17:34, Éric Araujo a écrit :
>>> Hi Victor,
>>> 
 PyDict_GetItem() and PyDict_SetItem() don't call __getitem__ and
 __setitem__
 for dict subclasses. Is there a reason for that?
>>> 
>>> http://bugs.python.org/issue10977 “Currently, the concrete object C API
>>> bypasses any methods defined on subclasses of builtin types.”
>> 
>> I think that's the correct behaviour. If you expect to get an arbitrary
>> mapping, just use the abstract API. You should use PyDict_GetItem when
>> you know the object is exactly a dict (generally because you have
>> created it yourself, or you know at least where and how it was created).
> 
> If anybody has spare time at their hands, they should go through the
> code base and eliminate all uses of concrete API where it's not certain
> that the object really is of the base class (unless I missed that
> somebody already did, and that any remaining occurrences would be just
> minor bugs).

Also check uses of PyList_SetItem and other uses of the concrete API.


Raymond

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com