[Python-Dev] feature request: inspect.isgenerator

[Michele Simionato]

Is there still time for new feature requests in Python 2.5?
I am missing a isgenerator function in the inspect module. Right now
I am using

def isgenerator(func):
return func.func_code.co_flags == 99

but it is rather ugly (horrible indeed).


   Michele Simionato


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] ssize_t: ints in header files

[Greg Ewing]

Neal Norwitz wrote:

> Should the following values be ints (limited to 2G)?
> 
>  * line #s and column #s

Anyone with a source file more than 2G lines long
or 2G chars wide deserves whatever they get. :-)

Not-investing-in-a-1.3e10-pixel-wide-screen-any-time-soon-ly,
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] test_gzip/test_tarfile failure om AMD64

[Bob Ippolito]

On May 28, 2006, at 5:34 PM, Thomas Wouters wrote:On 5/29/06, Bob Ippolito <[EMAIL PROTECTED]> wrote: On May 28, 2006, at 4:31 AM, Thomas Wouters wrote:>> I'm seeing a dubious failure of test_gzip and test_tarfile on my> AMD64 machine. It's triggered by the recent struct changes, but I'd> say it's probably caused by a bug/misfeature in zlibmodule: > zlib.crc32 is the result of a zlib 'crc32' functioncall, which> returns an unsigned long. zlib.crc32 turns that unsigned long into> a (signed) Python int, which means a number beyond 1<<31 goes > negative on 32-bit systems and other systems with 32-bit longs, but> stays positive on systems with 64-bit longs:>> (32-bit)> >>> zlib.crc32("foobabazr")> -271938108 >> (64-bit)> >>> zlib.crc32("foobabazr")> 4023029188>> The old structmodule coped with that:> >>> struct.pack(" > '\xc4\x8d\xca\xef'> >>> struct.pack("> '\xc4\x8d\xca\xef'>> The new one does not:> >>> struct.pack("> '\xc4\x8d\xca\xef' > >>> struct.pack("> Traceback (most recent call last):>   File "", line 1, in >   File "Lib/struct.py", line 63, in pack > return o.pack(*args)> struct.error: 'l' format requires -2147483647 <= number <= 2147483647>> The structmodule should be fixed (and a test added ;) but I'm also> wondering if zlib shouldn't be fixed. Now, I'm AMD64-centric, so my > suggested fix would be to change the PyInt_FromLong() call to> PyLong_FromUnsignedLong(), making zlib always return positive> numbers -- it might break some code on 32-bit platforms, but that> code is already broken on 64-bit platforms. But I guess I'm okay > with the long being changed into an actual 32-bit signed number on> 64-bit platforms, too.The struct module isn't what's broken here. All of the struct typeshave always had well defined bit sizes and alignment if you explicitly specify an endian, >I and >L are 32-bits everywhere, and >Q is supported on platforms that don't have long long. The onlything that's changed is that it actually checks for errorsconsistently now. Yes. And that breaks things. I'm certain the behaviour is used in real-world code (and I don't mean just the gzip module.) It has always, as far as I can remember, accepted 'unsigned' values for the signed versions of ints, longs and long-longs (but not chars or shorts.) I agree that that's wrong, but I don't think changing struct to do the right thing should be done in 2.5. I don't even think it should be done in 2.6 -- although 3.0 is fine.Well, the behavior change is in response to a bug . If nothing else, we should at least fix the standard library such that it doesn't depend on struct bugs. This is the only way to find them :)Basically the struct module previously only checked for errors if you don't specify an endian. That's really strange and leads to very confusing results. The only code that really should be broken by this additional check is code that existed before Python had a long type and only signed values were available.-bob___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] test_gzip/test_tarfile failure om AMD64

[Thomas Wouters]

On 5/29/06, Bob Ippolito <[EMAIL PROTECTED]> wrote:
Well, the behavior change is in response to a bug . If nothing else, we should at least fix the standard library such that it doesn't depend on struct bugs. This is the only way to find them :)
Feel free to comment how the zlib.crc32/gzip co-operation should be fixed. I don't see an obviously correct fix. The trunk is currently failing tests it shouldn't fail. Also note that the error isn't with feeding signed values to unsigned formats (which is what the bug is about) but the other way 'round, although I do believe both should be accepted for the time being, while generating a warning.
Basically the struct module previously only checked for errors if you don't specify an endian. That's really strange and leads to very confusing results. The only code that really should be broken by this additional check is code that existed before Python had a long type and only signed values were available.
Alas, reality is different. The fundamental difference between types in Python and in C causes this, and code using struct is usually meant specifically to bridge those two worlds. Furthermore, struct is often used *fix* that issue, by flipping sign bits if necessary:
>>> struct.unpack("(-1073741824,)>>> struct.unpack("(-1073741824,)
>>> struct.unpack("(-1073741824,)>>> struct.unpack("(-1073741824,)
Before this change, you didn't have to check whether the value is negative before the struct.unpack/pack dance, regardless of which format character you used. This misfeature is used (and many would consider it convenient, even Pythonic, for struct to DWIM), breaking it suddenly is bad.
-- Thomas Wouters <[EMAIL PROTECTED]>Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove METH_OLDARGS?

[Georg Brandl]

Neal Norwitz wrote:

>> These modules can be converted.
> 
> We can convert the one's in Python, but not the third party ones.
> It's also very difficult to search for something that's not there.
> You need to review every PyMethodDef entry to verify it has a METH_*
> flag.  These implicit METH_OLDARGS may well exist in Python,
> especially given who did the patch to add them in the first place. :-)
> 
> http://mail.python.org/pipermail/python-dev/2002-March/022009.html
> http://mail.python.org/pipermail/patches/2002-July/009023.html

I'll gradually try to convert them.

>> > I would like to get rid of the flag, but I'm not sure we can do it
>> > safely until 3.0.
>>
>> It has been deprecated since 2.2, so it'd be no surprise if it went away.
> 
> There's a difference between "it'd be no surprise" and "it should be
> no surpise".  While I agree in theory they are the same, in practice,
> they are not.  That's the difference between theory and practice. :-)

Yep, you're right.

>> Also, it would be easy to detect METH_OLDARGS in PyCFunction_New and raise
>> an appropriate exception.
> 
> I agree with Martin this should raise a deprecation warning in 2.5.

Patch is at #1496957 (another one for converting Tkinter is at #1496952).

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] ssize_t question: longs in header files

[Thomas Wouters]

On 5/29/06, Neal Norwitz <[EMAIL PROTECTED]> wrote:
 * ints:  Include/intobject.h:long ob_ival;I considered asking about this before, as it would give '64-bit power' to Win64 integers. It's a rather big change, though (lots of code assumes PyInts fit in signed longs, which would be untrue then.)
On the other hand, if a separate type was used for PyInts (Py_pyint_t or whatever, defaulting to long, except on LLP64 systems), EWT's request for a 64-bit integer represented by C 'long long's would be a simple configure switch.
-- Thomas Wouters <[EMAIL PROTECTED]>Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] test_gzip/test_tarfile failure om AMD64

[Bob Ippolito]

On May 29, 2006, at 3:14 AM, Thomas Wouters wrote:On 5/29/06, Bob Ippolito <[EMAIL PROTECTED]> wrote: Well, the behavior change is in response to a bug . If nothing else, we should at least fix the standard library such that it doesn't depend on struct bugs. This is the only way to find them :) Feel free to comment how the zlib.crc32/gzip co-operation should be fixed. I don't see an obviously correct fix. The trunk is currently failing tests it shouldn't fail. Also note that the error isn't with feeding signed values to unsigned formats (which is what the bug is about) but the other way 'round, although I do believe both should be accepted for the time being, while generating a warning. Well, first I'm going to just correct the modules that are broken (zlib, gzip, tarfile, binhex and probably one or two others).Basically the struct module previously only checked for errors if you don't specify an endian. That's really strange and leads to very confusing results. The only code that really should be broken by this additional check is code that existed before Python had a long type and only signed values were available. Alas, reality is different. The fundamental difference between types in Python and in C causes this, and code using struct is usually meant specifically to bridge those two worlds. Furthermore, struct is often used *fix* that issue, by flipping sign bits if necessary: Well, in C you get a compiler warning for stuff like this.>>> struct.unpack("(-1073741824,)>>> struct.unpack("(-1073741824,) >>> struct.unpack("(-1073741824,)>>> struct.unpack("(-1073741824,) Before this change, you didn't have to check whether the value is negative before the struct.unpack/pack dance, regardless of which format character you used. This misfeature is used (and many would consider it convenient, even Pythonic, for struct to DWIM), breaking it suddenly is bad. struct doesn't really DWIM anyway, since integers are up-converted to longs and will overflow past what the (old or new) struct module will accept. Before there was a long type or automatic up-converting, the sign agnosticism worked.. but it doesn't really work correctly these days.We have two choices, either fix it to behave consistently broken everywhere for numbers of every size (modulo every number that comes in so that it fits), or have it do proper range checking. A compromise is to do proper range checking as a warning, and do the modulo math anyway... but is that what we really want?-bob___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] test_gzip/test_tarfile failure om AMD64

[Thomas Wouters]

On 5/29/06, Bob Ippolito <[EMAIL PROTECTED]> wrote:
A compromise is to do proper range checking as a warning, and do the modulo math anyway... but is that what we really want?I don't know about the rest of 'us', but that's what I want, yes: backward compatibility, and a warning to tell people to fix their code 'or else'. The prevalence of the warnings (outside of the stdlib) should give us a clue whether to make it an exception in 
2.6 or wait for 2.7/3.0.Perhaps more people could chime in? Am I being too anal about backward compatibility here?-- Thomas Wouters <[EMAIL PROTECTED]
>Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Syntax errors on continuation lines

[Aahz]

On Sun, May 28, 2006, Roger Miller wrote:
>
> I am a recent subscriber to this list.  A couple of months ago
> I downloaded the Python source out of curiosity, then decided
> to see if I could scratch a couple of small itches.

Please post your patches to SourceForge, then post links back to the
list.
-- 
Aahz ([EMAIL PROTECTED])   <*> http://www.pythoncraft.com/

"I saw `cout' being shifted "Hello world" times to the left and stopped
right there."  --Steve Gonedes
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Syntax errors on continuation lines

[Michael Foord]

Aahz wrote:
> On Sun, May 28, 2006, Roger Miller wrote:
>   
>> I am a recent subscriber to this list.  A couple of months ago
>> I downloaded the Python source out of curiosity, then decided
>> to see if I could scratch a couple of small itches.
>> 
+1 on the change, the error messages that report the wrong line can be 
very confusing.

Michael Foord

>
> Please post your patches to SourceForge, then post links back to the
> list.
>   

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] DRAFT: python-dev summary for 2006-04-01 to 2006-04-15

[Steven Bethard]

Ok, here comes the first half of April.  Again, please let me know if
you have any comments or corrections (and thanks to those who already
got back to me on the last one).  I'm gonna try to keep these coming
over the next week or so until I'm all caught up.



=
Announcements
=

-
Python 2.5a1 Released
-

Python 2.5 alpha 1 was released on April 5th.  Please download it and
try it out, particularly if you are an extension writer or you embed
Python -- you may want to change things to support 64-bit sequences,
and if you have been using mismatched PyMem_* and PyObject_*
allocation calls, you will need to fix these as well.

Contributing threads:

- `TRUNK FREEZE. 2.5a1, 00:00 UTC, Wednesday 5th of April.
`__
- `outstanding items for 2.5
`__
- `chilling svn for 2.5a1
`__
- `Reminder: TRUNK FREEZE. 2.5a1, 00:00 UTC, Wednesday 5th of April.
`__
- `RELEASED Python 2.5 (alpha 1)
`__
- `TRUNK is UNFROZEN
`__
- `segfault (double free?) when '''-string crosses line
`__
- `pdb segfaults in 2.5 trunk?
`__
- `IMPORTANT 2.5 API changes for C Extension Modules and Embedders
`__

--
Contributor agreements
--

If you're listed in the `Python acknowledgments`_, and you haven't
signed a `contributor agreement`_, please submit one as soon as
possible.  Note that this includes even the folks that have been
around forever -- the PSF would like to be as careful as possible on
this one.

.. _Python acknowledgments:
http://svn.python.org/projects/python/trunk/Misc/ACKS
.. _contributor agreement: http://www.python.org/psf/contrib-form-python.html

Contributing threads:

- `PSF Contributor Agreement for pysqlite
`__

---
Firefox Python bug searches
---

Anthony Baxter has created a `Firefox searchbar`_ for finding Python
bugs by their SourceForge IDs.  There are also two Firefox sidebar
options: `Mark Hammond's`_ and `Daniel Lundin's`_.

.. _Firefox searchbar: http://www.python.org/~anthony/searchbar/
.. _Mark Hammond's: http://starship.python.net/~skippy/mozilla/
.. _Daniel Lundin's: http://projects.edgewall.com/python-sidebar/

Contributing thread:

- `Firefox searchbar engine for Python bugs
`__

-
Building Python with the free MS Toolkit compiler
-

Paul Moore documented how to build Python on Windows with the free MS
Toolkit C++ compiler `on the wiki`_.  The most up-to-date version of
these instructions is in `PCbuild/readme.txt`_.

.. _on the wiki:
http://wiki.python.org/moin/Building_Python_with_the_free_MS_C_Toolkit
.. _PCbuild/readme.txt:
http://svn.python.org/projects/python/trunk/PCbuild/readme.txt

Contributing thread:

- `Building Python with the free MS Toolkit compiler
`__

--
Please login to the wiki when you make changes
--

Skip Montanaro has requested that anyone posting to the wiki sign in
first, as this makes things easier for those monitoring changes to the
wiki. When you're logged in, your changes appear with your name, and
so can be immediately recognized as not being wiki-spam.

Contributing thread:

- `Suggestion: Please login to wiki when you make changes
`__

---
Checking an older blamelist
---

While the buildbot blamelist may scroll off the `main page`_ in a
matter of hours, you can still see the blamelist for a particular
revision by checking a particular build number, e.g. to see build 800
of the trunk on the G4 OSX, you could check
http://www.python.org/dev/buildbot/trunk/g4%20osx.4%20trunk/builds/800.

.. _main page: http://www.python.org/dev/buildbot/all/

Contributing threads:

- `Preserving the blamelist
`__
- `TODO Wiki (was: Preserving the blamelist)
`__

=
Summaries
=

-
Garb

Re: [Python-Dev] epoll implementation

[Barry Scott]

On May 27, 2006, at 04:59, Alex Martelli wrote:

>
> I believe it's: kqueue on FreeBSD (for recent-enough versions
> thereof), otherwise epoll where available and nonbuggy, otherwise
> poll ditto, otherwise select -- that's roughly what Twisted uses for

kqueue is not always faster. It depends on your application. The number
of FDs and the fd values affect the point that kqueue wins over select.

For a few FDs with low values select was always faster in my apps.

Barry

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Let's stop eating exceptions in dict lookup

[Armin Rigo]

Hi all,

I've finally come around to writing a patch that stops dict lookup from
eating all exceptions that occur during lookup, like rare bugs in user
__eq__() methods.  After another 2-hours long debugging session that
turned out to be caused by that, I had a lot of motivation.

  http://python.org/sf/1497053

The patch doesn't change the PyDict_GetItem() interface, which is the
historical core of the problem.  It works around this issue by just
moving the exception-eating bit there instead of in lookdict(), so it
gets away with changing only dictobject.c (plus ceval.c's direct usage
of ma_lookup for LOAD_GLOBAL).  The benefit of this patch is that all
other ways to work with dicts now correctly propagate exceptions, and
this includes all the direct manipulation from Python code (including
'x=d[key]').

The reason I bring this up here is that I'm going to check it in 2.5,
unless someone seriously objects.  About the objection "we need a better
fix, PyDict_GetItem() should really be fixed and all its usages
changed": this would be good, and also require some careful
compatibility considerations, and quite some work in total; it would
also give a patch which is basically a superset of mine, so I don't
think I'm going in the wrong direction there.


A bientot,

Armin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] ssize_t question: longs in header files

[Tim Peters]

[Neal Norwitz]
>> * ints:  Include/intobject.h:long ob_ival;

[Thomas Wouters]
> I considered asking about this before, as it would give '64-bit power' to
> Win64 integers. It's a rather big change, though (lots of code assumes
> PyInts fit in signed longs, which would be untrue then.)

I expect that "rather big" is understatement -- more that the sheer
bulk of it would make the Py_ssize_t changes look like a warmup
exercise.  Too late in the release cycle, IMO (and note that we're
still stumbling into Py_ssize_t glitches despite that it's been in
place for months).

> On the other hand, if a separate type was used for PyInts (Py_pyint_t or
> whatever, defaulting to long, except on LLP64 systems), EWT's request for a
> 64-bit integer represented by C 'long long's would be a simple configure
> switch.

But then _all_ "short ints" in Python would be 64 bits, and EWT did
not ask for a slowdown on all short int operations <0.5 wink>.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] ssize_t question: longs in header files

[Tim Peters]

[Neal Norwitz]
>  * hash values
> Include/abstract.h: long PyObject_Hash(PyObject *o);  // also in object.h
> Include/object.h:typedef long (*hashfunc)(PyObject *);

We should leave these alone for now.  There's no real connection
between the width of a hash value and the number of elements in a
container, and Py_ssize_t is conceptually only related to the latter.

As you noted later, there is an unusually strong connection in the
dict implementation, and it's been on my todo list for months to look
into that.  Certainly the struct _dictobject members need to change
from int to Py_ssize_t (including ma_mask), but that can't be done
mechanically (code changes may also be required, and this is one is
much more math- than compiler-warning- driven).

I'll note in passing that the debug-build obmalloc has no idea what to
do with byte counts that don't fit in 4 bytes (it will probably
left-truncate them in its internal debugging records).
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] test_gzip/test_tarfile failure om AMD64

[Tim Peters]

[Thomas Wouters]
> ...
> Perhaps more people could chime in? Am I being too anal about backward
> compatibility here?

Yes and no ;-)  Backward compatibility _is_ important, but there seems
no way to know in this case whether struct's range-checking sloppiness
was accidental or deliberate.  Having fixed bugs in the old code
several times, and been reduced to writing crap like this in the old
test_struct.py:

# XXX Most std integer modes fail to test for out-of-range.
# The "i" and "l" codes appear to range-check OK on 32-bit boxes, but
# fail to check correctly on some 64-bit ones (Tru64 Unix + Compaq C
# reported by Mark Favas).
BUGGY_RANGE_CHECK = "bBhHiIlL"

I can't help but note several things:

- If it _was_ intended that range-checking be sloppy, nobody
  bothered to document it.

- Or even to write a comment in the code explaining that obscure
  intent.

- When I implemented the Q (8-byte int) format code, I added
  correct range-checking in all cases, and nobody ever complained
  about that.

- As noted in the comment above, we have gotten complaints about
  failures of struct range-checking at other integer widths.

OTOH,  BUGGY_RANGE_CHECK existed because I was too timid to risk
making broken user code visibly broken.

So, in all, I'm 95% sure 2.4's behavior is buggy, but 50% unsure that
we need to warn about it before repairing it.  Since you (Thomas) want
warnings, and in theory it only affects the lightly-used "standard"
modes, I do lean in favor of leaving the standard modes that _are_
broken (as above, not all are) broken in 2.5 but warning that this
will change in 2.6.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] test_gzip/test_tarfile failure om AMD64

[Josiah Carlson]

"Thomas Wouters" <[EMAIL PROTECTED]> wrote:
> On 5/29/06, Bob Ippolito <[EMAIL PROTECTED]> wrote:
> >
> > A compromise is to do proper range checking as a warning, and do the
> > modulo math anyway... but is that what we really want?
> >
> 
> I don't know about the rest of 'us', but that's what I want, yes: backward
> compatibility, and a warning to tell people to fix their code 'or else'. The
> prevalence of the warnings (outside of the stdlib) should give us a clue
> whether to make it an exception in 2.6 or wait for 2.7/3.0.
> 
> Perhaps more people could chime in? Am I being too anal about backward
> compatibility here?

As a fairly heavy user of struct, I personally don't use struct to do
modulos and/or sign manipulation (I mask before I pass), but a change in
behavior seems foolish if people use that behavior.  So far, I'm not
aware of anyone complaining about Python 2.4's use, so it would seem to
suggest that the current behavior is not incorrect.

 - Josiah

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] test_gzip/test_tarfile failure om AMD64

[Guido van Rossum]

On 5/29/06, Tim Peters <[EMAIL PROTECTED]> wrote:
> [Thomas Wouters]
> > ...
> > Perhaps more people could chime in? Am I being too anal about backward
> > compatibility here?
>
> Yes and no ;-)  Backward compatibility _is_ important, but there seems
> no way to know in this case whether struct's range-checking sloppiness
> was accidental or deliberate.

I'm pretty sure it was deliberate. I'm more than likely the original
author of this code (since the struct module was originally mine), and
I know I put in things like that in a few places to cope with hex/oct
constants pasted from C headers, and the general messiness that ensued
because of Python's lack of an unsigned int type.

This is probably a case that was missed by PEP 237.

I think we should do as Thomas proposes: plan to make it an error in
2.6 (or 2.7 if there's a big outcry, which I don't expect) and accept
it with a warning in 2.5.

> - If it _was_ intended that range-checking be sloppy, nobody
>   bothered to document it.

Mea culpa. In those days we didn't document a lot of things.

> - Or even to write a comment in the code explaining that obscure
>   intent.

It never occurred to me that it was obscure; we did this all over the
place (in PyArg_Parse too).

> - When I implemented the Q (8-byte int) format code, I added
>   correct range-checking in all cases, and nobody ever complained
>   about that.

It's really only a practical concern for 32-bit values on 32-bit
machines, where reasonable people can disagree over whether 0x
is -1 or 4294967295.

> - As noted in the comment above, we have gotten complaints about
>   failures of struct range-checking at other integer widths.
>
> OTOH,  BUGGY_RANGE_CHECK existed because I was too timid to risk
> making broken user code visibly broken.
>
> So, in all, I'm 95% sure 2.4's behavior is buggy, but 50% unsure that
> we need to warn about it before repairing it.  Since you (Thomas) want
> warnings, and in theory it only affects the lightly-used "standard"
> modes, I do lean in favor of leaving the standard modes that _are_
> broken (as above, not all are) broken in 2.5 but warning that this
> will change in 2.6.

I'm not sure what we gain by leaving other std modules depending on
struct's brokenness broken. But I may be misinterpreting which modules
you're referring to.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Let's stop eating exceptions in dict lookup

[Raymond Hettinger]

From: "Armin Rigo" <[EMAIL PROTECTED]>
 
> I've finally come around to writing a patch that stops dict lookup from
> eating all exceptions that occur during lookup, like rare bugs in user
> __eq__() methods. 

Is there a performance impact?


Raymond

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] test_gzip/test_tarfile failure om AMD64

[Tim Peters]

[Guido]
> ...
> It's really only a practical concern for 32-bit values on 32-bit
> machines, where reasonable people can disagree over whether 0x
> is -1 or 4294967295.

Then maybe we should only let that one slide <0.5 wink>.

...

[Tim]
>> So, in all, I'm 95% sure 2.4's behavior is buggy, but 50% unsure that
>> we need to warn about it before repairing it.  Since you (Thomas) want
>> warnings, and in theory it only affects the lightly-used "standard"
>> modes, I do lean in favor of leaving the standard modes that _are_
>> broken (as above, not all are) broken in 2.5 but warning that this
>> will change in 2.6.

> I'm not sure what we gain by leaving other std modules depending on
> struct's brokenness broken. But I may be misinterpreting which modules
> you're referring to.

I think you're just reading "module" where I wrote "mode".  "Standard
mode" is struct-module terminology, as in

">b"
"!b"
">> struct.pack(">B", -32) # std mode doesn't complain
'\xe0'
>>> struct.pack("B", -32)  # native mode does
Traceback (most recent call last):
  File "", line 1, in ?
struct.error: ubyte format requires 0<=number<=255

>>> struct.pack(">b", 255)  # std mode doesn't complain
'\xff'
>>> struct.pack("b", 255) # native mode does
Traceback (most recent call last):
  File "", line 1, in ?
struct.error: byte format requires -128<=number<=127

On the other hand, as I noted last time, some standard modes _do_
range-check -- but not correctly on some 64-bit boxes -- and not
consistently across positive and negative out-of-range values, or
across input types.  Like:

>>> struct.pack(">i", 2**32-1)  # std and native modes complain
Traceback (most recent call last):
  File "", line 1, in ?
OverflowError: long int too large to convert to int
>>> struct.pack("i", 2**32-1)
Traceback (most recent call last):
  File "", line 1, in ?
OverflowError: long int too large to convert to int

>>> struct.pack("I", -1)  # neither std nor native modes complain
'\xff\xff\xff\xff'
>>> struct.pack(">I", -1)
'\xff\xff\xff\xff'

>>> struct.pack("I", -1L)  # but both complain if the input is long
Traceback (most recent call last):
  File "", line 1, in ?
OverflowError: can't convert negative value to unsigned long
>>> struct.pack(">I", -1L)
Traceback (most recent call last):
  File "", line 1, in ?
OverflowError: can't convert negative value to unsigned long

In short, there's no way to explain what struct checks for in 2.4.3
short of drawing up an exhaustive table of standard-vs-native mode,
format code, "which direction" a value may be out of range, and
whether the value is given as a Python int or a long.

At the sprint, I encouraged Bob to do complete range-checking.  That's
explainable.  If we have to back off from that, then since the new
code is consistent, I'm sure any warts he puts back in will be clearly
look like warts ;-)

> I think we should do as Thomas proposes: plan to make it an error in
> 2.6 (or 2.7 if there's a big outcry, which I don't expect) and accept
> it with a warning in 2.5.

That's what I arrived at, although 2.4.3's checking behavior is
actually so inconsistent that "it" needs some defining (what exactly
are we trying to still accept?  e.g., that -1 doesn't trigger "I"
complaints but that -1L does above?  that one's surely a bug).   To be
clear, Thomas proposed "accepting it" (whatever that means) until 3.0.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Let's stop eating exceptions in dict lookup

[Guido van Rossum]

On 5/29/06, Armin Rigo <[EMAIL PROTECTED]> wrote:
> Hi all,
>
> I've finally come around to writing a patch that stops dict lookup from
> eating all exceptions that occur during lookup, like rare bugs in user
> __eq__() methods.  After another 2-hours long debugging session that
> turned out to be caused by that, I had a lot of motivation.
>
>   http://python.org/sf/1497053
>
> The patch doesn't change the PyDict_GetItem() interface, which is the
> historical core of the problem.  It works around this issue by just
> moving the exception-eating bit there instead of in lookdict(), so it
> gets away with changing only dictobject.c (plus ceval.c's direct usage
> of ma_lookup for LOAD_GLOBAL).  The benefit of this patch is that all
> other ways to work with dicts now correctly propagate exceptions, and
> this includes all the direct manipulation from Python code (including
> 'x=d[key]').
>
> The reason I bring this up here is that I'm going to check it in 2.5,
> unless someone seriously objects.

+1, as long as (as you seem to imply) PyDict_GetItem() still swallows
all exceptions.

> About the objection "we need a better
> fix, PyDict_GetItem() should really be fixed and all its usages
> changed": this would be good, and also require some careful
> compatibility considerations, and quite some work in total; it would
> also give a patch which is basically a superset of mine, so I don't
> think I'm going in the wrong direction there.

Fixing PyDict_GetItem() is a py3k issue, I think. Until then, there
are way too many uses. I wouldn't be surprised if after INCREF and
DECREF it's the most commonly used C API method...

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] feature request: inspect.isgenerator

[Terry Reedy]

"Michele Simionato" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
> Is there still time for new feature requests in Python 2.5?
> I am missing a isgenerator function in the inspect module. Right now
> I am using
>
> def isgenerator(func):
>return func.func_code.co_flags == 99
>
> but it is rather ugly (horrible indeed).

To me, part of the beauty of Python is that is so easy to write such things 
so compactly, so that we do not need megaAPIs with hundreds of functions 
and methods.

tjr



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] test_gzip/test_tarfile failure om AMD64

[Guido van Rossum]

On 5/29/06, Tim Peters <[EMAIL PROTECTED]> wrote:
> > I think we should do as Thomas proposes: plan to make it an error in
> > 2.6 (or 2.7 if there's a big outcry, which I don't expect) and accept
> > it with a warning in 2.5.
>
> That's what I arrived at, although 2.4.3's checking behavior is
> actually so inconsistent that "it" needs some defining (what exactly
> are we trying to still accept?  e.g., that -1 doesn't trigger "I"
> complaints but that -1L does above?  that one's surely a bug).

No, it reflects that (up to 2.3 I believe) 0x was -1 but
0xL was 4294967295L.

> To be
> clear, Thomas proposed "accepting it" (whatever that means) until 3.0.

Fine with me.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] test_gzip/test_tarfile failure om AMD64

[Terry Reedy]

>Perhaps more people could chime in? Am I being too anal about backward 
> >compatibility here?

As a sometimes bug report reviewer, I would like the reported discrepancy 
between the public docs and visible code behavior fixed one way or the 
other (by changing the docs or code) since that is my working definition 
for whether something is a bug or not.

Terry Jan Reedy




___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove METH_OLDARGS?

[Guido van Rossum]

On 5/28/06, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
> Georg Brandl wrote:
> >> There's still a ton used under Modules.  Also, if no flag is
> >> specified, it will default to 0 (ie, METH_OLDARGS).  I wonder how many
> >> third party modules use METH_OLDARGS directly or more likely
> >> indirectly.
> >
> > These modules can be converted.
>
> I think that should be done for 2.5, but nothing else. Or, perhaps
> a deprecation warning could be generated if it is still used.

I'll let Martin decide this one.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Let's stop eating exceptions in dict lookup

[Armin Rigo]

Hi Guido,

On Mon, May 29, 2006 at 12:34:30PM -0700, Guido van Rossum wrote:
> +1, as long as (as you seem to imply) PyDict_GetItem() still swallows
> all exceptions.

Yes.

> Fixing PyDict_GetItem() is a py3k issue, I think. Until then, there
> are way too many uses. I wouldn't be surprised if after INCREF and
> DECREF it's the most commonly used C API method...

Alternatively, we could add a new C API function, and gradually replace
PyDict_GetItem() uses with the the new one.  I can't think of an obvious
name, though...

Maybe code should just start using PyMapping_GetItem() instead.  It's
not incredibly slower than PyDict_GetItem(), at least in the
non-KeyError case.


A bientot,

Armin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Let's stop eating exceptions in dict lookup

[Armin Rigo]

Hi Raymond,

On Mon, May 29, 2006 at 12:20:44PM -0700, Raymond Hettinger wrote:
> > I've finally come around to writing a patch that stops dict lookup from
> > eating all exceptions that occur during lookup, like rare bugs in user
> > __eq__() methods. 
> 
> Is there a performance impact?

I believe that this patch is good anyway, because I consider my (and
anybody's) debugging hours worth more than a few seconds of a
long-running process.  You get *really* obscure bugs this way.

I would also point out that this is the kind of feature that should not
be traded off for performance, otherwise we'd loose much of the point of
Python.  IMHO.

As it turns out, I measured only 0.5% performance loss in Pystone.


A bientot,

Armin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] r46300 - in python/trunk: Lib/socket.py Lib/test/test_socket.py Lib/test/test_struct.py Modules/_struct.c Modules/arraymodule.c Modules/socketmodule.c

[Guido van Rossum]

[python-checkins]
> >> * Added socket.recv_buf() and socket.recvfrom_buf() methods, that
> >> use the buffer
> >>   protocol (send and sendto already did).
> >>
> >> * Added struct.pack_to(), that is the corresponding buffer
> >> compatible method to
> >>   unpack_from().

[Guido]
> > Hm... The file object has a similar method readinto(). Perhaps the
> > methods introduced here could follow that lead instead of using two
> > different new naming conventions?

On 5/27/06, Bob Ippolito <[EMAIL PROTECTED]> wrote:
> (speaking specifically about struct and not socket)
>
> pack_to and unpack_from are named as such because they work with objects
> that support the buffer API (not file-like-objects). I couldn't find any
> existing convention for objects that manipulate buffers in such a way.

Actually, .readinto() is the convention I
started long ago (at the time the buffer was introduced.

> If there is an existing convention then I'd be happy to rename these.
>
> "readinto" seems to imply that some kind of position is being
> incremented.

No -- it's like read() but instead of returning a new object it reads
"into" an object you pass.

> Grammatically it only works if it's implemented on all buffer
> objects, but
> in this case it's implemented on the Struct type.

It looks like .pack_to(, ) is
similar to .readinto() in that the buffer
object receives the result of packing / reading.

(Files don't have a writefrom() method because write() can be
polymorphic. read() can't be due to the asymmetry in the API. I guess
struct objects are different.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] feature request: inspect.isgenerator

[Fredrik Lundh]

Terry Reedy wrote:

>> def isgenerator(func):
>>return func.func_code.co_flags == 99
>>
>> but it is rather ugly (horrible indeed).
> 
> To me, part of the beauty of Python is that is so easy to write such things 
> so compactly, so that we do not need megaAPIs with hundreds of functions 
> and methods.

so what's so "easy" about the magic constant 99 ?

is there some natural and obvious connection between generators and that 
number that I'm missing, or is that constant perhaps best hidden inside 
some introspection support module?

(I'm still not sure about the use case, though.  shouldn't a program 
treat a generator just like any other callable that returns an iterator?)



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 42 - New kind of Temporary file

[Guido van Rossum]

I think it's a very minor feature at best -- but I do admit to having
implemented this at least once so I'm not against adding it as a new
feature to the tempfile module. It's getting awfully late for 2.5
though... (feature freeze is at 2.5b1 which is planned for June 14
IIRC).

--Guido

On 5/28/06, Andrew Chambers <[EMAIL PROTECTED]> wrote:
> The mailing-list bot said introduce myself so...
>
> I'm Andy Chambers.  I've been lurking on the web-interface to the list
> for a while.  I recently started trying to implement one of the items on
> PEP 42 so I thought I should join the list and make myself known.
>
> The item I'm working on is the new kind of temporary file that only
> turns into a file when required (before which time presumably its a
> StringIO).
>
> Do we still want this?  I suggest it be exposed as a class TempFile in
> the tempfile module.  If yes, I'll write the test cases and submit a
> patch.
>
> Regards,
> Andy
>
>
>
> ___
> Inbox full of spam? Get leading spam protection and 1GB storage with All New 
> Yahoo! Mail. http://uk.docs.yahoo.com/nowyoucan.html
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove METH_OLDARGS?

[Fredrik Lundh]

Neal Norwitz wrote:

>> Also, it would be easy to detect METH_OLDARGS in PyCFunction_New and raise
>> an appropriate exception.
> 
> I agree with Martin this should raise a deprecation warning in 2.5.

why?

this is a clear case of unnecessary meddling.  removing it won't remove 
much code (a whopping 11 lines is dedicated to this), nor give any speed 
ups whatsoever; all you're doing is generating extra work and support 
issues for a bunch of third-party developers.  trust me, we have better 
things to do with our time.

-1 on meddling with this before 3.0.



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] feature request: inspect.isgenerator

[Georg Brandl]

Fredrik Lundh wrote:
> Terry Reedy wrote:
> 
>>> def isgenerator(func):
>>>return func.func_code.co_flags == 99
>>>
>>> but it is rather ugly (horrible indeed).
>> 
>> To me, part of the beauty of Python is that is so easy to write such things 
>> so compactly, so that we do not need megaAPIs with hundreds of functions 
>> and methods.
> 
> so what's so "easy" about the magic constant 99 ?
> 
> is there some natural and obvious connection between generators and that 
> number that I'm missing, or is that constant perhaps best hidden inside 
> some introspection support module?

It seems to be a combination of CO_NOFREE, CO_GENERATOR, CO_OPTIMIZED and
CO_NEWLOCALS.

The first four CO_ constants are already in inspect.py, the newer ones
(like CO_GENERATOR) aren't.

I wonder whether a check shouldn't just return (co_flags & 0x20), which
is CO_GENERATOR.

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Let's stop eating exceptions in dict lookup

[Fredrik Lundh]

Armin Rigo wrote:

> As it turns out, I measured only 0.5% performance loss in Pystone.

umm.  does Pystone even call lookdict?



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] test_gzip/test_tarfile failure om AMD64

[Bob Ippolito]

On May 29, 2006, at 12:44 PM, Guido van Rossum wrote:

> On 5/29/06, Tim Peters <[EMAIL PROTECTED]> wrote:
>>> I think we should do as Thomas proposes: plan to make it an error in
>>> 2.6 (or 2.7 if there's a big outcry, which I don't expect) and  
>>> accept
>>> it with a warning in 2.5.
>>
>> That's what I arrived at, although 2.4.3's checking behavior is
>> actually so inconsistent that "it" needs some defining (what exactly
>> are we trying to still accept?  e.g., that -1 doesn't trigger "I"
>> complaints but that -1L does above?  that one's surely a bug).
>
> No, it reflects that (up to 2.3 I believe) 0x was -1 but
> 0xL was 4294967295L.

Python 2.3 did a FutureWarning on 0x but its value was -1.

Anyway, my plan is to make it such that all non-native format codes  
will behave exactly like C casting, but will do a DeprecationWarning  
for input numbers that were initially out of bounds. This behavior  
will be consistent across (python) int and long, and will be easy  
enough to explain in the docs (but still more complicated than  
"values not representable by this data type will raise struct.error").

This means that I'm also changing it so that struct.pack will not  
raise OverflowError for some longs, it will always raise struct.error  
or do a warning (as long as the input is int or long).

Pseudocode looks kinda like this:

def wrap_unsigned(x, CTYPE):
if not (0 <= x <= CTYPE_MAX):
DeprecationWarning()
x &= CTYPE_MAX
return x

Actually, should this be a FutureWarning or a DeprecationWarning?

-bob

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Contributor agreements (was Re: DRAFT: python-dev summary for 2006-04-01 to 2006-04-15)

[Terry Reedy]

--
Contributor agreements
--

If you're listed in the `Python acknowledgments`_, and you haven't
signed a `contributor agreement`_, please submit one as soon as
possible.  Note that this includes even the folks that have been
around forever -- the PSF would like to be as careful as possible on
this one.
---
I believe I did submit something some time ago, but I am not positive, and 
even if I did, I don't know whether it was a recent enough version.  So I 
suggest that

>.. _Python acknowledgments:
>  http://svn.python.org/projects/python/trunk/Misc/ACKS

put a star besides names for which there is a recent enough form.
Another problem with that list is that it apparently includes people who 
have not actually contributed text or code in the distribution and who 
therefor do not (?) need to send in an agreement.

Also, it says 'not in any useful order', which is weird, because it is 
alphabetical, which is quite useful for finding a particular name (or not 
;-).

>.. _contributor agreement: 
>http://www.python.org/psf/contrib-form-python.html

By itself, this link is inadequate since there is a blank for 'Initial 
License' with no choices indicated.  So the summary needs a link to an 
instruction page also.  The form is also a bit confusing since it appears 
to talk about future contributions but the last line tacked on to the 
bottom refers to the past.  If it means both, then why not say so in the 
main text in a way that non-lawyers can understand?

Terry Jan Reedy



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] test_gzip/test_tarfile failure om AMD64

[Tim Peters]

[Guido]
>>> I think we should do as Thomas proposes: plan to make it an error in
>>> 2.6 (or 2.7 if there's a big outcry, which I don't expect) and accept
>>> it with a warning in 2.5.

[Tim]
>> That's what I arrived at, although 2.4.3's checking behavior is
>> actually so inconsistent that "it" needs some defining (what exactly
>> are we trying to still accept?  e.g., that -1 doesn't trigger "I"
>> complaints but that -1L does above?  that one's surely a bug).

[Guido]
> No, it reflects that (up to 2.3 I believe) 0x was -1 but
> 0xL was 4294967295L.

But is it "a bug" _now_?  That's what I mean by "a bug".  To me, this
is simply inexplicable in 2.4 (and 2.5+):

>>> struct.pack("I", -1)  # neither std nor native modes complain
'\xff\xff\xff\xff'
>>> struct.pack(">I", -1)
'\xff\xff\xff\xff'

>>> struct.pack("I", -1L)  # but both complain if the input is long
Traceback (most recent call last):
 File "", line 1, in ?
OverflowError: can't convert negative value to unsigned long
>>> struct.pack(">I", -1L)
Traceback (most recent call last):
 File "", line 1, in ?
OverflowError: can't convert negative value to unsigned long

Particulary for the standard modes, the behavior also varies across
platforms (i.e., it wasn't true in any version of Python that
0x == -1 on most 64-bit boxes, and to have "standard mode"
behavior vary according to platform isn't particularly standard :-)).

>> To be clear, Thomas proposed "accepting it" (whatever that means) until 3.0.

> Fine with me.

So who has a definition for what "it" means?  I don't.  Does every
glitch have to be reproduced across the cross product of platform X
format-code X input-type X native-vs-standard X
direction-of-out-of-range?

Or would people be happier if struct simply never checked for
out-of-bounds?  At least the latter is doable with finite effort, and
is also explainable ("for an integral code of N bytes, pack() stores
the least-significant N bytes of the input viewed as a 2's-complement
integer").  I'd be happier with that than with something that can't be
explained short of exhaustive listing of cases across 5 dimensions.

Ditto with saying that for an integral type using N bytes, pack()
accepts any integer in -(2**(8*N-1)) through 2**(8*N)-1, complains
outside that range, and makes no complaining distinctions based on
platform, input type, standard-vs-native mode, signed-or-unsigned
format code, or direction of out-of-range.  That's also explainable.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Contributor agreements (was Re: DRAFT: python-dev summary for 2006-04-01 to 2006-04-15)

[Nick Coghlan]

Terry Reedy wrote:
>> .. _contributor agreement: 
>> http://www.python.org/psf/contrib-form-python.html
> 
> By itself, this link is inadequate since there is a blank for 'Initial 
> License' with no choices indicated.  So the summary needs a link to an 
> instruction page also.  The form is also a bit confusing since it appears 
> to talk about future contributions but the last line tacked on to the 
> bottom refers to the past.  If it means both, then why not say so in the 
> main text in a way that non-lawyers can understand?

That's the modified version of the form for people like me that have 
contributed stuff before sending the form in (I have one sitting on my desk 
that will be going in the mail real soon now, honest. . .).

There's another page somewhere in the PSF section that talks about contributor 
agreements in general - maybe that would be a better link for the summary?

Cheers,
Nick.

-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
 http://www.boredomandlaziness.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] test_gzip/test_tarfile failure om AMD64

[Thomas Wouters]

On 5/29/06, Tim Peters <[EMAIL PROTECTED]> wrote:
[Tim]>> To be clear, Thomas proposed "accepting it" (whatever that means) until 3.0.[Guido]> Fine with me.So who has a definition for what "it" means?
I know which 'it' I meant: the same 'it' as struct already accepts in Python 2.4 and before. Yes, it's inconsistent between formatcodes and valuetypes -- fixing that the whole point of the change -- but that's how you define 'compatibility'; struct, by default, should do what it did for Python 
2.4, for all operating modes. It doesn't have to be more liberal than 2.4 (and preferably shouldn't, as that could break backward compatibility of some code -- much less common, though.)Making a list of which formatcodes accept what values (for what valuetypes) for 
2.4 is easy enough (and should be added to the test suite, too ;-) -- I can do that in a few days if no one gets to it.-- Thomas Wouters <[EMAIL PROTECTED]>
Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove METH_OLDARGS?

[Thomas Wouters]

On 5/29/06, Fredrik Lundh <[EMAIL PROTECTED]> wrote:
this is a clear case of unnecessary meddling.  removing it won't removemuch code (a whopping 11 lines is dedicated to this), nor give any speedups whatsoever; all you're doing is generating extra work and support
issues for a bunch of third-party developers.  trust me, we have betterthings to do with our time.-1 on meddling with this before 3.0.-1 from me, too, for the same reason. It would be nice if the use of PyArg_Parse could generate a (C) compile-time warning, but since it can't, I think a runtime warning for this minor case is just overkill. (The C compile-time warning would be useful in other cases, too... hmm, perhaps we can do some post-processing of .o files, looking for deprecated symbols left undefined...)
-- Thomas Wouters <[EMAIL PROTECTED]>Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] test_gzip/test_tarfile failure om AMD64

[Fredrik Lundh]

Thomas Wouters wrote:

> I know which 'it' I meant: the same 'it' as struct already accepts in 
> Python 2.4 and before. Yes, it's inconsistent between formatcodes and 
> valuetypes -- fixing that the whole point of the change -- but that's 
> how you define 'compatibility'; struct, by default, should do what it 
> did for Python 2.4, for all operating modes.

that's how you define compatibility for bug fix releases in Python.  2.5 
is not 2.4.4.



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove METH_OLDARGS?

[Georg Brandl]

Thomas Wouters wrote:
> 
> On 5/29/06, *Fredrik Lundh* <[EMAIL PROTECTED]
> > wrote:
> 
> 
> this is a clear case of unnecessary meddling.  removing it won't remove
> much code (a whopping 11 lines is dedicated to this), nor give any speed
> ups whatsoever; all you're doing is generating extra work and support
> issues for a bunch of third-party developers.  trust me, we have better
> things to do with our time.
> 
> -1 on meddling with this before 3.0.
> 
> 
> -1 from me, too, for the same reason. It would be nice if the use of
> PyArg_Parse could generate a (C) compile-time warning, but since it
> can't, I think a runtime warning for this minor case is just overkill.
> (The C compile-time warning would be useful in other cases, too... hmm,
> perhaps we can do some post-processing of .o files, looking for
> deprecated symbols left undefined...)

Isn't there at least a GCC __attribute__((deprecated))?

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] test_gzip/test_tarfile failure om AMD64

[Thomas Wouters]

On 5/29/06, Fredrik Lundh <[EMAIL PROTECTED]> wrote:
Thomas Wouters wrote:> I know which 'it' I meant: the same 'it' as struct already accepts in> Python 2.4 and before. Yes, it's inconsistent between formatcodes and> valuetypes -- fixing that the whole point of the change -- but that's
> how you define 'compatibility'; struct, by default, should do what it> did for Python 2.4, for all operating modes.that's how you define compatibility for bug fix releases in Python.  2.5is not 
2.4.4.Correct, and it is also not 2.6.  Breaking perfectly working (and more to the point, non-complaining) code, even if it is for a good reason, is bad. It means, for instance, that I can not upgrade Python on any of the servers I manage, where Python gets used by clients. This is not a hypothetical problem, I've had it happen too often (although fortunately not often with Python, because of the sane policy on backward compatibility.)
If 2.5 warns and does the old thing, the upgrade path is easy and defendable. This is also why there are future statements -- I distinctly recall making the same argument back then :-) The cost of continuing the misfeatures in struct for one release does not weigh up to the cost of breaking compatibility unwarned.
-- Thomas Wouters <[EMAIL PROTECTED]>Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Let's stop eating exceptions in dict lookup

[Raymond Hettinger]

> On Mon, May 29, 2006 at 12:20:44PM -0700, Raymond Hettinger wrote:
>> > I've finally come around to writing a patch that stops dict lookup from
>> > eating all exceptions that occur during lookup, like rare bugs in user
>> > __eq__() methods.
>>
>> Is there a performance impact?
>
> I believe that this patch is good anyway, because I consider my (and
> anybody's) debugging hours worth more than a few seconds of a
> long-running process.  You get *really* obscure bugs this way.
>
> I would also point out that this is the kind of feature that should not
> be traded off for performance, otherwise we'd loose much of the point of
> Python.  IMHO.
>
> As it turns out, I measured only 0.5% performance loss in Pystone.

Please run some better benchmarks and do more extensive assessments on the 
performance impact.

The kind of obscure bug you're trying to kill does not affect 99.9% of Python 
users; however, loss of performance will affect everyone.  This is arguably the 
most actively exercised part of the Python and should not be changed without 
carefully weighing purity vs practicality.

FWIW, I applied a similar patch to set objects in Py2.5 so that they wouldn't 
eat exceptions.  So, I'm truly sympathetic to the cause.  However, dicts are 
much more critical.  There needs to be a careful judgement based on 
measurements 
and assessments of whether there are real benefits for everyday Python users.


Raymond








___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove METH_OLDARGS?

[Martin v. Löwis]

Georg Brandl wrote:
> Isn't there at least a GCC __attribute__((deprecated))?

Yes, but it applies to functions only. I guess you could make
a __deprecated__ function, and then expand METH_OLDARGS to a
call of that function. That won't help in cases where there are
no flags given at all in the method structures.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] ssize_t question: longs in header files

[Martin v. Löwis]

Neal Norwitz wrote:
> minus comments, etc  yields several questions about whether some
> values should use Py_ssize_t rather than C longs.  In particular:
> 
> * hash values
> Include/abstract.h: long PyObject_Hash(PyObject *o);  // also in
> object.h
> Include/object.h:typedef long (*hashfunc)(PyObject *);

I don't think these should be Py_ssize_t. Of course, a pointer type
cannot naturally convert to a long on some systems, so where we use
a pointer as a hash, we should do so carefully if sizeof(void*) >
sizeof(long); an xor of low and high word is probably good enough.

Currently, the code does something way more complex (actually
creating a long object); that should be replaced either by inlining
long_hash, or by using an even simpler hash function for 64-bit
pointers.

> * ints:  Include/intobject.h:long ob_ival;

As Tim says, this is way out of scope for 2.5. Guido said it is ok
to change this to 64-bit ints in 2.6, but I expect some embedded
system developers will start screaming when they hear that: 64-bit
arithmetic is expensive on a 32-bit machine.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove METH_OLDARGS?

[Georg Brandl]

Martin v. Löwis wrote:
> Georg Brandl wrote:
>> Isn't there at least a GCC __attribute__((deprecated))?
> 
> Yes, but it applies to functions only. I guess you could make
> a __deprecated__ function, and then expand METH_OLDARGS to a
> call of that function. That won't help in cases where there are
> no flags given at all in the method structures.

I thought more about PyArg_Parse for __deprecated__.

If you look at the patch on SF, it's creating warnings whenever
a new CFunction is created with a METH_OLDARGS flag. I'm not sure
this is the right way since it may mean that users of extensions
using METH_OLDARGS might get a warning each time they get a
bound method using it.

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Let's stop eating exceptions in dict lookup

[Armin Rigo]

Hi Raymond,

On Mon, May 29, 2006 at 02:02:25PM -0700, Raymond Hettinger wrote:
> Please run some better benchmarks and do more extensive assessments on the 
> performance impact.

At the moment, I'm trying to, but 2.5 HEAD keeps failing mysteriously on
the tests I try to time, and even going into an infinite loop consuming
all my memory - since the NFS sprint.  Am I allowed to be grumpy here,
and repeat that speed should not be used to justify bugs?  I'm proposing
a bug fix, I honestly don't care about 0.5% of speed.

With a benchmark that passed, and that heavily uses instance and class
attribute look-ups (richards), I don't even see any relevant difference.

> and assessments of whether there are real benefits for everyday Python
> users.

It would have saved me two hours-long debugging sessions, and I consider
myself an everyday Python user, so yes, I think so.


Grumpy-ly yours,

Armin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] ssize_t question: longs in header files

[Thomas Wouters]

On 5/29/06, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
Neal Norwitz wrote:> minus comments, etc  yields several questions about whether some> values should use Py_ssize_t rather than C longs.  In particular:> * ints:  Include/intobject.h:long ob_ival;
As Tim says, this is way out of scope for 2.5. Guido said it is okto change this to 64-bit ints in 2.6, but I expect some embeddedsystem developers will start screaming when they hear that: 64-bitarithmetic is expensive on a 32-bit machine.
Well, those systems shouldn't have a 64-bit Py_ssize_t anyway, should they?-- Thomas Wouters <[EMAIL PROTECTED]>Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] ssize_t: ints in header files

[Martin v. Löwis]

Neal Norwitz wrote:
> Should the following values be ints (limited to 2G)?
> 
>  * dict counts (ma_fill, ma_used, ma_mask)

I think Tim said he'll look into them.

>  * line #s and column #s

I think we should really formalize a limit, and then enforce it
throughout. column numbers shouldn't exceed 16-bits, and line #s
shouldn't exceed 31 bits.

>  * AST (asdl.h) sequences

we should first limit all the others, and then it should come
out naturally that AST sequences can be happily limited to 31 bits.

>  * recursion limit

This should be Py_ssize_t. While the stack is typically more limited,
it should be possible to configure it to exceed 4GiB on a 64-bit
machine.

>  * read/write/send/recv results and buf size

This is very tricky. Often, the underlying C library has int there
(e.g. msvcrt). Eventually, we should get rid of msvcrt, and then
hope that the underlying system API can deal with larger buffers.

>  * code object values like # args, # locals, stacksize, # instructions

IMO, they should all get formally limited to 15 bits (i.e. short).
I think some are already thus limited through the byte code format.
Somebody would have to check, and document what the limits are.

Either we end up with different limits for each one, or, more likely,
the same limit, in which case I would suggest to introduce another
symbolic type (e.g. Py_codesize_t or some such). Then we should
consistently reject source code that exceeds such a limit, and
elsewhere rely on the guarantee that the values will be always
limited much more than the underlying data structures.

>  * sre (i think i have a patch to fix this somewhere)

This is a huge set of changes, I think. SRE *should* support strings
larger than 4GiB. I could happily accept limitations for the regexes
themselves (e.g. size of the compiled expression, number of repeats,
etc).

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Contributor agreements (was Re: DRAFT: python-dev summary for 2006-04-01 to 2006-04-15)

[Steven Bethard]

On 5/29/06, Nick Coghlan <[EMAIL PROTECTED]> wrote:
> Terry Reedy wrote:
> >> .. _contributor agreement:
> >> http://www.python.org/psf/contrib-form-python.html
> >
> > By itself, this link is inadequate since there is a blank for 'Initial
> > License' with no choices indicated.  So the summary needs a link to an
> > instruction page also.  The form is also a bit confusing since it appears
> > to talk about future contributions but the last line tacked on to the
> > bottom refers to the past.  If it means both, then why not say so in the
> > main text in a way that non-lawyers can understand?
>
> That's the modified version of the form for people like me that have
> contributed stuff before sending the form in (I have one sitting on my desk
> that will be going in the mail real soon now, honest. . .).
>
> There's another page somewhere in the PSF section that talks about contributor
> agreements in general - maybe that would be a better link for the summary?

Sounds like it.  (That wasn't anywhere in the thread.)  Does
http://www.python.org/psf/contrib/ seem like the right page to link
to?

Thanks,

STeVe
-- 
Grammar am for people who can't think for myself.
--- Bucky Katt, Get Fuzzy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] ssize_t question: longs in header files

[Martin v. Löwis]

Thomas Wouters wrote:
> Neal Norwitz wrote:
> > minus comments, etc  yields several questions about whether some
> > values should use Py_ssize_t rather than C longs.  In particular:
> 
> > * ints:  Include/intobject.h:long ob_ival;
> 
> As Tim says, this is way out of scope for 2.5. Guido said it is ok
> to change this to 64-bit ints in 2.6, but I expect some embedded
> system developers will start screaming when they hear that: 64-bit
> arithmetic is expensive on a 32-bit machine. 
> 
> 
> Well, those systems shouldn't have a 64-bit Py_ssize_t anyway, should they?

No - but Guido said he wanted Python int to have a 64-bit representation
everywhere, not a Py_ssize_t representation.

I agree using Py_ssize_t would be a "smaller" change, and one that
likely has less performance impact. It would still be a large change,
and should be only done if we know for sure we don't want it to be
a 64-bit type always the next day.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove METH_OLDARGS?

[Martin v. Löwis]

Georg Brandl wrote:
> I thought more about PyArg_Parse for __deprecated__.

Ah, PyArg_Parse can, of course. Having it actually do that should not
hurt, either - but you need a reliable check whether the compiler
supports __deprecated__.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove METH_OLDARGS?

[Martin v. Löwis]

Guido van Rossum wrote:
>> I think that should be done for 2.5, but nothing else. Or, perhaps
>> a deprecation warning could be generated if it is still used.
> 
> I'll let Martin decide this one.

I just sent a message to some that producing a DeprecationWarning is
fine, but now I read some opposition; given that this really just needs
very little code to support it, and given that run-time warnings annoy
the users, not the module authors, I agree that this should stay
silently until 3.0.

We should still continue to remove all uses we have ourselves.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Contributor agreements (was Re: DRAFT: python-dev summary for 2006-04-01 to 2006-04-15)

[Martin v. Löwis]

Steven Bethard wrote:
>> There's another page somewhere in the PSF section that talks about 
>> contributor
>> agreements in general - maybe that would be a better link for the summary?
> 
> Sounds like it.  (That wasn't anywhere in the thread.)  Does
> http://www.python.org/psf/contrib/ seem like the right page to link
> to?

That is the page giving instructions on how to fill it out, yes.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] ssize_t question: longs in header files

[Thomas Wouters]

On 5/29/06, "Martin v. Löwis" <
[EMAIL PROTECTED]> wrote:
I agree using Py_ssize_t would be a "smaller" change, and one thatlikely has less performance impact. It would still be a large change,and should be only done if we know for sure we don't want it to be

a 64-bit type always the next day.Well, implementing PyInt in terms of a new symbolic type, even if it defaults to 64-bit, should make choosing it to be a 32-bit type a lot easier, shouldn't it?
Not that I think defaulting PyInt to use 'long long' is a good thing, considering we still only compile 'long long' support optionally. Better to stick with a common native type until all machines have 64-bit registers.
Better for performance, too.But switching PyInts to use (a symbolic type of the same size as) Py_ssize_t means that, when the time comes that 32-bit architectures are rare, Win64 isn't left as the only platform (barring other LLP64 systems) that has slow 33-to-64-bit Python numbers (because they'd be PyLongs there, even though the platform has 64-bit registers.) Given the timeframe and the impact, though, perhaps we should just do it -- now -- in the p3yk branch and forget about 
2.x; gives people all the more reason to switch, two years from now.-- Thomas Wouters <
[EMAIL PROTECTED]>Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Let's stop eating exceptions in dict lookup

[Fredrik Lundh]

Armin Rigo wrote:

> At the moment, I'm trying to, but 2.5 HEAD keeps failing mysteriously on
> the tests I try to time, and even going into an infinite loop consuming
> all my memory - since the NFS sprint.  Am I allowed to be grumpy here,
> and repeat that speed should not be used to justify bugs?

not unless you can produce some code.  unfounded accusations don't 
belong on this list (it's not like the sprinters didn't test the code on 
a whole bunch of platforms), and neither does lousy benchmarks (why are 
you repeating the 0.5% figure when pystone doesn't even test non-string 
dictionary behaviour?  PyString_Eq cannot fail...)



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Let's stop eating exceptions in dict lookup

[Armin Rigo]

Re-hi,

On Mon, May 29, 2006 at 11:34:28PM +0200, Armin Rigo wrote:
> At the moment, I'm trying to, but 2.5 HEAD keeps failing mysteriously on
> the tests I try to time, and even going into an infinite loop consuming
> all my memory

Ah, it's a corner case of str.find() whose behavior just changed.
Previously, 'abc'.find('', 100) would return -1, and now it returns 100.
Just to confuse matters, the same test with unicode returns 100, and has
always done so in the past.  (Oh well, one of these again...)

So, we need to decide which behavior is "right".  One could argue that
the current 2.5 HEAD now has a consistent behavior, so it could be kept;
but there is an opposite argument as well, which is that some existing
programs like the one I was testing are now thrown into annoying
infinite loops because str.find() never returns -1 any more, even with
larger and larger start arguments.  I believe that it's a pattern that
could be common in string-processing scripts, trying to match substrings
at various points in a string trusting that str.find() will eventually
return -1.  It's harder to think of a case where a program previously
relied on unicode.find('',n) never returning -1.  Also, introducing a
new way for programs to be caught in an infinite loop is probably not a
good idea.

Hum, my apologies for my grumpy comments about the NFS sprint.  At
least, the unification of the string and unicode algorithm that was
started there is a good move, also because it exposes pre-existing
inconsistencies.


A bientot,

Armin.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integer representation (Was: ssize_t question: longs in header files)

[Martin v. Löwis]

Thomas Wouters wrote:
> But switching PyInts to use (a symbolic type of the same size as)
> Py_ssize_t means that, when the time comes that 32-bit architectures are
> rare, Win64 isn't left as the only platform (barring other LLP64
> systems) that has slow 33-to-64-bit Python numbers (because they'd be
> PyLongs there, even though the platform has 64-bit registers.) Given the
> timeframe and the impact, though, perhaps we should just do it -- now --
> in the p3yk branch and forget about 2.x; gives people all the more
> reason to switch, two years from now.

I thought Py3k will have a single integer type whose representation
varies depending on the value being represented.

I haven't seen an actual proposal for such a type, so let me make
one:

struct PyInt{
  struct PyObject ob;
  Py_ssize_t value_or_size;
  char is_long;
  digit ob_digit[1];
};

If is_long is false, then value_or_size is the value (represented
as Py_ssize_t), else the value is in ob_digit, and value_or_size
is the size.

PyLong_* will be synonyms for PyInt_*. PyInt_FromLong/AsLong will
continue to exist; PyInt_AsLong will indicate an overflow with -1.
Likewise, PyArg_ParseTuple "i" will continue to produce int, and
raise an exception (OverflowError?) when the value is out of range.

C code can then decide whether to parse a Python integer as
C int, long, long long, or ssize_t.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove METH_OLDARGS?

[Georg Brandl]

Martin v. Löwis wrote:
> Guido van Rossum wrote:
>>> I think that should be done for 2.5, but nothing else. Or, perhaps
>>> a deprecation warning could be generated if it is still used.
>> 
>> I'll let Martin decide this one.
> 
> I just sent a message to some that producing a DeprecationWarning is
> fine, but now I read some opposition; given that this really just needs
> very little code to support it, and given that run-time warnings annoy
> the users, not the module authors, I agree that this should stay
> silently until 3.0.
> 
> We should still continue to remove all uses we have ourselves.

The only modules METH_OLDARGS is now used in are fl, sv, gl, cl
and Tkinter (for the latter there is my patch on SF for review).

sv and cl are listed in PEP 356 as possibly being removed, I don't know
whether the other ones are used anymore.

Georg

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Let's stop eating exceptions in dict lookup

[Fredrik Lundh]

Armin Rigo wrote:

> Ah, it's a corner case of str.find() whose behavior just changed.
> Previously, 'abc'.find('', 100) would return -1, and now it returns 100.
> Just to confuse matters, the same test with unicode returns 100, and has
> always done so in the past.  (Oh well, one of these again...)
> 
> So, we need to decide which behavior is "right".  One could argue that
> the current 2.5 HEAD now has a consistent behavior, so it could be kept;
> but there is an opposite argument as well, which is that some existing
> programs like the one I was testing are now thrown into annoying
> infinite loops because str.find() never returns -1 any more, even with
> larger and larger start arguments.  I believe that it's a pattern that
> could be common in string-processing scripts, trying to match substrings
> at various points in a string trusting that str.find() will eventually
> return -1.

well, the empty string is a valid substring of all possible strings 
(there are no "null" strings in Python).  you get the same behaviour 
from slicing, the "in" operator, "replace" (this was discussed on the 
list last week), "count", etc.

if you're actually searching for a *non-empty* string, find() will 
always return -1 sooner or later.

 > Hum, my apologies for my grumpy comments about the NFS sprint.  At
 > least, the unification of the string and unicode algorithm that was
 > started there is a good move, also because it exposes pre-existing
 > inconsistencies.

here's my current favourite:

 >>> "abc".count("", 100)
-96



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Let's stop eating exceptions in dict lookup

[Armin Rigo]

Hi Fredrik,

On Tue, May 30, 2006 at 12:01:46AM +0200, Fredrik Lundh wrote:
> not unless you can produce some code.  unfounded accusations don't 
> belong on this list (it's not like the sprinters didn't test the code on 
> a whole bunch of platforms), and neither does lousy benchmarks (why are 
> you repeating the 0.5% figure when pystone doesn't even test non-string 
> dictionary behaviour?  PyString_Eq cannot fail...)

Sorry, I do apologize for my wording.  I must admit that I was a bit
apalled by the amount of reference leaks that Michael had to fix after
the sprint, so jumped to conclusions a bit too fast when I saw by 1GB
laptop swap after less than a minute.  See my e-mail, which crossed
yours, for the explanation.

The reason I did not quote examples involving non-string dicts is that
my patch makes the non-string case simpler, so -- as I expected, and as
I have now measured -- marginally faster.  All in all it's hard to say
if there is a global consistent performance change.  At this point I'd
rather like to spend my time more interestingly; this might be by
defending my point of view that very minor performance hits should not
get in the way of fixes that avoid very obscure bugs, even only
occasionally-occuring but still very obscure bugs.


A bientot,

Armin.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Let's stop eating exceptions in dict lookup

[Raymond Hettinger]

Armin Rigo wrote:

  Hi Raymond,

On Mon, May 29, 2006 at 02:02:25PM -0700, Raymond Hettinger wrote:
  
  
Please run some better benchmarks and do more extensive assessments on the 
performance impact.

  
  
At the moment, I'm trying to, but 2.5 HEAD keeps failing mysteriously on
the tests I try to time, and even going into an infinite loop consuming
all my memory - since the NFS sprint.  Am I allowed to be grumpy here,
and repeat that speed should not be used to justify bugs?  I'm proposing
a bug fix, I honestly don't care about 0.5% of speed.
  

If it is really 0.5%, then we're fine.  Just remember that PyStone is
an amazingly uninformative and crappy benchmark.

The "justify bugs" terminology is pejorative and inaccurate.  It is
clear that the current dict behavior was a concious design decision and
documented as such.  Perhaps the decision sucked and should be changed,
but it is not a bug.



   and I consider
myself an everyday Python user,
  


Something may have been lost in translation.  Using it every day is not
the same as being an everyday user ;-)  There is no doubt that you
routinely stress the language in ways ways that are not at all
commonplace.

All I'm asking is that there be a well thought-out assessment of
whether the original design decision was struck the appropriate balance
between practicality 

Raymond



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Let's stop eating exceptions in dict lookup

[Armin Rigo]

Hi Fredrik,

On Tue, May 30, 2006 at 12:23:04AM +0200, Fredrik Lundh wrote:
> well, the empty string is a valid substring of all possible strings 
> (there are no "null" strings in Python).  you get the same behaviour 
> from slicing, the "in" operator, "replace" (this was discussed on the 
> list last week), "count", etc.
> 
> if you're actually searching for a *non-empty* string, find() will 
> always return -1 sooner or later.

I know this.  These corner cases are debatable and different answers
could be seen as correct, as I think is the case for find().  My point
was different: I was worrying that the recent change in str.find() would
needlessly send existing and working programs into infinite loops, which
can be a particularly bad kind of failure for some applications.

At least, it gave a 100% performance loss on the benchmark I was trying
to run :-)


A bientot,

Armin.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] test_gzip/test_tarfile failure om AMD64

[Bob Ippolito]

On May 29, 2006, at 1:18 PM, Bob Ippolito wrote:

>
> On May 29, 2006, at 12:44 PM, Guido van Rossum wrote:
>
>> On 5/29/06, Tim Peters <[EMAIL PROTECTED]> wrote:
 I think we should do as Thomas proposes: plan to make it an  
 error in
 2.6 (or 2.7 if there's a big outcry, which I don't expect) and
 accept
 it with a warning in 2.5.
>>>
>>> That's what I arrived at, although 2.4.3's checking behavior is
>>> actually so inconsistent that "it" needs some defining (what exactly
>>> are we trying to still accept?  e.g., that -1 doesn't trigger "I"
>>> complaints but that -1L does above?  that one's surely a bug).
>>
>> No, it reflects that (up to 2.3 I believe) 0x was -1 but
>> 0xL was 4294967295L.
>
> Python 2.3 did a FutureWarning on 0x but its value was -1.
>
> Anyway, my plan is to make it such that all non-native format codes
> will behave exactly like C casting, but will do a DeprecationWarning
> for input numbers that were initially out of bounds. This behavior
> will be consistent across (python) int and long, and will be easy
> enough to explain in the docs (but still more complicated than
> "values not representable by this data type will raise struct.error").
>
> This means that I'm also changing it so that struct.pack will not
> raise OverflowError for some longs, it will always raise struct.error
> or do a warning (as long as the input is int or long).
>
> Pseudocode looks kinda like this:
>
> def wrap_unsigned(x, CTYPE):
>   if not (0 <= x <= CTYPE_MAX):
>   DeprecationWarning()
>   x &= CTYPE_MAX
>   return x
>
> Actually, should this be a FutureWarning or a DeprecationWarning?

OK, this behavior is implemented in revision 46537:

(this is from ./python.exe -Wall)

 >>> import struct
 >>> struct.pack('B', 256)
Traceback (most recent call last):
   File "", line 1, in 
   File "/Users/bob/src/python/Lib/struct.py", line 63, in pack
 return o.pack(*args)
struct.error: ubyte format requires 0 <= number <= 255
 >>> struct.pack('B', -1)
Traceback (most recent call last):
   File "", line 1, in 
   File "/Users/bob/src/python/Lib/struct.py", line 63, in pack
 return o.pack(*args)
struct.error: ubyte format requires 0 <= number <= 255
 >>> struct.pack('>> struct.pack('>> struct.pack('>> struct.pack('http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] test_gzip/test_tarfile failure om AMD64

[Delaney, Timothy (Tim)]

Thomas Wouters wrote:

> If 2.5 warns and does the old thing, the upgrade path is easy and
> defendable. This is also why there are future statements -- I
> distinctly recall making the same argument back then :-) The cost of
> continuing the misfeatures in struct for one release does not weigh
> up to the cost of breaking compatibility unwarned.

This really sounds to me like a __future__ import would be useful to get
the fixed behaviour.

Tim Delaney
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Let's stop eating exceptions in dict lookup

[Scott Dial]

Raymond Hettinger wrote:
> If it is really 0.5%, then we're fine.  Just remember that PyStone is an 
> amazingly uninformative and crappy benchmark.

Since Armin seems to not like having to justify his patch with any 
performance testing, I wrote a handful of dict insertion exercises and 
could find no real performance impact. In the case of an exception, it 
was much faster, but that is because it is not inserting anything into 
the dictionary.

IOW if it is a bad change in behavior. Previously, the exception was 
swallowed and it was assumed to be a new key despite the exception. This 
is an obscure use case that could creep up it's ugly head.

class K(int):
def __cmp__(self, o): raise Exception()

d = {}
for i in xrange(10):
 d[K()] = i
for k in d.keys():
 print d[k]

Despite the incomparability, this throws no error in previous versions 
and the dict is still usable for the expected purpose.

-- 
Scott Dial
[EMAIL PROTECTED]
[EMAIL PROTECTED]
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] test_gzip/test_tarfile failure om AMD64

[Bob Ippolito]

On May 29, 2006, at 4:35 PM, Delaney, Timothy (Tim) wrote:

> Thomas Wouters wrote:
>
>> If 2.5 warns and does the old thing, the upgrade path is easy and
>> defendable. This is also why there are future statements -- I
>> distinctly recall making the same argument back then :-) The cost of
>> continuing the misfeatures in struct for one release does not weigh
>> up to the cost of breaking compatibility unwarned.
>
> This really sounds to me like a __future__ import would be useful  
> to get
> the fixed behaviour.

That would be potentially useful but is definitely not implementable.  
__future__ statements only really affect the compiler, and the struct  
module doesn't have anything to do with the compiler.

The only alternative is to have an entirely separate API (or another  
module altogether) and then try to convince people to use that.  
Warnings on incorrect input is a pretty good compromise.

The pre-2.5 behavior is totally confusing and totally undefined by  
the docs. The trunk behavior is (should be) completely consistent and  
backwards compatible. It actually allows even more broken code to  
work than the 2.4.x struct module, since it provides consistent  
behavior across 32-bit and 64-bit platforms and does the same thing  
whether you give it long or int. The only real downside is that it  
rather harmlessly tells you when some code is taking advantage of  
struct misfeatures.

-bob

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Let's stop eating exceptions in dict lookup

[Greg Ewing]

Fredrik Lundh wrote:

> well, the empty string is a valid substring of all possible strings 
> (there are no "null" strings in Python).  you get the same behaviour 
> from slicing, the "in" operator, "replace" (this was discussed on the 
> list last week), "count", etc.

Although Tim pointed out that replace() only regards
n+1 empty strings as existing in a string of lenth
n. So for consistency, find() should only find them
in those places, too.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] test_gzip/test_tarfile failure om AMD64

[Tim Peters]

[Bob Ippolito]
>> ...
>> Actually, should this be a FutureWarning or a DeprecationWarning?

Since it was never documented, UndocumentedBugGoingAwayError ;-)
Short of that, yes, DeprecationWarning.  FutureWarning is for changes
in non-exceptional behavior (.e.g, if we swapped the meanings of "<"
and ">" in struct format codes, that would rate a FutureWarning
subclass, line InsaneFutureWarning).

> OK, this behavior is implemented in revision 46537:
>
> (this is from ./python.exe -Wall)
>
>  >>> import struct

...

>  >>> struct.pack(' /Users/bob/src/python/Lib/struct.py:63: DeprecationWarning: struct
> integer wrapping is deprecated
>return o.pack(*args)
> /Users/bob/src/python/Lib/struct.py:63: DeprecationWarning: 'B'
> format requires 0 <= number <= 255
>return o.pack(*args)
> '\xff'

We certainly don't want to see two deprecation warnings for a single
deprecated behavior.  I suggest eliminating the "struct integer
wrapping" warning, mostly because I had no idea what it _meant_
before reading the comments in _struct.c  ("wrapping" is used most
often in a proxy or delegation context in Python these days).   "'B'
format requires 0 <= number <= 255" is perfectly clear all by itself.

> ...
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Let's stop eating exceptions in dict lookup

[Guido van Rossum]

On 5/29/06, Greg Ewing <[EMAIL PROTECTED]> wrote:
> Fredrik Lundh wrote:
>
> > well, the empty string is a valid substring of all possible strings
> > (there are no "null" strings in Python).  you get the same behaviour
> > from slicing, the "in" operator, "replace" (this was discussed on the
> > list last week), "count", etc.
>
> Although Tim pointed out that replace() only regards
> n+1 empty strings as existing in a string of lenth
> n. So for consistency, find() should only find them
> in those places, too.

And "abc".count("") should return 4.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integer representation (Was: ssize_t question: longs in header files)

[Guido van Rossum]

[Adding the py3k list; please remove python-dev in followups.]

On 5/29/06, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
> I thought Py3k will have a single integer type whose representation
> varies depending on the value being represented.

That's one proposal. Another is to have an abstract 'int' type with
two concrete subtypes, e.g. 'short' and 'long', corresponding to
today's int and long. At the C level the API should be unified so C
programmers are isolated from the difference (they aren't today).

> I haven't seen an actual proposal for such a type,

I'm not sure that my proposal above has ever been said out loud. I'm
also not partial; I think we may have to do an experiment to decide.

> so let me make one:
>
> struct PyInt{
>   struct PyObject ob;
>   Py_ssize_t value_or_size;
>   char is_long;
>   digit ob_digit[1];
> };
>
> If is_long is false, then value_or_size is the value (represented
> as Py_ssize_t), else the value is in ob_digit, and value_or_size
> is the size.

Nice. I guess if we store the long value in big-endian order we could
drop is_long, since the first digit of the long would always be
nonzero. This would save a byte (on average) for the longs, but it
would do nothing for the wasted space for short ints.

> PyLong_* will be synonyms for PyInt_*.

Why do we need to keep the PyLong_* APIs at all? Even at the Python
level we're not planning any backward compatibility features; at the C
level I like even more freedom to break things.

> PyInt_FromLong/AsLong will
> continue to exist; PyInt_AsLong will indicate an overflow with -1.
> Likewise, PyArg_ParseTuple "i" will continue to produce int, and
> raise an exception (OverflowError?) when the value is out of range.
>
> C code can then decide whether to parse a Python integer as
> C int, long, long long, or ssize_t.

Nice. I like the unified API and I like using Py_ssize_t instead of
long for the value; this ensures that an int can hold a pointer (if we
allow for signed pointers) and matches the native word size better on
Windows (I guess it makes no difference for any other platform, where
ssize_t and long already have the same size).

I worry about all the wasted space for alignment caused by the extra
flag byte though. That would be 4 byte per integer on 32-bit machines
(where they are currently 12 bytes) and 8 bytes on 64-bit machines
(where they are currently 24 bytes).

That's why I'd like my alternative proposal (int as ABC and two
subclasses that may remain anonymous to the Python user); it'll save
the alignment waste for short ints and will let us use a smaller int
type for the size for long ints (if we care about the latter).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove METH_OLDARGS?

[Neal Norwitz]

On 5/29/06, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
> Georg Brandl wrote:
> > I thought more about PyArg_Parse for __deprecated__.
>
> Ah, PyArg_Parse can, of course. Having it actually do that should not
> hurt, either - but you need a reliable check whether the compiler
> supports __deprecated__.

Already done for gcc, see Py_DEPRECATED in pyport.h.  Would be nice if
someone could add support on Windows.

n
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Let's stop eating exceptions in dict lookup

[Tim Peters]

[Greg Ewing]
>> Although Tim pointed out that replace() only regards
>> n+1 empty strings as existing in a string of lenth
>> n. So for consistency, find() should only find them
>> in those places, too.

[Guido]
> And "abc".count("") should return 4.

And it does, but too much context was missing in Greg's reply to make
sense of his comment.  In context Greg was seeming to say that

"abc".count("", 100)

should return 0, and

"abc".find("", 100)

should return -1, since "the only" empty substrings of "abc" are at
slices 0:0, 1:1, 2:2 and 3:3.  In fact we have

>>> "abc".count("", 100)
1
>>> u"abc".count("", 100)
1
>>> "abc".find("", 100)
100
>>> u"abc".find("", 100)
100

today, although the idea that find() can return an index that doesn't
exist in the string is particularly jarring.  Since we also have:

>>> "abc".rfind("", 100)
3
>>> u"abc".rfind("", 100)
3

it's twice as jarring that "searching from the right" finds the target
at a _smaller_ index than "searching from the left".
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove METH_OLDARGS?

[Neal Norwitz]

On 5/29/06, Thomas Wouters <[EMAIL PROTECTED]> wrote:
> On 5/29/06, Fredrik Lundh <[EMAIL PROTECTED]> wrote:
> >
> > this is a clear case of unnecessary meddling.  removing it won't remove
> > much code (a whopping 11 lines is dedicated to this), nor give any speed
> > ups whatsoever; all you're doing is generating extra work and support
> > issues for a bunch of third-party developers.  trust me, we have better
> > things to do with our time.
> >
> > -1 on meddling with this before 3.0.
>
> -1 from me, too, for the same reason. It would be nice if the use of
> PyArg_Parse could generate a (C) compile-time warning, but since it can't, I
> think a runtime warning for this minor case is just overkill. (The C
> compile-time warning would be useful in other cases, too... hmm, perhaps we
> can do some post-processing of .o files, looking for deprecated symbols left
> undefined...)

How can users find the implicit use of METH_OLDARGS in code like this:

 static struct PyMethodDef gestalt_methods[] = {
   {"gestalt", gestalt_gestalt},
   {NULL, NULL} /* Sentinel */
 };

 static PyMethodDef SwiMethods[]= {
 { "swi", swi_swi,0},
  { NULL, NULL}
};

The first one was taken straight from one of Georg's checkins, the
second one was modified.
How many people knew we were still using METH_OLDARGS in these places?
 I know this can be done with some simple scripts, but that won't
catch all cases.  And many people won't run such a script, much less
write one.

OTOH, perhaps a deprecation warning on PyArgs_Parse() is sufficient?
What about that?  It doesn't address other cases where OLDARGS are
used without PyArgs_Parse though.

n
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Weekly Python Patch/Bug Summary

[Kurt B. Kaiser]

Patch / Bug Summary
___

Patches :  375 open ( -3) /  3264 closed (+26) /  3639 total (+23)
Bugs:  910 open ( +3) /  5851 closed (+20) /  6761 total (+23)
RFE :  217 open ( -1) /   220 closed ( +3) /   437 total ( +2)

New / Reopened Patches
__

Minor Correction to urllib2 HOWTO  (2006-05-20)
CLOSED http://python.org/sf/1492147  opened by  Mike Foord

None missing from keyword module  (2006-05-20)
CLOSED http://python.org/sf/1492218  opened by  Žiga Seilnacht

Socket-object convenience function: getpeercred().  (2006-05-20)
   http://python.org/sf/1492240  opened by  Heiko Wundram

urllib2 HOWTO - Further (minor) Corrections  (2006-05-20)
CLOSED http://python.org/sf/1492255  opened by  Mike Foord

Windows CE support (part 1)  (2006-05-21)
CLOSED http://python.org/sf/1492356  opened by  Luke Dunstan

Windows CE support (part 2)  (2006-05-27)
   http://python.org/sf/1495999  opened by  Luke Dunstan

Unification of list-comp and for syntax  (2006-05-21)
   http://python.org/sf/1492509  opened by  Heiko Wundram

distinct error type from shutil.move()  (2006-05-22)
   http://python.org/sf/1492704  opened by  Zooko O'Whielacronx

Improvements to ceval.c  (2006-05-22)
   http://python.org/sf/1492828  opened by  mrjbq7

Speed up gzip.readline (~40%)  (2005-09-04)
CLOSED http://python.org/sf/1281707  reopened by  marumari

Speed up gzip.readline (~40%)  (2005-09-04)
CLOSED http://python.org/sf/1281707  reopened by  marumari

Allow build without tracing  (2006-05-22)
CLOSED http://python.org/sf/1493102  opened by  Steve Holden

Performance enhancements for struct module  (2006-05-23)
CLOSED http://python.org/sf/1493701  opened by  Bob Ippolito

Documentation for new Struct object  (2006-05-24)
   http://python.org/sf/1494140  opened by  Bob Ippolito

PyUnicode_Resize cannot resize shared unicode object  (2006-05-25)
CLOSED http://python.org/sf/1494487  opened by  Hirokazu Yamamoto

Numeric characters not recognized.  (2006-05-24)
CLOSED http://python.org/sf/1494554  opened by  Anders Chrigström

BaseWidget.destroy updates master's childern too early  (2006-05-24)
   http://python.org/sf/1494750  opened by  Greg Couch

Scalable zipfile extension  (2003-09-27)
CLOSED http://python.org/sf/813436  reopened by  jafo

Remove types.InstanceType and new.instance  (2006-05-26)
CLOSED http://python.org/sf/1495675  opened by  Collin Winter

Fix test_exceptions.py  (2006-05-27)
   http://python.org/sf/1496135  opened by  Collin Winter

urllib2 HTTPPasswordMgr: default ports  (2006-05-28)
CLOSED http://python.org/sf/1496206  opened by  John J Lee

Convert Tkinter to METH_VARARGS style  (2006-05-29)
   http://python.org/sf/1496952  opened by  Georg Brandl

deprecate METH_OLDARGS  (2006-05-29)
CLOSED http://python.org/sf/1496957  opened by  Georg Brandl

urllib2: ensure digest auth happens in preference to basic  (2006-05-29)
CLOSED http://python.org/sf/1497027  opened by  John J Lee

Let dicts propagate the exceptions in user __eq__  (2006-05-29)
   http://python.org/sf/1497053  opened by  Armin Rigo

Patches Closed
__

Minor Correction to urllib2 HOWTO  (2006-05-21)
   http://python.org/sf/1492147  closed by  quiver

None missing from keyword module  (2006-05-20)
   http://python.org/sf/1492218  closed by  gbrandl

urllib2 HOWTO - Further (minor) Corrections  (2006-05-21)
   http://python.org/sf/1492255  closed by  quiver

Windows CE support (part 1)  (2006-05-21)
   http://python.org/sf/1492356  closed by  loewis

PC new-logo-based icon set  (2006-05-17)
   http://python.org/sf/1490384  closed by  loewis

Cleaned up 16x16px icons for windows.  (2006-05-03)
   http://python.org/sf/1481304  closed by  loewis

property to get the docstring from fget  (2004-08-08)
   http://python.org/sf/1005461  closed by  gbrandl

Speed up gzip.readline (~40%)  (2005-09-04)
   http://python.org/sf/1281707  closed by  etrepum

Speed up gzip.readline (~40%)  (2005-09-04)
   http://python.org/sf/1281707  closed by  etrepum

Speed up gzip.readline (~40%)  (2005-09-04)
   http://python.org/sf/1281707  closed by  etrepum

scary frame speed hacks  (2004-01-13)
   http://python.org/sf/876206  closed by  tim_one

Allow build without tracing  (2006-05-22)
   http://python.org/sf/1493102  closed by  gbrandl

MacOSX: distutils support for -arch and -isysroot flags  (2006-05-13)
   http://python.org/sf/1488098  closed by  ronaldoussoren

Performance enhancements for struct module  (2006-05-23)
   http://python.org/sf/1493701  closed by  etrepum

Fix for int(string, base) wrong answers (take 2)  (2005-10-24)
   http://python.org/sf/1335972  closed by  tim_one

remove 4 ints from PyFrameObject  (2005-10-25)
   http://python.org/sf/1337051  closed by  tim_one

PyUnicode_Resize cannot resize shared unicode object  (2006-05-24)
   http://python.org/sf/1494487  closed by  doerwalter

Numeric characters n

Re: [Python-Dev] Let's stop eating exceptions in dict lookup

[Tim Peters]

[Armin Rigo]
> ...
> ...
> Am I allowed to be grumpy here, and repeat that speed should not be
> used to justify bugs?

As a matter of fact, you are.  OTOH, nobody at the sprint made that
argument, so nobody actually feels shame on that count :-)

I apologize for the insufficiently reviewed exception-rework patch
anyway.  The massive try/raise/except slowdown in 2.5a2 (compared to
2.4.3) was depressing and surprising, and figuring it out & repairing
it consumed various peoples' time across the whole week.  By the time
Saturday came around, I was fried enough that "well, people have
hacked on this all week, the slowdown finally got replaced by a
speedup, and all the tests pass -- enough already, let's go home!"
ruled.  So, as everyone suspected all along, it's Neal's fault for not
running his refleak test more often ;-)

> ...
> It would have saved me two hours-long debugging sessions, and I consider
> myself an everyday Python user, so yes, I think so.

I'll try to review your patch carefully Tuesday, but I'm willing to
accept a slowdown for this too (although from a quick look I doubt
there's a measurable one, and your approach is wholly sane).
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] feature request: inspect.isgenerator

[Neal Norwitz]

On 5/29/06, Georg Brandl <[EMAIL PROTECTED]> wrote:
> Fredrik Lundh wrote:
> >
> > is there some natural and obvious connection between generators and that
> > number that I'm missing, or is that constant perhaps best hidden inside
> > some introspection support module?
>
> It seems to be a combination of CO_NOFREE, CO_GENERATOR, CO_OPTIMIZED and
> CO_NEWLOCALS.
>
> The first four CO_ constants are already in inspect.py, the newer ones
> (like CO_GENERATOR) aren't.

All these constants are declared in compiler.consts.  Some are also
defined in __future__.  It would be better to have them all in a
single place.

> I wonder whether a check shouldn't just return (co_flags & 0x20), which
> is CO_GENERATOR.

Makes more sense.

n
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integer representation (Was: ssize_t question: longs in header files)

[Martin v. Löwis]

Guido van Rossum wrote:
>> struct PyInt{
>>   struct PyObject ob;
>>   Py_ssize_t value_or_size;
>>   char is_long;
>>   digit ob_digit[1];
>> };
>>
> 
> Nice. I guess if we store the long value in big-endian order we could
> drop is_long, since the first digit of the long would always be
> nonzero. This would save a byte (on average) for the longs, but it
> would do nothing for the wasted space for short ints.

Right; alternatively, the top-most bit of ob_digit[0] could also be
used, as longs have currently 15-bit digits.

> Why do we need to keep the PyLong_* APIs at all? Even at the Python
> level we're not planning any backward compatibility features; at the C
> level I like even more freedom to break things.

Indeed, they should get dropped.

> I worry about all the wasted space for alignment caused by the extra
> flag byte though. That would be 4 byte per integer on 32-bit machines
> (where they are currently 12 bytes) and 8 bytes on 64-bit machines
> (where they are currently 24 bytes).

I think ints should get managed by PyMalloc in Py3k. With my proposal,
an int has 16 bytes on a 32-bit machine, so there wouldn't be any
wastage for PyMalloc (which allocates 16 bytes for 12-byte objects,
anyway). On a 64-bit machine, it would indeed waste 8 bytes per
int.

> That's why I'd like my alternative proposal (int as ABC and two
> subclasses that may remain anonymous to the Python user); it'll save
> the alignment waste for short ints and will let us use a smaller int
> type for the size for long ints (if we care about the latter).

I doubt they can remain anonymous. People often dispatch by type
(e.g. pickle, xmlrpclib, ...), and need to put the type into a
dictionary. If the type is anonymous, they will do

   dispatch[type(0)] = marshal_int
   dispatch[type(sys.maxint+1)] = marshal_int

Plus, their current code as

   dispatch[int] = marshal_int

which will silently break (although it won't be silent if they also
have dispatch[long] = marshal_long).

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove METH_OLDARGS?

[Martin v. Löwis]

Neal Norwitz wrote:
> How can users find the implicit use of METH_OLDARGS in code like this:
> 
>  static struct PyMethodDef gestalt_methods[] = {
>{"gestalt", gestalt_gestalt},
>{NULL, NULL} /* Sentinel */
>  };
> 
>  static PyMethodDef SwiMethods[]= {
>  { "swi", swi_swi,0},
>   { NULL, NULL}
> };

They can't know they do. Of course, if they do, they likely also use
PyArg_Parse to process the arguments.

> OTOH, perhaps a deprecation warning on PyArgs_Parse() is sufficient?
> What about that?  It doesn't address other cases where OLDARGS are
> used without PyArgs_Parse though.

What other cases remain? People might have complex argument processing
procedure not involving PyArg_Parse, these would just break with a
runtime error in Py3k. If the module is maintained, it should be easy
to fix it. If the module is unmaintained, producing a warning now
might not help, either.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Let's stop eating exceptions in dict lookup

[Fredrik Lundh]

Guido van Rossum wrote:

>>> well, the empty string is a valid substring of all possible strings
>>> (there are no "null" strings in Python).  you get the same behaviour
>>> from slicing, the "in" operator, "replace" (this was discussed on the
>>> list last week), "count", etc.
 >
>> Although Tim pointed out that replace() only regards
>> n+1 empty strings as existing in a string of lenth
>> n. So for consistency, find() should only find them
>> in those places, too.

depends on how you interpret the reference to "slices" in the docs. 
"abc"[100:] is an empty string, and so is "abc"[100:100].

> And "abc".count("") should return 4.

it does, and has always done (afaik).  "abc".count("", 100) did use to 
return -96, though, which is hard to explain in terms of anything else.



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Let's stop eating exceptions in dict lookup

[Fredrik Lundh]

Armin Rigo wrote:

> I know this.  These corner cases are debatable and different answers
> could be seen as correct, as I think is the case for find().  My point
> was different: I was worrying that the recent change in str.find() would
> needlessly send existing and working programs into infinite loops, which
> can be a particularly bad kind of failure for some applications.

since "abc".find("", 0) == 0, I would have thought that a program that 
searched for an empty string in a loop wouldn't get anywhere at all.



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove METH_OLDARGS?

[Neal Norwitz]

On 5/29/06, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
>
> > OTOH, perhaps a deprecation warning on PyArgs_Parse() is sufficient?
> > What about that?  It doesn't address other cases where OLDARGS are
> > used without PyArgs_Parse though.
>
> What other cases remain? People might have complex argument processing
> procedure not involving PyArg_Parse, these would just break with a
> runtime error in Py3k. If the module is maintained, it should be easy
> to fix it. If the module is unmaintained, producing a warning now
> might not help, either.

One case would be if the function doesn't take any args, but the args
aren't checked, just ignored.  Another case I was thinking about was
PyArg_NoArgs which is a macro, but that just calls PyArg_Parse, so it
would get flagged.  I can't imagine there would be enough of these
cases to matter.

I'd be satisfied with a deprecation warning for PyArg_Parse, though we
(*) should really figure out how to make it work on Windows.  I
haven't seen anyone object to the C compiler deprecation warning.

n
--
* Where we means someone with a Windows box of course. :-)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] fixing buildbots

[Neal Norwitz]

I've been starting to get some of the buildbots working again.  There
was some massive problem on May 25 where a ton of extra files were
left around.  I can't remember if I saw something about that at the
NFS sprint or not.

There is a lingering problem that I can't fix on all the boxes.  Namely:

  cp Modules/Setup{.dist,}

Should we always do that step before we build on the buildbots?  The
warning about Setup.dist being newer than Setup is displayed.  I can
understand why we wouldn't want to unconditionally overwrite a user's
modified Setup.  However, for the buildbots, it seems safer to always
overwrite it.

Any objections?  Any ideas for the best way to implement this?  A
separate BB step for Unix clients?   In the makefile?

Martin, I would have fixed it on your Solaris box, but I don't think I
can get access to the buildbot's account.

n
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com