Re: [Python-Dev] pprint(iterator)
Paul Moore wrote: 2009/1/30 Walter Dörwald : Paul Moore wrote: [...] In all honesty, I think pkgutil.simplegeneric should be documented, exposed, and moved to a library of its own[1]. http://pypi.python.org/pypi/simplegeneric Thanks, I was aware of that. I wasn't aware of the fact that simplegeneric is part of the stdlib, albeit in a strange spot. I assume that the barrier to getting this into the stdlib will be higher than to simply exposing an implementation already available in the stdlib. At least we'd need documentation and tests. And of course the code must be stable and there must be someone willing to maintain it (then again it's less than 40 lines of code). There should be enough third-party module that use it to justify making simplegeneric an official part of the stdlib. The best spot for generic() is probably in the functools module. To be honest, all I would like is for these regular "let's have another special method" discussions to become unnecessary... Me too. Servus, Walter ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Partial function application 'from the right'
Ludvig Ericson wrote:
> Well, I was trying to be funny and was under the impression that Python
> 3.0 had Unicode identifiers, but apparently it doesn't. (I used …, not ...)
It does, but they may not contain characters of the category
"Punctuation, other":
>>> import unicodedata
>>> unicodedata.category("…")
'Po'
- Hagen
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Should there be a source-code checksum in module objects?
As I've mentioned, I've been re-examining from ground up the whole state of affairs in writing a debugger. One of the challenges of a debugger or any source-code analysis tool is verifying that the source-code that the tool is reporting on corresponds to the compiled object under execution. (For debuggers, this problem becomes more likely to occur when you are debugging on a computer that isn't the same as the computer where the code is running.) Is there a checksum of the source text computed and stored in compilation? If not, should there be one? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] C API for appending to arrays
The array.array type is an excellent type for storing a large amount of
"native" elements, such as integers, chars, doubles, etc., without
involving the heavy machinery of numpy. It's both blazingly fast and
reasonably efficient with memory. The one thing missing from the array
module is the ability to directly access array values from C.
This might seem superfluous, as it's perfectly possible to manipulate
array contents from Python/C using PyObject_CallMethod and friends. The
problem is that it requires the native values to be marshalled to Python
objects, only to be immediately converted back to native values by the
array code. This can be a problem when, for example, a numeric array
needs to be filled with contents, such as in this hypothetical example:
/* error checking and refcounting subtleties omitted for brevity */
PyObject *load_data(Source *src)
{
PyObject *array_type = get_array_type();
PyObject *array = PyObject_CallFunction(array_type, "c", 'd');
PyObject *append = PyObect_GetAttrString(array, "append");
while (!source_done(src)) {
double num = source_next(src);
PyObject *f = PyFloat_FromDouble(num);
PyObject *ret = PyObject_CallFunctionObjArgs(append, f, NULL);
if (!ret)
return NULL;
Py_DECREF(ret);
Py_DECREF(f);
}
Py_DECREF(array_type);
return array;
}
The inner loop must convert each C double to a Python Float, only for
the array to immediately extract the double back from the Float and
store it into the underlying array of C doubles. This may seem like a
nitpick, but it turns out that more than half of the time of this
function is spent creating and deleting those short-lived floating-point
objects.
Float creation is already well-optimized, so opportunities for speedup
lie elsewhere. The array object exposes a writable buffer, which can be
used to store values directly. For test purposes I created a faster
"append" specialized for doubles, defined like this:
int array_append(PyObject *array, PyObject *appendfun, double val)
{
PyObject *ret;
double *buf;
Py_ssize_t bufsize;
static PyObject *zero;
if (!zero)
zero = PyFloat_FromDouble(0);
// append dummy zero value, created only once
ret = PyObject_CallFunctionObjArgs(appendfun, zero, NULL);
if (!ret)
return -1;
Py_DECREF(ret);
// append the element directly at the end of the C buffer
PyObject_AsWriteBuffer(array, (void **) &buf, &bufsize));
buf[bufsize / sizeof(double) - 1] = val;
return 0;
}
This hack actually speeds up array creation by a significant percentage
(30-40% in my case, and that's for code that was producing the values by
parsing a large text file).
It turns out that an even faster method of creating an array is by using
the fromstring() method. fromstring() requires an actual string, not a
buffer, so in C++ I created an std::vector with a contiguous
array of doubles, passed that array to PyString_FromStringAndSize, and
called array.fromstring with the resulting string. Despite all the
unnecessary copying, the result was much faster than either of the
previous versions.
Would it be possible for the array module to define a C interface for
the most frequent operations on array objects, such as appending an
item, and getting/setting an item? Failing that, could we at least make
fromstring() accept an arbitrary read buffer, not just an actual string?
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pprint(iterator)
Paul Moore wrote: > 2009/2/2 Nick Coghlan : >> A trio of patches that: >> 1. promoted simplegeneric from pkgutil to functools (with appropriate >> documentation and tests) >> 2. changed pkgutil to use functools.simplegeneric instead of its current >> internal version >> 3. updated pprint to be a generic function (and hence more easily >> extensible via the ABC mechanism, while still keeping it's current >> special cases as necessary) >> >> would certainly be an interesting thing to see (with patch 3 being the >> poster child for why patch 1 is a good idea). > > I'll see what I can do. I can't promise much for (3) as I don't > personally have a need for it, so my understanding of the use cases is > limited at best. But (1) and (2) should be possible. > > Can I assign the patches to you (for review, at least)? Sure - I've had plenty to do with functools in the past, and will no doubt have plenty to do with it in the future. Given that I believe Guido was one of the folks whose brain was hurt by PJE's attempts to explain PEP 3124, I'll be bringing the discussion back here before committing anything though :) (and don't worry too much about 3 - it will give me an opportunity to road test the functools patch by using it to refactor pprint and check the performance implications) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] C API for appending to arrays
[Hrvoje Niksic] The one thing missing from the array module is the ability to directly access array values from C. Please put a feature request on the bug tracker. Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Should there be a source-code checksum in module objects?
On Mon, Feb 2, 2009 at 00:52, Rocky Bernstein wrote: > As I've mentioned, I've been re-examining from ground up the whole > state of affairs in writing a debugger. > > One of the challenges of a debugger or any source-code analysis tool > is verifying that the source-code that the tool is reporting on > corresponds to the compiled object under execution. (For debuggers, > this problem becomes more likely to occur when you are debugging on a > computer that isn't the same as the computer where the code is > running.) > > Is there a checksum of the source text computed and stored in > compilation? No, only the bytecode version used, the mtime of the source the bytecode is derived from, and the bytecode itself. > If not, should there be one? If you are planning to read this directly from the .pyc file then it is not needed as the mtime timestamp in the .pyc should in general be enough to detect changes to the source. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pprint(iterator)
Walter Dörwald wrote: > Paul Moore wrote: >> 2009/1/30 Walter Dörwald : >>> Paul Moore wrote: >>> [...] In all honesty, I think pkgutil.simplegeneric should be documented, exposed, and moved to a library of its own[1]. >>> http://pypi.python.org/pypi/simplegeneric >> >> Thanks, I was aware of that. > > I wasn't aware of the fact that simplegeneric is part of the stdlib, > albeit in a strange spot. Officially, it isn't - it's an implementation detail of pkgutil. As such, a leading underscore wouldn't have hurt, but the lack of mention in the pkgutil documentation (or pkgutil.__all__) is hopefully enough of a hint that it really isn't intended as a general purpose programming tool. >> I assume that the barrier to getting this >> into the stdlib will be higher than to simply exposing an >> implementation already available in the stdlib. > > At least we'd need documentation and tests. And of course the code must > be stable and there must be someone willing to maintain it (then again > it's less than 40 lines of code). > > There should be enough third-party module that use it to justify making > simplegeneric an official part of the stdlib. > > The best spot for generic() is probably in the functools module. >> To be honest, all I >> would like is for these regular "let's have another special method" >> discussions to become unnecessary... > > Me too. A trio of patches that: 1. promoted simplegeneric from pkgutil to functools (with appropriate documentation and tests) 2. changed pkgutil to use functools.simplegeneric instead of its current internal version 3. updated pprint to be a generic function (and hence more easily extensible via the ABC mechanism, while still keeping it's current special cases as necessary) would certainly be an interesting thing to see (with patch 3 being the poster child for why patch 1 is a good idea). The major virtue of such a basic generic framework is that it is much easier to explain than the all-singing all-dancing overloading system described in PEP 3124. Type-based dispatch on the first argument is fairly straightforward to describe in terms that make sense to anyone that is already familiar with dynamic dispatch of class and instance methods. It's only when you get into more exotic dispatch conditions and up-calls and the like that people's eyes start to glaze over. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 3.0.1/3.1.0 summary
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Jan 31, 2009, at 2:43 PM, Martin v. Löwis wrote: How about Friday February 13? Fine with me (although next Friday (Feb 6) would work slightly better) Feb 6 won't work for me. Would the 20th be better for you Martin? Barry -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSYdDbnEjvBPtnXfVAQKfGwQAjow7pXouQ+e+qAOzgvYm7x5atqMeLbUI iQJ2o83Gdci8jtJZQgd3jRccE2qqST9yJrOkYwA20M1KamktoqSQJJQ6fQcKMSSa nP1ZCrnNmKrt4NjeKUdB/g626mEpowmk6X5pgrITVUL1g35h+n+bqlgDrJ5HzaqQ rPiBtt2WCh4= =Svnl -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pprint(iterator)
2009/2/2 Nick Coghlan : > Paul Moore wrote: >> 2009/2/2 Nick Coghlan : >>> A trio of patches that: >>> 1. promoted simplegeneric from pkgutil to functools (with appropriate >>> documentation and tests) >>> 2. changed pkgutil to use functools.simplegeneric instead of its current >>> internal version >>> 3. updated pprint to be a generic function (and hence more easily >>> extensible via the ABC mechanism, while still keeping it's current >>> special cases as necessary) >>> >>> would certainly be an interesting thing to see (with patch 3 being the >>> poster child for why patch 1 is a good idea). >> >> I'll see what I can do. I can't promise much for (3) as I don't >> personally have a need for it, so my understanding of the use cases is >> limited at best. But (1) and (2) should be possible. >> >> Can I assign the patches to you (for review, at least)? > > Sure - I've had plenty to do with functools in the past, and will no > doubt have plenty to do with it in the future. http://bugs.python.org/issue5135 A single patch covering (1) and (2). Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pprint(iterator)
On 29 Jan 2009, at 21:54, Nick Coghlan wrote:
For the "reiterable" cases like dictionary views (where the object is
not consumed), an appropriate __str__ or __repr__ should be written).
Indeed, instead of having a __pprint__ why not just allow a __repr__
to reformat its output?
dict having:
def __repr__(self, pretty=False):
if pretty:
return "{\n a: 1\n b: 2\n}"
else:
return "{a: 1, b: 2}"
That way you can specify your own pretty format, intending it to still
be a valid __repr__ and pprint can do:
try:
print(obj.__repr__(pretty=True))
except TypeError:
print(prettify(repr(obj)))
That way it's opt in, doesn't have a special method, and includes the
mental prompt "this should eval() to obj"
Matt
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pprint(iterator)
Tres Seaver wrote: > Nick Coghlan wrote: >> (and don't worry too much about 3 - it will give me an opportunity to >> road test the functools patch by using it to refactor pprint and check >> the performance implications) > > /me wonders about the performance-criticality of anything using > 'pprint'. Or were you just planning to use it as a means to benchmark > the 'simplegeneric' stuff? I would think something with a lot lower > intrinsic overhead would be a better benchmark target. Don't read too much into that comment - generics make the most sense in cases (like pprint) where extensibility is a more important feature than raw speed. I'm talking about more subjective impressions of performance as well as things like "if I register a handler for Sequence, is there any performance advantage in registering direct handlers for the builtin subclasses?". Really drastic performance degradations would be on the radar as well - slowing pprint() down by 10% is unlikely to bother anyone, but slowing it down by 100% would be a bad thing (not that I think such a degradation is likely, I'm just trying to give an impression of the magnitude of change I'll be trying to measure). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pprint(iterator)
On 2009-02-02 14:50, Nick Coghlan wrote: Really drastic performance degradations would be on the radar as well - slowing pprint() down by 10% is unlikely to bother anyone, but slowing it down by 100% would be a bad thing (not that I think such a degradation is likely, I'm just trying to give an impression of the magnitude of change I'll be trying to measure). Using the pretty module I referenced earlier, which basically uses simplegeneric's lookup algorithm: In [11]: %timeit x=pprint.pformat(sys.modules) 10 loops, best of 3: 27.5 ms per loop In [12]: %timeit x=pretty.pretty(sys.modules) 10 loops, best of 3: 39.9 ms per loop In [13]: len(sys.modules) Out[13]: 517 The comparison is somewhat dodgy, though. pretty is not so much a refactoring of pprint as a reimplementation, so the time differences may not be due to the dispatch. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3.0, tp_compare not used for == test?
On Mon, Feb 2, 2009 at 11:38 AM, Nick Coghlan wrote: > That's what the -3 command line switch is for - it sets a boolean flag > that C code can check to see if it should emit warnings for things that > are going to break in Python 3.x. Understood. My worry was that we'd get a lot of false positives--- i.e., warnings about pieces of Python 2.7 that don't need to change. But I guess that's okay. > There's already at least one example > of its use in typeobject.c (relating to inheritance of the tp_hash > slot). There are a few other examples about the place (e.g. I believe > that when -3 is defined the return value of dict.keys and friends is > changed to a list subclass that warns on certain operations that have > different semantics in 3.x). Thanks for the pointers. I'll take a look and see if I can come up with a patch. But I'd like to complete the rest of the cmp removal stuff first. Mark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 3.0.1/3.1.0 summary
>> Fine with me (although next Friday (Feb 6) would work slightly better) > > Feb 6 won't work for me. Would the 20th be better for you Martin? No, they are both busy days - Feb 13 is then slightly better than Feb 20. I have about an hour in the morning (around 8:00 UTC), and then after 15:00 UTC - so I should be done in my evening, if that's good enough. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 3.0.1/3.1.0 summary
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Feb 2, 2009, at 4:48 PM, Martin v. Löwis wrote: Fine with me (although next Friday (Feb 6) would work slightly better) Feb 6 won't work for me. Would the 20th be better for you Martin? No, they are both busy days - Feb 13 is then slightly better than Feb 20. I have about an hour in the morning (around 8:00 UTC), and then after 15:00 UTC - so I should be done in my evening, if that's good enough. Yep, that will work. B -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSYdqo3EjvBPtnXfVAQLe9AQAh5e7P+7tO0Ibd8f/2Yvj7xfoYoQk0Z5n V/nMpN+UbSI55w5ZvPrjL1a0zFknTaqBxeSTbhdbIeNXVAUEc8oWauajMvJjCH/r WXZjAze9WW6BqW3UJCmMmmMtRc6biKyv++avhuu/ursiGdaV9JMYk/q0R970ny2V s+FF4gZKGpo= =+MjC -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3.0, tp_compare not used for == test?
> I'm wondering if Mark should add the exception he recently removed back > in as a Deprecation Warning when tp_compare is defined, but > tp_richcompare is not. Such a warning should also be present when > running with -3 in 2.7 (assuming it isn't already there). > > Otherwise we're going to get more questions like Campbell's wondering if > it is intentional that tp_compare is no longer getting called in the 3.x > series (the warnings would both make it explicit and give guidance on > what to do about it). Mark removed the cmpfunc typedef; that should cause compilation in many extension modules (except those which don't actually cast to cmpfunc, but get the type right in the first place). Also, if you then investigate why it doesn't get called, you'll notice quickly that the slot doesn't even have the right name anymore. So Campbell's surprise would be gone - he wouldn't have been able to still invoke tp_compare through the cmp() builtin. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3.0, tp_compare not used for == test?
> Understood. My worry was that we'd get a lot of false positives--- > i.e., warnings about pieces of Python 2.7 that don't need to change. > But I guess that's okay. I don't personally use the -3 flag, but I would expect that these would indeed be annoying. We could arrange to exempt them, by introducing a "Core" type flag, to be included in the default type flags when the core is compiled (and not even declared otherwise). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Removing tp_compare?
> Don't you rememeber the PC:s in the late 1980th? It was based on > Intel's 80286-processor, and Microsoft's C compiler supported > three or four different memory models, called things like "TINY", > "SMALL", "LARGE", and "HUGE". Most of these memory models had > different sized data and function pointers. I don't think this is factually correct: - tiny, small: code and data each 16 bit - large, huge: code and data each 32 bit - medium: code 32 bit, data 16 bit - compact: code 16 bit, data 32 bit So most of the memory models (66%) have same-sized data and function pointers, only few of them (33%) have them differently-sized. In any case, I don't think Python supports 8086 segmented mode in any reasonable way today. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3.0, tp_compare not used for == test?
Terry Reedy wrote: > Campbell Barton wrote: > In 3.0, the build-in cmp() should have been removed and will be for > 3.0.1. Do not use it. I believe the tp_compare slot should not be used > either. It will become reserved. It will not be removed only because > that would change the binary layout. I'm wondering if Mark should add the exception he recently removed back in as a Deprecation Warning when tp_compare is defined, but tp_richcompare is not. Such a warning should also be present when running with -3 in 2.7 (assuming it isn't already there). Otherwise we're going to get more questions like Campbell's wondering if it is intentional that tp_compare is no longer getting called in the 3.x series (the warnings would both make it explicit and give guidance on what to do about it). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Strange locale problem with Python 3
On 2009-02-01 19:44, Reto Schüttel wrote: > Hi > > While helping Brandon Rhodes to port PyEphem[1] to Python 3 we struggled > over a > strange locale-related problem on OS X. PyEphem is a library which can do > astronomical computations like tracking the position of stars, planets and > earth satellites relative to the earth's position. When testing out the > Python > 3 release of PyEphem I noticed that on my OS X laptop a lot of calculations > were wrong (not completely wrong, but clearly not accurate) compared to > Python > 2.5. We (well mostly Brandon) were able to track down the problem to the > TLE > parser (TLE are data file containing the orbital elements of an object) > which > appears to read most values wrong with python 3. In fact it cut of the > decimal > parts of all floats (1.123232 got 1, etc). Manually setting LANG and > LC_ALL to > C solved the problem. > > It turns out that some parts of Python 3 got more locale dependent on some > platforms. The only platform I am aware of is OS X, on Linux Python 3 > appears > to behave like Python 2.x did. This is probably due to the unconditional call to setlocale() in pythonrun.c: /* Set up the LC_CTYPE locale, so we can obtain the locale's charset without having to switch locales. */ setlocale(LC_CTYPE, ""); In Python 2, no such call is made and as a result the C lib defaults to the "C" locale. Calling setlocale() in an application is always dangerous due to the many side-effects this can have on the C lib parsing and formatting APIs. If this is done just to figure the environment's locale settings, then it's better to reset the locale to the one that was active before the setlocale() call. Python 2 uses this approach. Python 3 does not. > In case of PyEphem the problem was in the C extension which got more locale > dependent, for example atof() or scanf() with Python 3 now expected the > german > decimal-delimiter ',' instead of the '.' in floats (3.14 vs. 3,14). On the > other hand the constructor of float still expects '.' all the time. But the > built-in function strptime() honors locales with Python 3 and expects > german > week day. > > I've written a simple script and a simple C extension which illustrates the > problem. Both the extension and the script run python 2.x and python 3, > so you > can easily compare the result while executing the script in different > environments. > > I was only able to reproduce the problem on OS X (10.5) and using a german > locale like "de_CH.UTF-8". When manually setting LC_ALL=C, the differences > disappears. > > I can't imagine that his behavior was really intended, and I hope the > test case > helps you guys to identify/fix this problem. > > Download the test case from: > http://github.com/retoo/py3k-locale-problem/tarball/master > or get it using git: > git://github.com/retoo/py3k-locale-problem.git > > You can use the following steps to build it: > > $ python2.5 setup.py build > $ python3.0 setup.py build > > To run the tests with python 2.5, enter: > $ (cd build/lib*-2.5; python2.5 py3k_locale_problem.py) > ... for 3.0 ... > $ (cd build/lib*-3.0; python3.0 py3k_locale_problem.py) > > In the file 'results.txt' you can see the output from my OS X system. > > Cheers, > Reto Schüttel > > [1] http://rhodesmill.org/pyephem/ -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 02 2009) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try our new mxODBC.Connect Python Database Interface for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] mail libraries streamlined support for ssl and starttls
Sorry for being a pest here... still, I'd be grateful if someone could please try and take look at the patches I've posted on bugs.python.org for smtplib, imaplib and poplib, since I think they could be useful for a future 2.x release of the standard library, and I've verified they port almost as they are to python3. They are almost stupid, I know; still, smtp over ssl in the standard library is not working, and the starttls method is declared as being the preferred one since RFC 2595 time... """7. imaps and pop3s ports Separate "imaps" and "pop3s" ports were registered for use with SSL. Use of these ports is discouraged in favor of the STARTTLS or STLS commands. [...] """ Thank you all very much... Yours, lorenzo +-+--+ | Lorenzo M. Catucci | Centro di Calcolo e Documentazione | | catu...@ccd.uniroma2.it | Università degli Studi di Roma "Tor Vergata" | | | Via O. Raimondo 18 ** I-00173 ROMA ** ITALY | | Tel. +39 06 7259 2255 | Fax. +39 06 7259 2125| +-+--+___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Strange locale problem with Python 3
Sorry! Somehow my mailclient managed to mess up my message, here another copy of it, hopefully in better conditions than last time. Aahz: Good idea, I've created a bug report. Bug ID 5125 Hi While helping Brandon Rhodes to port PyEphem[1] to Python 3 we struggled over a strange locale-related problem on OS X. PyEphem is a library which can do astronomical computations like tracking the position of stars, planets and earth satellites relative to the earth's position. When testing out the Python 3 release of PyEphem I noticed that on my OS X laptop a lot of calculations were wrong (not completely wrong, but clearly not accurate) compared to Python 2.5. We (well mostly Brandon) were able to track down the problem to the TLE parser (TLE are data file containing the orbital elements of an object) which appears to read most values wrong with python 3. In fact it cut of the decimal parts of all floats (1.123232 got 1, etc). Manually setting LANG and LC_ALL to C solved the problem. It turns out that some parts of Python 3 got more locale dependent on some platforms. The only platform I am aware of is OS X, on Linux Python 3 appears to behave like Python 2.x did. In case of PyEphem the problem was in the C extension which got more locale dependent, for example atof() or scanf() with Python 3 now expected the german decimal-delimiter ',' instead of the '.' in floats (3.14 vs. 3,14). On the other hand the constructor of float still expects '.' all the time. But the built-in function strptime() honors locales with Python 3 and expects german week day. I've written a simple script and a simple C extension which illustrates the problem. Both the extension and the script run python 2.x and python 3, so you can easily compare the result while executing the script in different environments. I was only able to reproduce the problem on OS X (10.5) and using a german locale like "de_CH.UTF-8". When manually setting LC_ALL=C, the differences disappears. I can't imagine that his behavior was really intended, and I hope the test case helps you guys to identify/fix this problem. Download the test case from: http://github.com/retoo/py3k-locale-problem/tarball/master or get it using git: git://github.com/retoo/py3k-locale-problem.git You can use the following steps to build it: $ python2.5 setup.py build $ python3.0 setup.py build To run the tests with python 2.5, enter: $ (cd build/lib*-2.5; python2.5 py3k_locale_problem.py) ... for 3.0 ... $ (cd build/lib*-3.0; python3.0 py3k_locale_problem.py) In the file 'results.txt' you can see the output from my OS X system. Cheers, Reto Schuettel [1] http://rhodesmill.org/pyephem/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] mail libraries streamlined support for ssl and starttls [2nd try!]
Sorry for being a pest here... and for forgetting to put pointers to the issues in the previous attempt... the issues on bugs.python.org are: http://bugs.python.org/issue4470 - smtplib SMTP_SSL not working http://bugs.python.org/issue4471 - IMAP4 missing support for starttls http://bugs.python.org/issue4473 - POP3 missing support for starttls still, I'd be grateful if someone could please try and take look at the patches I've posted on bugs.python.org for smtplib, imaplib and poplib, since I think they could be useful for a future 2.x release of the standard library, and I've verified they port almost as they are to python3. They are almost stupid, I know; still, smtp over ssl in the standard library is not working, and the starttls method is declared as being the preferred one since RFC 2595 time... """7. imaps and pop3s ports Separate "imaps" and "pop3s" ports were registered for use with SSL. Use of these ports is discouraged in favor of the STARTTLS or STLS commands. [...] """ Thank you all very much... (once more!), yours, lorenzo +-+--+ | Lorenzo M. Catucci | Centro di Calcolo e Documentazione | | catu...@ccd.uniroma2.it | Università degli Studi di Roma "Tor Vergata" | | | Via O. Raimondo 18 ** I-00173 ROMA ** ITALY | | Tel. +39 06 7259 2255 | Fax. +39 06 7259 2125| +-+--+___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3.0, tp_compare not used for == test?
Mark Dickinson wrote: > On Mon, Feb 2, 2009 at 10:36 AM, Nick Coghlan wrote: >> Such a warning should also be present when >> running with -3 in 2.7 (assuming it isn't already there). > > I'm not sure how/whether that would work, given that there are > probably still plenty of 2.7 modules in the distribution that > (quite legitimately) define tp_compare but not tp_richcompare. That's what the -3 command line switch is for - it sets a boolean flag that C code can check to see if it should emit warnings for things that are going to break in Python 3.x. There's already at least one example of its use in typeobject.c (relating to inheritance of the tp_hash slot). There are a few other examples about the place (e.g. I believe that when -3 is defined the return value of dict.keys and friends is changed to a list subclass that warns on certain operations that have different semantics in 3.x). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3.0, tp_compare not used for == test?
On Mon, Feb 2, 2009 at 10:36 AM, Nick Coghlan wrote: > I'm wondering if Mark should add the exception he recently removed back > in as a Deprecation Warning when tp_compare is defined, but > tp_richcompare is not. This sounds reasonable to me. A third-party module that implements tp_compare but not tp_richcompare is almost certainly not going to be giving the behaviour that its author intended, after 3.0.1. Currently, the only warning that such an author gets is a possible compiler warning about incompatible pointer types (type of tp_compare versus type of tp_reserved), should he/she happen to recompile and be watching the compiler output closely. > Such a warning should also be present when > running with -3 in 2.7 (assuming it isn't already there). I'm not sure how/whether that would work, given that there are probably still plenty of 2.7 modules in the distribution that (quite legitimately) define tp_compare but not tp_richcompare. Mark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Financial aid for PyCon 2009 is now available
I'm happy to announce that the Python Software Foundation has allocated funds to help people attend PyCon 2009! If you would like to come to PyCon but can't afford it, the PSF may be able to help you pay for registration, lodging/hotel costs and transportation (flight etc.). Please see http://us.pycon.org/2009/registration/financial-aid/ for full details, or email pycon-...@python.org with questions. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Should there be a source-code checksum in module objects?
On Mon, Feb 2, 2009 at 10:48 AM, wrote: > Brett Cannon writes: > > On Mon, Feb 2, 2009 at 00:52, Rocky Bernstein wrote: > > > As I've mentioned, I've been re-examining from ground up the whole > > > state of affairs in writing a debugger. > > > > > > One of the challenges of a debugger or any source-code analysis tool > > > is verifying that the source-code that the tool is reporting on > > > corresponds to the compiled object under execution. (For debuggers, > > > this problem becomes more likely to occur when you are debugging on a > > > computer that isn't the same as the computer where the code is > > > running.) > > > > > > Is there a checksum of the source text computed and stored in > > > compilation? > > > > No, only the bytecode version used, the mtime of the source the > > bytecode is derived from, and the bytecode itself. > > > > > If not, should there be one? > > > > If you are planning to read this directly from the .pyc file then it > > is not needed as the mtime timestamp in the .pyc should in general be > > enough to detect changes to the source. > > I'm not sure I understand. I am debugging program P on computer A > which may have (probably did?) do a compile. I am on computer B which > I have the source code. Maybe it is checked out from a version control > system same as on A. Maybe it has bytecodes, maybe not. But the > expectation is that the programmer thinks it matches what is currently > on A that the programmer is debuggging. Can I tell for certain? > > Suppose I'm writing a code coverage tool which can accumulate > statistics over many runs. I notice that the date on the Python file > changes, again one way it might is that it might be checked out fresh > and subversion for example will put in a current timestamp. How can > the code coverage tool know that the source hasn't changed even though > the file may have disappeared or maybe was modified several times but > in the end stays the same? Unfortunately this use case (remote debugging) was not considered when the code object was designed and implemented. I don't know how important the use case is in general, but clearly it is important to you. I suggest that you move this discussion to python-ideas to ferret out a possible implementation and API; or to find out work-arounds. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pprint(iterator)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Nick Coghlan wrote: > Paul Moore wrote: >> 2009/2/2 Nick Coghlan : >>> A trio of patches that: >>> 1. promoted simplegeneric from pkgutil to functools (with appropriate >>> documentation and tests) >>> 2. changed pkgutil to use functools.simplegeneric instead of its current >>> internal version >>> 3. updated pprint to be a generic function (and hence more easily >>> extensible via the ABC mechanism, while still keeping it's current >>> special cases as necessary) >>> >>> would certainly be an interesting thing to see (with patch 3 being the >>> poster child for why patch 1 is a good idea). >> I'll see what I can do. I can't promise much for (3) as I don't >> personally have a need for it, so my understanding of the use cases is >> limited at best. But (1) and (2) should be possible. >> >> Can I assign the patches to you (for review, at least)? > > Sure - I've had plenty to do with functools in the past, and will no > doubt have plenty to do with it in the future. > > Given that I believe Guido was one of the folks whose brain was hurt by > PJE's attempts to explain PEP 3124, I'll be bringing the discussion back > here before committing anything though :) > > (and don't worry too much about 3 - it will give me an opportunity to > road test the functools patch by using it to refactor pprint and check > the performance implications) /me wonders about the performance-criticality of anything using 'pprint'. Or were you just planning to use it as a means to benchmark the 'simplegeneric' stuff? I would think something with a lot lower intrinsic overhead would be a better benchmark target. Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software "Excellence by Design"http://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJhu6I+gerLs4ltQ4RAvdcAJ4mFTcdfgG7ZVvdKXulIw+fWgLJxwCeOrTO 9rriK3+zyl4K63doUjbckF4= =++qD -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pprint(iterator)
2009/2/2 Nick Coghlan : > A trio of patches that: > 1. promoted simplegeneric from pkgutil to functools (with appropriate > documentation and tests) > 2. changed pkgutil to use functools.simplegeneric instead of its current > internal version > 3. updated pprint to be a generic function (and hence more easily > extensible via the ABC mechanism, while still keeping it's current > special cases as necessary) > > would certainly be an interesting thing to see (with patch 3 being the > poster child for why patch 1 is a good idea). I'll see what I can do. I can't promise much for (3) as I don't personally have a need for it, so my understanding of the use cases is limited at best. But (1) and (2) should be possible. Can I assign the patches to you (for review, at least)? > The major virtue of such a basic generic framework is that it is much > easier to explain than the all-singing all-dancing overloading system > described in PEP 3124. Type-based dispatch on the first argument is > fairly straightforward to describe in terms that make sense to anyone > that is already familiar with dynamic dispatch of class and instance > methods. It's only when you get into more exotic dispatch conditions and > up-calls and the like that people's eyes start to glaze over. Agreed. And given that the simple case probably covers 90% of the practical requirements, that's a shame. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Should there be a source-code checksum in module objects?
Brett Cannon writes: > On Mon, Feb 2, 2009 at 00:52, Rocky Bernstein wrote: > > As I've mentioned, I've been re-examining from ground up the whole > > state of affairs in writing a debugger. > > > > One of the challenges of a debugger or any source-code analysis tool > > is verifying that the source-code that the tool is reporting on > > corresponds to the compiled object under execution. (For debuggers, > > this problem becomes more likely to occur when you are debugging on a > > computer that isn't the same as the computer where the code is > > running.) > > > > Is there a checksum of the source text computed and stored in > > compilation? > > No, only the bytecode version used, the mtime of the source the > bytecode is derived from, and the bytecode itself. > > > If not, should there be one? > > If you are planning to read this directly from the .pyc file then it > is not needed as the mtime timestamp in the .pyc should in general be > enough to detect changes to the source. I'm not sure I understand. I am debugging program P on computer A which may have (probably did?) do a compile. I am on computer B which I have the source code. Maybe it is checked out from a version control system same as on A. Maybe it has bytecodes, maybe not. But the expectation is that the programmer thinks it matches what is currently on A that the programmer is debuggging. Can I tell for certain? Suppose I'm writing a code coverage tool which can accumulate statistics over many runs. I notice that the date on the Python file changes, again one way it might is that it might be checked out fresh and subversion for example will put in a current timestamp. How can the code coverage tool know that the source hasn't changed even though the file may have disappeared or maybe was modified several times but in the end stays the same? Thanks. > > -Brett > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Should there be a source-code checksum in module objects?
ro...@gnu.org wrote: But the expectation is that the programmer thinks it matches what is currently on A that the programmer is debuggging. Can I tell for certain? You can always find out by compiling the source and comparing the resulting bytecode with what is currently on A. Not that this is necessarily a *convenient* way... -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] [patch] Duplicate sections detection in ConfigParser
Greetings Pythonistas,
ConfigParser as you most certainly know is a .ini parser included in
the Python standard library. It's documentation mentions a
DuplicateSectionError but I was puzzled after hunting a bug in our
application that this error was not raised when parsing a file with
duplicate sections. After looking at the code, it turns out that
DuplicateSectionError is only raised when using the builder interface;
the parser interface will never throw it.
The attached patch is compatible with both the 2.x and the 3.x
branches; it adds a `unique_sects` parameter to the constructor of
RawConfigParser and a test in the parser loop that raises
DuplicateSectionError if a section is seen more then once and that
unique_sects is True.
This is just a proof of concept and I'd like your opinion on it before
working on the final version. I see two main issues regarding
backward compatibility and uniformity. For uniformity `unique_sects`
should also apply to the builder interface. However, if it does, it
should default to True since it was the default behavior for the
builder interface to raise on duplicate sections. On the other hand,
the default behavior for the parser interface was to ignore duplicate
sections so making `unique_sects` default to True might break existing
code.
To summarize, I ask you all:
* Is the lack of duplicates detection in the parser a problem worth
addressing?
* Should `unique_sects` control the builder interface?
* Should it default to True?
Best regards,
--
Yannick Gingras
http://ygingras.net
Index: Lib/ConfigParser.py
===
--- Lib/ConfigParser.py (revision 68546)
+++ Lib/ConfigParser.py (working copy)
@@ -215,13 +215,14 @@
class RawConfigParser:
-def __init__(self, defaults=None, dict_type=dict):
+def __init__(self, defaults=None, dict_type=dict, unique_sects=False):
self._dict = dict_type
self._sections = self._dict()
self._defaults = self._dict()
if defaults:
for key, value in defaults.items():
self._defaults[self.optionxform(key)] = value
+self._unique_sects = unique_sects
def defaults(self):
return self._defaults
@@ -468,7 +469,10 @@
if mo:
sectname = mo.group('header')
if sectname in self._sections:
-cursect = self._sections[sectname]
+if self._unique_sects:
+raise DuplicateSectionError(sectname)
+else:
+cursect = self._sections[sectname]
elif sectname == DEFAULTSECT:
cursect = self._defaults
else:
signature.asc
Description: This is a digitally signed message part.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
