Re: [Python-Dev] 2.3.5 schedule, and something I'd like to get in
On Jan 6, 2005, at 15:03, Bob Ippolito wrote: On Jan 6, 2005, at 14:59, Ronald Oussoren wrote: On 6-jan-05, at 14:04, Jack Jansen wrote: On 6 Jan 2005, at 00:49, Martin v. Löwis wrote: The "new" solution is basically to go back to the Unix way of building an extension: link it against nothing and sort things out at runtime. Not my personal preference, but at least we know that loading an extension into one Python won't bring in a fresh copy of a different interpreter or anything horrible like that. This sounds good, except that it only works on OS X 10.3, right? What about older versions? 10.3 or later. For older OSX releases (either because you build Python on 10.2 or earlier, or because you've set MACOSX_DEPLOYMENT_TARGET to a value of 10.2 or less) we use the old behaviour of linking with "-framework Python". Wouldn't it be better to link with the actual dylib inside the framework on 10.2? Otherwise you can no longer build 2.3 extensions after you've installed 2.4. It would certainly be better to do this for 10.2. This patch implements the proposed direct framework linking: http://python.org/sf/1097739 -bob ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Minor change to behaviour of csv module
>I'm considering a change to the csv module that could potentially break >some obscure uses of the module (but CSV files usually quote, rather >than escape, so the most common uses aren't effected). > >Currently, with a non-default escapechar='\\', input like: > >field one,field \ >two,field three > >Returns: > >["field one", "field \\\ntwo", "field three"] > >In the 2.5 series, I propose changing this to return: > >["field one", "field \ntwo", "field three"] > >Is this reasonable? Is the old behaviour desirable in any way (we could >add a switch to enable to new behaviour, but I feel that would only >allow the confusion to continue)? Thinking about this further, I suspect we have to retain the current behaviour, as broken as it is, as the default: it's conceivable that someone somewhere is post-processing the result to remove the backslashes, and if we fix the csv module, we'll break their code. Note that PEP-305 had nothing to say about escaping, nor does the module reference manual. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.3.5 schedule, and something I'd like to get in
On 7 Jan 2005, at 11:08, Bob Ippolito wrote: 10.3 or later. For older OSX releases (either because you build Python on 10.2 or earlier, or because you've set MACOSX_DEPLOYMENT_TARGET to a value of 10.2 or less) we use the old behaviour of linking with "-framework Python". Wouldn't it be better to link with the actual dylib inside the framework on 10.2? Otherwise you can no longer build 2.3 extensions after you've installed 2.4. It would certainly be better to do this for 10.2. This patch implements the proposed direct framework linking: http://python.org/sf/1097739 Looks good, I'll incorporate it. And as I haven't heard of any showstoppers for the -undefined dynamic_lookup (and Anthony seems to be offline this week) I'll put that in too. -- Jack Jansen, <[EMAIL PROTECTED]>, http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Re: [Csv] Minor change to behaviour of csv module
Andrew McNamara <[EMAIL PROTECTED]>: > [snip] > Currently, with a non-default escapechar='\\', input like: > > field one,field \ > two,field three > > Returns: > > ["field one", "field \\\ntwo", "field three"] > > In the 2.5 series, I propose changing this to return: > > ["field one", "field \ntwo", "field three"] IMO this is the *only* reasonable behaviour. I don't understand why the escape character should be left in; this is one of the reason why UNIX-style colon-separated values don't work with the current module. If one wanted the first version, one would (I presume) write field one,field \\\ two,field three -- Magnus Lie Hetland Fallen flower I see / Returning to its branch http://hetland.org Ah! a butterfly. [Arakida Moritake] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
RE: [Python-Dev] Re: [Csv] Minor change to behaviour of csv module
Andrew explains that in the CSV module, escape characters are not properly removed. Magnus writes: > IMO this is the *only* reasonable behaviour. I don't understand why > the escape character should be left in; this is one of the reason why > UNIX-style colon-separated values don't work with the current module. Andrew writes back later: > Thinking about this further, I suspect we have to retain the current > behaviour, as broken as it is, as the default: it's conceivable that > someone somewhere is post-processing the result to remove the backslashes, > and if we fix the csv module, we'll break their code. I'm with Magnus on this. No one has 4 year old code using the CSV module. The existing behavior is just simply WRONG. Sure, of course we should try to maintain backward compatibility, but surely SOME cases don't require it, right? Can't we treat this misbehavior as an outright bug? -- Michael Chermside ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: [Csv] Minor change to behaviour of csv module
On 2005 Jan 07, at 14:45, Michael Chermside wrote: Andrew explains that in the CSV module, escape characters are not properly removed. Magnus writes: IMO this is the *only* reasonable behaviour. I don't understand why the escape character should be left in; this is one of the reason why UNIX-style colon-separated values don't work with the current module. Andrew writes back later: Thinking about this further, I suspect we have to retain the current behaviour, as broken as it is, as the default: it's conceivable that someone somewhere is post-processing the result to remove the backslashes, and if we fix the csv module, we'll break their code. I'm with Magnus on this. No one has 4 year old code using the CSV module. The existing behavior is just simply WRONG. Sure, of course we should try to maintain backward compatibility, but surely SOME cases don't require it, right? Can't we treat this misbehavior as an outright bug? +1 -- the nonremoval of escape characters smells like a bug to me, too. Alex ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an idea for improving struct.unpack api
Bob Ippolito <[EMAIL PROTECTED]> writes: > On Jan 6, 2005, at 8:17, Michael Hudson wrote: > >> Ilya Sandler <[EMAIL PROTECTED]> writes: >> >>> A problem: >>> >>> The current struct.unpack api works well for unpacking C-structures >>> where >>> everything is usually unpacked at once, but it >>> becomes inconvenient when unpacking binary files where things >>> often have to be unpacked field by field. Then one has to keep track >>> of offsets, slice the strings,call struct.calcsize(), etc... >> >> IMO (and E), struct.unpack is the primitive atop which something more >> sensible is built. I've certainly tried to build that more sensible >> thing at least once, but haven't ever got the point of believing what >> I had would be applicable to the general case... maybe it's time to >> write such a thing for the standard library. > > This is my ctypes-like attempt at a high-level interface for struct. > It works well for me in macholib: > http://svn.red-bean.com/bob/py2app/trunk/src/macholib/ptypes.py Unsurprisingly, that's fairly similar to mine :) Cheers, mwh -- If trees could scream, would we be so cavalier about cutting them down? We might, if they screamed all the time, for no good reason. -- Jack Handey ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an idea for improving struct.unpack api
> Bob Ippolito <[EMAIL PROTECTED]> writes: >> This is my ctypes-like attempt at a high-level interface for struct. >> It works well for me in macholib: >> http://svn.red-bean.com/bob/py2app/trunk/src/macholib/ptypes.py Michael Hudson <[EMAIL PROTECTED]> writes: > > Unsurprisingly, that's fairly similar to mine :) So, why don't you both use the original? ctypes works on the mac, too ;-) Thomas ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an idea for improving struct.unpack api
On Jan 7, 2005, at 9:33 AM, Thomas Heller wrote: Bob Ippolito <[EMAIL PROTECTED]> writes: This is my ctypes-like attempt at a high-level interface for struct. It works well for me in macholib: http://svn.red-bean.com/bob/py2app/trunk/src/macholib/ptypes.py Michael Hudson <[EMAIL PROTECTED]> writes: Unsurprisingly, that's fairly similar to mine :) So, why don't you both use the original? ctypes works on the mac, too ;-) I did use the original for the prototype of macholib! Then I wrote a version in pure python to eliminate the compiler dependency and ended up adding way more features than I actually needed (variable length nested structures and stuff like that). Eventually, I scaled it back to this so that it was easier to maintain and so that I could make some performance optimizations (well as many as you can make with the struct module). -bob ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an idea for improving struct.unpack api
Thomas Heller <[EMAIL PROTECTED]> writes: >> Bob Ippolito <[EMAIL PROTECTED]> writes: > >>> This is my ctypes-like attempt at a high-level interface for struct. >>> It works well for me in macholib: >>> http://svn.red-bean.com/bob/py2app/trunk/src/macholib/ptypes.py > > Michael Hudson <[EMAIL PROTECTED]> writes: >> >> Unsurprisingly, that's fairly similar to mine :) > > So, why don't you both use the original? > ctypes works on the mac, too ;-) Well, I probably wrote mine before ctypes worked on the Mac... and certainly when I was away from internet access. I guess I should look at ctypes' interface, at least... Cheers, mwh -- I located the link but haven't bothered to re-read the article, preferring to post nonsense to usenet before checking my facts. -- Ben Wolfson, comp.lang.python ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: Subscribing to PEP updates
Barry Warsaw wrote: As an experiment, I just added a PEP topic to the python-checkins mailing list. You could subscribe to this list and just select the PEP topic (which matches the regex "PEP" in the Subject header or first few lines of the body). Give it a shot and let's see if that does the trick. Neat - I've subscribed to that topic now :) Do you mind if I suggest this to interested people on c.l.p? Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: Subscribing to PEP updates
On Fri, 2005-01-07 at 10:05, Nick Coghlan wrote: > Barry Warsaw wrote: > > As an experiment, I just added a PEP topic to the python-checkins > > mailing list. You could subscribe to this list and just select the PEP > > topic (which matches the regex "PEP" in the Subject header or first few > > lines of the body). > > > > Give it a shot and let's see if that does the trick. > > Neat - I've subscribed to that topic now :) > > Do you mind if I suggest this to interested people on c.l.p? Please do (he says, hoping it works :). -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
RE: [Python-Dev] Re: Subscribing to PEP updates
Title: RE: [Python-Dev] Re: Subscribing to PEP updates [Barry Warsaw] > As an experiment, I just added a PEP topic to the python-checkins > mailing list. You could subscribe to this list and just select the PEP > topic (which matches the regex "PEP" in the Subject header or first few > lines of the body). > > Give it a shot and let's see if that does the trick. Can the defaults be configured? Because now the config is this: - Which topic categories would you like to subscribe to? (default: "pep" not checked) - Do you want to receive messages that do not match any topic filter? (default: No) So, happens the following (happened to me, je): You subscribe to the list, and confirm the registration, and you never get a message unless you change this. Thanks. . Facundo Bitácora De Vuelo: http://www.taniquetil.com.ar/plog PyAr - Python Argentina: http://pyar.decode.com.ar/ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ADVERTENCIA. La información contenida en este mensaje y cualquier archivo anexo al mismo, son para uso exclusivo del destinatario y pueden contener información confidencial o propietaria, cuya divulgación es sancionada por la ley. Si Ud. No es uno de los destinatarios consignados o la persona responsable de hacer llegar este mensaje a los destinatarios consignados, no está autorizado a divulgar, copiar, distribuir o retener información (o parte de ella) contenida en este mensaje. Por favor notifíquenos respondiendo al remitente, borre el mensaje original y borre las copias (impresas o grabadas en cualquier medio magnético) que pueda haber realizado del mismo. Todas las opiniones contenidas en este mail son propias del autor del mensaje y no necesariamente coinciden con las de Telefónica Comunicaciones Personales S.A. o alguna empresa asociada. Los mensajes electrónicos pueden ser alterados, motivo por el cual Telefónica Comunicaciones Personales S.A. no aceptará ninguna obligación cualquiera sea el resultante de este mensaje. Muchas Gracias. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: [Csv] csv module TODO list
[Andrew McNamara] >> Also, review comments from Jeremy Hylton, 10 Apr 2003: >> >>I've been reviewing extension modules looking for C types that should >>participate in garbage collection. I think the csv ReaderObj and >>WriterObj should participate. The ReaderObj it contains a reference to >>input_iter that could be an arbitrary Python object. The iterator >>object could well participate in a cycle that refers to the ReaderObj. >>The WriterObj has a reference to a writeline callable, which could well >>be a method of an object that also points to the WriterObj. > I finally got around to looking at this, only to realise Jeremy did the > work back in Apr 2003 (thanks). One question, however - the GC doco in > the Python/C API seems to suggest to me that PyObject_GC_Track should be > called on the newly minted object prior to returning from the initialiser > (and correspondingly PyObject_GC_UnTrack should be called prior to > dismantling). This isn't being done in the module as it stands. Is the > module wrong, or is my understanding of the reference manual incorrect? The purpose of "tracking" and "untracking" is to let cyclic gc know when it (respectively) is and isn't safe to call an object's tp_traverse method. Primarily, when an object is first created at the C level, it may contain NULLs or heap trash in pointer slots, and then the object's tp_traverse could segfault if it were called while the object remained in an insane (wrt tp_traverse) state. Similarly, cleanup actions in the tp_dealloc may make a tp_traverse-sane object tp_traverse-insane, so tp_dealloc should untrack the object before that occurs. If tracking is never done, then the object effectively never participates in cyclic gc: its tp_traverse will never get called, and it will effectively act as an external root (keeping itself and everything reachable from it alive). So, yes, track it during construction, but not before all the members referenced by its tp_traverse are in a sane state. Putting the track call "at the end" of the constructor is usually best practice. tp_dealloc should untrack it then. In a debug build, that will assert-fail if the object hasn't actually been tracked. PyObject_GC_Del will untrack it for you (if it's still tracked), but it's risky to rely on that -- it's too easy to forget that Py_DECREFs on contained objects can end up executing arbitrary Python code (via __del__ and weakref callbacks, and via allowing other threads to run), which can in turn trigger a round of cyclic gc *while* your tp_dealloc is still running. So it's safest to untrack the object very early in tp_dealloc. I doubt this happens in the csv module, but an untrack/track pair should also be put around any block of method code that temporarily puts the object into a tp_traverse-insane state and that contains any C API calls that may end up triggering cyclic gc. That's very rare. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: Subscribing to PEP updates
Batista, Facundo wrote: [Barry Warsaw] > As an experiment, I just added a PEP topic to the python-checkins > mailing list. You could subscribe to this list and just select the PEP > topic (which matches the regex "PEP" in the Subject header or first few > lines of the body). > > Give it a shot and let's see if that does the trick. Can the defaults be configured? Because now the config is this: - Which topic categories would you like to subscribe to? (default: "pep" not checked) - Do you want to receive messages that do not match any topic filter? (default: No) So, happens the following (happened to me, je): You subscribe to the list, and confirm the registration, and you never get a message unless you change this. However, there is an additional line in the description: "If no topics of interest are selected above, then you will receive every message sent to the mailing list." In other words, don't check any topics, and you get everything. If you *do* check a topic, you only get the messages belonging to that topic. This seems to me a reasonable default. -- Sjoerd Mullender <[EMAIL PROTECTED]> signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: super() harmful?
On Jan 6, 2005, at 5:45 PM, Delaney, Timothy C (Timothy) wrote: Well, there's my autosuper recipe you've seen before: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/286195 which does basically what Philip descibes ... You missed the most important part of the example -- the automatic argument passing through unknowing methods. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Re: [Csv] Minor change to behaviour of csv module
Andrew> I'm considering a change to the csv module that could Andrew> potentially break some obscure uses of the module (but CSV files Andrew> usually quote, rather than escape, so the most common uses Andrew> aren't effected). I'm with the other respondents. This looks like a bug that should be squashed. Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: Subscribing to PEP updates
Barry Warsaw wrote: Please do (he says, hoping it works :). Speaking of which. . . care to poke PEP 0 or one of the other PEP's? There's probably a couple of PEP's which could be moved from 'Open' to 'Accepted' or 'Accepted' to 'Implemented' to try it out. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Re: super() harmful?
On Jan 6, 2005, at 12:13 PM, Guido van Rossum wrote:
So it has nothing to do with the new paradigm, just with backwards
compatibility. I appreciate those issues (more than you'll ever know)
but I don't see why you should try to discourage others from using the
new paradigm, which is what your article appears to do.
This is where I'm coming from:
In my own code, it is very rare to have diamond inheritance structures.
And if there are, even more rare that both sides need to cooperatively
override a method. Given that, super has no necessary advantage. And it
has disadvantages.
- Backwards compatibility issues
- Going along with that, inadvertent mixing of paradigms (you have to
remember which classes you use super with and which you don't or your
code might have hard-to-find errors).
- Take your choice of: a) inability to add optional arguments to your
methods, or b) having to use *args, **kwargs on every method and call
super with those.
- Having to try/catch AttributeErrors from super if you use interfaces
instead of a base class to define the methods in use.
So, I am indeed attempting to discourage people from using it, despite
its importance. And also trying to educate people as to what they need
to do if they have a case where it is necessary to use or if they just
decide I'm full of crap and want to use it anyways.
In order to make super really nice, it should be easier to use right.
Again, the two major issues that cause problems are: 1) having to
declare every method with *args, **kwargs, and having to pass those
and
all the arguments you take explicitly to super,
That's only an issue with __init__ or with code written without
cooperative MI in mind. When using cooperative MI, you shouldn't
redefine method signatures, and all is well.
I have two issues with that statement. Firstly, it's often quite useful
to be able to add optional arguments to methods. Secondly, that's not a
property of cooperative MI, but one of cooperative MI in python.
As a counterpoint, with Dylan, you can add optional keyword arguments
to a method as long as the generic was defined with the notation #key
(specifying that it will accept keyword arguments at all). This is of
course even true in a single inheritance situation like in the example
below.
Now please don't misunderstand me, here. I'm not at all trying to say
that Python sucks because it's not Dylan. I don't even particularly
like Dylan, but it does have a number of good ideas. Additionally,
Python and Dylan differ in fundamental ways: Python has classes and
inheritance, Dylan has generic functions/multimethods. Dylan is (I
believe) generally whole-program-at-a-time compiled/optimized, Python
is not. So, I think a solution for python would have to be
fundamentally different as well. But anyways, an example of what I'm
talking about:
define generic g (arg1 :: , #key);
define method g (arg1 :: , #key)
format-out("number.g\n");
end method g;
define method g (arg1 :: , #key base :: = 10)
next-method();
format-out("rational.g %d\n", base);
end method g;
define method g (arg1 :: , #key)
next-method();
format-out("integer.g\n");
end method g;
// Prints:
// number.g
// rational.g 1
// integer.g
g(1, base: 1);
// Produces: Error: Unrecognized keyword (base) as the second argument
in call of g
g(1.0, base: 1);
Cooperative MI doesn't have a really good solution for __init__.
Defining and calling __init__ only with keyword arguments is a good
solution. But griping about "traditionally" is a backwards
compatibility issue, which you said you were leaving behind.
Well, kind of. In my mind, it was a different kind of issue, as it
isn't solved by everyone moving over to using super. As nearly all the
code that currently uses super does so without using keyword arguments
for __init__, I considered it not so much backwards compatibility as a
re-educating users kind of issue, the same as the requirement for
passing along all your arguments.
Exactly. What is your next_method statement supposed to do?
Well that's easy. It's supposed to call the next function in the MRO
with _all_ the arguments passed along, even the ones that the current
function didn't explicitly ask for. I was afraid you might ask a hard
question, like: if E2 inherits C's __init__, how the heck is it
supposed to manage to take two arguments nonetheless. That one I
*really* don't have an answer for.
No need to reply except when you've changed the article. I'm tired of
the allegations.
Sigh.
James
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Concurrency and Python
Today's Slashdot (http://slashdot.org/articles/05/01/07/158236.shtml?tid=137) points to: "The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software" by Herb Sutter at "http://www.gotw.ca/publications/concurrency-ddj.htm";. Is Python a suitable language for concurrent programming? Should Python be a good language for concurrent programming? Python nicely satisfies several user needs now including teaching beginners, scripting, algorithm development, non time-critical code, and wrapping libraries. Which of these users will be needing concurrency? What is the state of programming theory for concurrency? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Re: [Python-checkins] python/dist/src/Python pythonrun.c, 2.161.2.15, 2.161.2.16
How's the merge going? And if I haven't already said thanks, then, thanks for doing it! Jeremy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Concurrency and Python
On Jan 7, 2005, at 2:44 PM, Edward C. Jones wrote: Is Python a suitable language for concurrent programming? Depends on what you mean. Python is not very good for shared-memory concurrent programming. (That is, threads). The library doesn't have enough good abstractions for locks/synchronization/etc, and, of course the big issue of CPython only allowing one thread to execute bytecode at a time. At the moment, threads are the fad, but I don't believe that will be scaling very well. As you scale up the number of CPUs, the amount of time wasted on memory synchronization similarly goes up, until you're wasting more time on memory consistency than doing actual work. Thus, I expect the trend to be more towards async message passing architectures (that is, multiple processes each with their own memory), instead, and I think Python is about as good for that as any existing language. Which is to say: reasonable, but not insanely great. What is the state of programming theory for concurrency? For an example of the kind of new language being developed around a asynchronous message passing model, see IBM's poorly-named "X10" language. I saw a talk on it and thought it sounded very promising. What it adds over the usual message passing system is an easier way to name and access remote data and to spawn parallel activities that operates on that data. The part about arrays of data spread out over a number of different "places" (roughly, a CPU and its own memory) and how to operate on them I found especially interesting. I tried to find their project website, but since their name conflicts with the home automation system, it's hard to google for. Or perhaps they don't have a website. Short summary information: http://www.csail.mit.edu/events/eventcalendar/calendar.php? show=event&id=131 Talk slides: http://www.cs.ualberta.ca/~amaral/cascon/CDP04/slides/sarkar.pdf More talk slides, and a video: http://www.research.ibm.com/vee04/video.html#sarkar "Vivek Sarkar, Language and Virtual Machine Challenges for Large-Scale Parallel Systems" James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: [Python-checkins] python/dist/src/Python pythonrun.c, 2.161.2.15, 2.161.2.16
Jeremy Hylton <[EMAIL PROTECTED]> writes: > How's the merge going? Looks like it's done. I tagged ast-branch when I finished: merged_from_MAIN_07JAN05 Right now I'm trying to get Python-ast.c to compile. It wasn't modified by the merge, so there's some other issue. > And if I haven't already said thanks, then, thanks for doing it! You're welcome! I volunteer to keep ast-branch synch'd, how often do you want to do it? -- KBK ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: Re: [Python-Dev] Re: [Python-checkins] python/dist/src/Python pythonrun.c, 2.161.2.15, 2.161.2.16
> > From: [EMAIL PROTECTED] (Kurt B. Kaiser) > Date: 2005/01/07 Fri PM 09:18:11 GMT > To: python-dev@python.org > Subject: Re: [Python-Dev] Re: [Python-checkins] python/dist/src/Python > pythonrun.c, 2.161.2.15, 2.161.2.16 > > Jeremy Hylton <[EMAIL PROTECTED]> writes: > > > How's the merge going? > > Looks like it's done. I tagged ast-branch when I finished: > > merged_from_MAIN_07JAN05 > > Right now I'm trying to get Python-ast.c to compile. It wasn't > modified by the merge, so there's some other issue. > Python-ast.c should be autogenerated in the make process by asdl_c.py. There are still some bugs in it. The fix I think you need is posted. A full diff against the current python_ast.c is attached to patch 742621. @@ -1310,7 +1310,7 @@ free_expr(o->v.Repr.value); break; case Num_kind: -Py_DECREF(o->v.Num.n); +free_expr(o->v.Num.n); break; case Str_kind: Py_DECREF(o->v.Str.s) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: [Python-checkins] python/dist/src/Python pythonrun.c, 2.161.2.15, 2.161.2.16
On Fri, 07 Jan 2005 16:18:11 -0500, Kurt B. Kaiser <[EMAIL PROTECTED]> wrote: > Looks like it's done. I tagged ast-branch when I finished: > > merged_from_MAIN_07JAN05 > > Right now I'm trying to get Python-ast.c to compile. It wasn't > modified by the merge, so there's some other issue. I'm getting a compilation failure in symtable.c: gcc -pthread -c -fno-strict-aliasing -g -Wall -Wstrict-prototypes -I. -I../Include -DPy_BUILD_CORE -o Python/symtable.o ../Python/symtable.c ../Python/symtable.c: In function `symtable_new': ../Python/symtable.c:193: structure has no member named `st_tmpname' Do you see that? There is this one ugly corner of Python-ast.c. There's a routine that expects to take a pointer to a node, but instead gets passed an int. The generated code is bogus, and I haven't decided if it needs to be worried about. You need to manually edit the generated code to add a cast. > > And if I haven't already said thanks, then, thanks for doing it! > > You're welcome! I volunteer to keep ast-branch synch'd, how often > do you want to do it? I don't think we'll need to merge again. This last merge got all the language changes that were made for 2.4. Since we've agreed to a moratorium on more compiler/bytecode changes, we shouldn't need to merge from the head again. Jeremy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: [Python-checkins] python/dist/src/Python pythonrun.c, 2.161.2.15, 2.161.2.16
Jeremy Hylton <[EMAIL PROTECTED]> writes: > ../Python/symtable.c:193: structure has no member named `st_tmpname' > > Do you see that? Yeah, the merge eliminated it from the symtable struct in symtable.h. You moved it to symtable_entry at rev 2.12 in MAIN :-) I'll research it. Apparently my build differs enough so that I'm still stuck in Python-ast.c (once I had fixed pythonrun.c). > There is this one ugly corner of Python-ast.c. There's a routine > that expects to take a pointer to a node, but instead gets passed an > int. The generated code is bogus, and I haven't decided if it needs > to be worried about. You need to manually edit the generated code to > add a cast. OK, I was looking in that direction. Problem is with cmpop stuff. Three hard errors when compiling. OpenBSD. [...] > I don't think we'll need to merge again. This last merge got all the > language changes that were made for 2.4. Since we've agreed to a > moratorium on more compiler/bytecode changes, we shouldn't need to > merge from the head again. Is the plan to merge ast-branch to MAIN? If so, it's a little tricky since all the changes to MAIN are on ast-branch. So just before the final merge we need to merge MAIN to ast-branch once more and then merge the diff from HEAD to ast-branch back to MAIN. Or something like that. -- KBK ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
RE: [Python-Dev] an idea for improving struct.unpack api
I will try to respond to all comments at once.
But first a clarification:
-I am not trying to design a high-level API on top of existing
struct.unpack and
-I am not trying to design a replacement for struct.unpack
(If I were to replace struct.unpack(), then I would probably go along
the lines of StructReader suggested by Raymond)
I view struct module as a low-level (un)packing library on top on which
a more complex stuff can be built and I am simply suggesting a way to
improve this low level functionality...
> > We could have an optional offset argument for
> >
> > unpack(format, buffer, offset=None)
> >
> > the offset argument is an object which contains a single integer field
> > which gets incremented inside unpack() to point to the next byte.
> As for "passing offset implies the length is calcsize(fmt)" sub-concept,
> I find that slightly more controversial. It's convenient,
> but somewhat ambiguous; in other cases (e.g. string methods) passing a
> start/offset and no end/length means to go to the end.
I am not sure I agree: in most cases starting offset and no
length/end just means: "start whatever you are doing at this offset and
stop it whenever you are happy.."
At least that's the way I was alway thinking about functions like
string.find() and friends
Suggested struct.unpack() change seems to fit this mental model very well
>> the offset argument is an object which contains a single integer field
>> which gets incremented inside unpack() to point to the next byte.
> I find this just too "magical".
Why would it be magical? There is no guessing of user intentions involved.
The function simply returns/uses an extra piece of information if the user
asks for it. And the function already computes this piece of
information..
> It's only useful when you're specifically unpacking data bytes that are
> compactly back to back (no "filler" e.g. for alignment purposes)
Yes, but it's a very common case when dealing with binary files formats.
Eg. I just looked at xdrlib.py code and it seems that almost every
invocation of struct._unpack would shrink from 3 lines to 1 line of code
(i = self.__pos
self.__pos = j = i+4
data = self.__buf[i:j]
return struct.unpack('>l', data)[0]
would become:
return struct.unpack('>l', self.__buf, self.__pos)[0]
)
There are probably other places in stdlib which would benefit from this
api and stdlib does not deal with binary files that much..
>and pays some conceptual price -- introducing a new specialized type
> to play the role of "mutable int"
but the user does not have to pay anything if he does not need it! The
change is backward compatible. (Note that just supporting int offsets
would eliminate slicing, but it would not eliminate other annoyances,
and it's possible to support both Offset and int args, is it worth the
hassle?)
> and having an argument mutated, which is not usual in Python's library.
Actually, it's so common that we simply stop noticing it :-)
Eg. when we call a superclass's method:
SuperClass.__init__(self)
So, while I agree that there is an element of unusualness in the
suggested unpack() API, this element seems pretty small to me
> All in all, I suspect that something like.
> hdrsize = struct.calcsize(hdr_fmt)
> itemsize = struct.calcsize(item_fmt)
> reclen = length_of_each_record
> rec = binfile.read(reclen)
> hdr = struct.unpack(hdr_fmt, rec, 0, hdrsize)
> for offs in itertools.islice(xrange(hdrsize, reclen, itemsize),
hdr[0]):
> item = struct.unpack(item_fmt, rec, offs, itemsize)
> # process item
>might be a better compromise
I think I again disagree: your example is almost as verbose as the current
unpack() api and you still need to call calcsize() explicitly and I
don't think there is any chance of gaining any noticeable
perfomance benefit. Too little gain to bother with any changes...
> struct.pack/struct.unpack is already one of my least-favourite parts
> of the stdlib. Of the modules I use regularly, I pretty much only ever
> have to go back and re-read the struct (and re) documentation because
> they just won't fit in my brain. Adding additional complexity to them
> seems like a net loss to me.
Net loss to the end programmer? But if he does not need new
functionality he doesnot have to use it! In fact, I started with providing
an example of how new api makes client code simpler
> I'd much rather specify the format as something like a tuple of values -
>(INT, UINT, INT, STRING) (where INT &c are objects defined in the
>struct module). This also then allows users to specify their own formats
>if they have a particular need for something
I don't disagree, but I think it's orthogonal to offset issue
Ilya
On Thu, 6 Jan 2005, Raymond Hettinger wrote:
> [Ilya Sandler]
> > A problem:
> >
> > The current struct.unpack api works well for unpacking C-structures
> where
> > everything is usually unpacked at once, but it
> > becomes
Re: [Python-Dev] an idea for improving struct.unpack api
Ilya Sandler wrote: item=unpack( "", rec, offset) How about making offset a standard integer, and change the signature to return a tuple when it is used: item = unpack(format, rec) # Full unpacking offset = 0 item, offset = unpack(format, rec, offset) # Partial unpacking The second item in the returned tuple being the offset of the first byte after the end of the unpacked item. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
RE: [Python-Dev] an idea for improving struct.unpack api
[Ilya Sandler] > I view struct module as a low-level (un)packing library on top on which > a more complex stuff can be built and I am simply suggesting a way to > improve this low level functionality... > > > We could have an optional offset argument for > > > > > > unpack(format, buffer, offset=None) > > > > > > the offset argument is an object which contains a single integer field > > > which gets incremented inside unpack() to point to the next byte. -1 on any modification of the existing unpack() function. It is already at its outer limits of complexity. Attaching a stateful tracking object (needing its own constructor and api) is not an improvement IMO. Also, I find the proposed "offset" object to be conceptually difficult to follow for anything other than the simplest case -- for anything else, it will make designing, reviewing, and debugging more difficult than it is now. In contrast, code built using the StructReader proposal leads to more flexible, readable code. Experience with the csv module points to reader objects being a better solution. [Nick Coghlan] > How about making offset a standard integer, and change the signature to > return a > tuple when it is used: > >item = unpack(format, rec) # Full unpacking >offset = 0 >item, offset = unpack(format, rec, offset) # Partial unpacking > > The second item in the returned tuple being the offset of the first byte > after > the end of the unpacked item Using standard integers helps improve the proposal by making the operation less obscure. But having than having the signature change is bad; create a separate function instead: item, offset = unpack_here(format, rec, offset) One other wrinkle is that "item" is itself a tuple and the whole thing looks odd if unpacked: ((var0, var1, var2, var3), offset) = unpack_here(fmtstr, rec, offset) Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an idea for improving struct.unpack api
> How about making offset a standard integer, and change the signature to
> return tuple when it is used:
> item, offset = unpack(format, rec, offset) # Partial unpacking
Well, it would work well when unpack results are assigned to individual
vars:
x,y,offset=unpack( "ii", rec, offset)
but it gets more complicated if you have something like:
coords=unpack("10i", rec)
How would you pass/return offsets here? As an extra element in coords?
coords=unpack("10i", rec, offset)
offset=coords.pop()
But that would be counterintuitive and somewhat inconvinient..
Ilya
On Sat, 8 Jan 2005, Nick Coghlan wrote:
> Ilya Sandler wrote:
> > item=unpack( "", rec, offset)
>
> How about making offset a standard integer, and change the signature to
> return a
> tuple when it is used:
>
>item = unpack(format, rec) # Full unpacking
>offset = 0
>item, offset = unpack(format, rec, offset) # Partial unpacking
>
> The second item in the returned tuple being the offset of the first byte after
> the end of the unpacked item.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia
> ---
> http://boredomandlaziness.skystorm.net
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/ilya%40bluefir.net
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: [Python-checkins] python/dist/src/Python pythonrun.c, 2.161.2.15, 2.161.2.16
[EMAIL PROTECTED] (Kurt B. Kaiser) writes: > [JH] >> ../Python/symtable.c:193: structure has no member named `st_tmpname' >> >> Do you see that? > > Yeah, the merge eliminated it from the symtable struct in symtable.h. > You moved it to symtable_entry at rev 2.12 in MAIN :-) > > I'll research it. I think it would be more efficient if you tackled it since almost all the work is in compile.c ==> newcompile.c The relevant changes are compile.c 2.286 symtable.h 2.12 symtable.c 2.11 www.python.org/sf/734869 > Apparently my build differs enough so that I'm still stuck in > Python-ast.c (once I had fixed pythonrun.c). I resolved all the errors/warnings and diffed to against respository. I was astonished to see the same changes, slightly different, being replaced by mine. Those were /your/ tweaks. Apparently the $(AST_H) $(AST_C): target ran and Python-ast.c was recreated (without the changes). It's not clear to me how/why that happened. I did start with a clean checkout, but it seems that the target only runs if Python-ast.c and/or its .h are missing (they should have been in the checkout), or older than Python.asdl, which they are not. I don't see them in the .cvsignore. Very amusing. -- KBK ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an idea for improving struct.unpack api
Ilya Sandler wrote:
How about making offset a standard integer, and change the signature to
return tuple when it is used:
item, offset = unpack(format, rec, offset) # Partial unpacking
Well, it would work well when unpack results are assigned to individual
vars:
x,y,offset=unpack( "ii", rec, offset)
but it gets more complicated if you have something like:
coords=unpack("10i", rec)
How would you pass/return offsets here? As an extra element in coords?
coords=unpack("10i", rec, offset)
offset=coords.pop()
But that would be counterintuitive and somewhat inconvinient..
I was thinking more along the lines of returning a 2-tuple with the 'normal'
result of unpack as the first element:
coords, offset = unpack("ii", rec, offset)
x, y = coords
Raymond's suggestion of a separate function like 'unpack_here' is probably a
good one, as magically changing function signatures are evil. Something like:
def unpack_here(format, record, offset = 0):
end = offset + calcsize(format)
return (unpack(format, record[offset:end]), end)
Presumably, a C version could avoid the slicing and hence be significantly more
efficient.
Yes, the return type is a little clumsy, but it should still make it easier to
write more efficient higher-level API's that unpack the structure a piece at a time.
Cheers,
Nick.
--
Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia
---
http://boredomandlaziness.skystorm.net
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
