Re: [Python-Dev] Expression optimizations
On Tue, Feb 10, 2009 at 10:24 AM, Cesare Di Mauro wrote: > Could it be applyable to other operations as well? So, if I wrote: > c = not(a < b) > the compiler and/or peephole optimizer can generate bytecodes instructions > which, instead, execute the following operation: > c = a >= b > Those two expressions are equivalent for integers, but not necessarily equivalent for objects that define their own comparison operator. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Expression optimizations
On Tue, Feb 10, 2009 at 11:16 AM, Steve Holden wrote: > That's true, but the same *could* be said about the existing > optimizations for objects that define their own __contains__. > No, because there isn't a __not_contains__, so you cannot define the inverse operation differently. "not a in b" and "a not in b" have exactly the same effects. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Expression optimizations
On Tue, Feb 10, 2009 at 2:36 PM, Cesare Di Mauro
wrote:
> OK, so I can make assumptions only for built-in types.
>
Yes, but even there you have to be careful of odd corner-cases, such as:
>>> nan = float('nan')
>>> nan < nan
False
>>> nan >= nan
False
--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Choosing a best practice solution for Python/extension modules
On Fri, Feb 20, 2009 at 1:44 PM, Brett Cannon wrote: > Now, from what I can tell, Antoine is suggesting having _pyio and a _io and > then io is simply: > > try: from _io import * > except ImportError: from _pyio import * > > That works for testing as you can then have test classes have an attribute > for the module to use and then create two subclasses which set what module > to use (kind of like how test_warnings currently does it). But this only > really works for complete module replacements, not modules like pickle where > only key portions have been rewritten (which happens more often than the > complete rewrite). > A slight change would make it work for modules where only key functions have been rewritten. For example, pickle.py could read: from _pypickle import * try: from _pickle import * except ImportError: pass -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Choosing a best practice solution for Python/extension modules
On Fri, Feb 20, 2009 at 5:27 PM, Nick Coghlan wrote: > Brett Cannon wrote: > > If you import pickle and call pickle.A() you will get -13 which is not > > what you are after. > > Ah, you may want to think about that a bit more. There's a reason > globals are looked up when they're used rather than when their function > is defined. Even in your own example, _B isn't defined at all when you > define A. > No, I'm afraid Brett is quite right. Globals are looked up when the function is executed, true, but they are looked up within the module that defined the function. Functions defined in _pypickle would only call the _pypickle version of functions. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] .pythonrc.py in man page
On Sun, Feb 22, 2009 at 8:25 PM, Guido van Rossum wrote: > > Perhaps this entry in the man page is obsolete and should be removed? > > Not at all. > For what it's worth, the 2.6.1 documentation states: "Deprecated since version 2.6: The user module has been removed in Python 3.0." If user.py has indeed been removed, then Mitchell is correct. http://docs.python.org/library/user.html -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa
On Wed, Mar 4, 2009 at 12:54 PM, Guido van Rossum wrote: > The same as always. We don't change APIs in bugfix releases. > This question is actually for the Zope folks and others who have had problems with the 2.6 asyncore/asynchat: Are any of the problems due to a change in the documented API... or are they all due to changes in undocumented internals that your code relied on? Myself, the only change that I noticed going from 2.5 to 2.6 was that the undocumented signature of asynchat.__init__() changed from taking a parameter called "conn" to one with the same meaning called "sock". -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa
On Wed, Mar 4, 2009 at 3:14 PM, Ivan Krstić < [email protected]> wrote: > I spent about a half hour sometime in the last month talking this through > with Itamar, though not in great detail. I'd be interested in sitting down > with both of you and trying to establish more precisely how much work is > necessary to get something to actually happen here. I won't outright promise > a PEP, but I'll promise at the very least to write down elaborate notes that > someone could turn into a PEP relatively straightforwardly. Deal? > Will any or all of you be at PyCon? I'd be willing to put in the extra work to turn your notes into a PEP. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa
On Thu, Mar 5, 2009 at 1:30 PM, Neil Schemenauer wrote: > What I would like to see is a module that provides a low-level API > for doing cross-platform asynchronous IO. The two necessary parts > are: > >* a wrapper that allows non-blocking reads and writes on > channels (sockets, file descriptors, serial ports, etc) > >* a select() or epoll like interface that allows waiting on > multiple channels > Two thoughts: If you have a working select(), it will tell you the sockets on which read() and write() won't block, so non-blocking reads and writes are not necessary. On Windows, sockets, pipes, and files are each completely distinct types with their own functions and unifying them under one select()-like interface requires faking it using threads behind-the-scenes AFAIK. Personally, I'd be happy with continuing to only support socket objects on Windows. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pickler/Unpickler API clarification
On Fri, Mar 6, 2009 at 5:45 AM, Antoine Pitrou wrote: > If these strings are not interned, then perhaps they should be. > There is a similar optimization proposal (w/ patch) for attribute names: > http://bugs.python.org/issue5084 > If I understand correctly, that would help with unpickling, but wouldn't solve Michael's problem as, without memo, each pickle would still need to store a copy. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa
On Sat, Mar 7, 2009 at 8:28 PM, wrote: > Anybody interested in working on this at a PyCon Sprint? I won't be > attending the conference proper, but plan to spend a couple days sprinting, > I'll be there and interested. :) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] wait time [was: Ext4 data loss]
On Thu, Mar 12, 2009 at 4:09 PM, "Martin v. Löwis" wrote:
> So when you select "Save" in your application, would you like the data
> to be saved, or would you accept that they get lost? If the latter,
> what kind of interaction would you perform with your application to
> indicate that you *do* want the data to appear on disk?
>
I accept that if the computer crashes at just the wrong moment as I click
Save, my changes will not actually be Saved. No amount of diligence in the
implementation of close() can prevent that since the computer can crash
before the program calls close().
I oppose applications that lose or corrupt both my new save and my
*previous* save if the computer crashes at the wrong moment. That would
cause me to lose not only my most recent changes (an inconvenience), but
also all the work I have ever done on the file (a major headache for anyone
who doesn't make regular backups).
However, defaulting to calling fsync() when closing a file will:
1) Cripple performance for the many applications that don't need it (e.g.,
temporary files)
2) Fail to prevent data loss for applications that use the
truncate-and-rewrite paradigm for saving
Consider the following example:
with open('mysavefile', 'w') as f:
f.write(data)
f.flush()
os.fsync(f.fileno())
f.close()
If the system crashes after the call to open(), but before the call to
fsync(), then both the old and the new mysavefile may be gone.
Since needing to safely replace a file with new data is a moderately common
task, perhaps it would be useful to have a convenience class that looks like
a file, but takes care of the ugly details behind-the-scenes? Something
vaguely like this flawed and untested class:
class open_for_safe_replacement(file): # needs a better name
def __init__(self, path, flags):
if 'w' not in flags:
raise RuntimeError, 'Writing without writing?'
self.path = path
self.tmp_name =
some_function_that_generates_a_safe_temporary_filename() # good luck
file.__init__(self.tmp_name, flags)
def close(self):
self.flush()
os.fsync(self.fileno())
self.close()
os.rename(self.tmp_name, self.path) # won't work on Windows :-(
then we could simply:
with appropriate_module.open_for_safe_replacement('mysavefile', 'w'):
f.write(data)
--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] speeding up PyObject_GetItem
In Python 2.5, list objects were special-cased to skip PyObject_GetItem and go straight to PyList_GET_ITEM. That special case gave made other sequences 20% slower than lists for getitem operations. The special case was removed in Python 3 (haven't checked 2.6). Today I was tracing through how PyObject_GetItem works, trying to figure it why it's so much slower, to see if we can get back some of that speed without special-casing just one type. Below is a rough sketch of what PyObject_GetItem does for the common case of a sequence with a valid positive integer parameter, and my observations on where there is room for improvement. I'm willing to write patches and test them, but I wanted to get some general feedback first. After all, it may be the way it is for A Very Good Reason that I'm not aware of. ;-) The code paths for PyObject_SetItem and PyObject_DelItem are essentially the same, so any speed improvments to one could easily be applied to the other two operations. PyObject_GetItem ob i ob==NULL # can't be null anyway i==NULL # can't be null anyway is mapping? ob is sequence? ob PyIndex_Check i tp_as_number i tp_flags i tp_as_number->nb_index i PyNumber_AsSsize_t i PyNumber_Index i i==NULL # still can't be null PyLong_Check i i==NULL # still can't be null PyLong_Check i # redundant => v v == -1 PySequence_GetItem ob v ob==NULL # still can't be null is sequence? ob# redundant sq_item? ob sq_item ob v I think there are two avenues for speed improvement, both of which involve removing redundancy: 1) Assume the index is a PyLong until proven otherwise The PyIndex_Check in PyObject_GetItem looks pretty useless. If it returns false, PyObject_GetItem throws a type error. If we skipped the PyIndex_Check when it would have returned false, PyNumber_AsSsize_t would later call PyIndex_Check and throw a type error. Unless I'm missing something, this is a clear win and makes the code simpler. In PyNumber_AsSsize_t, we could speed up the common case by trying PyLong_Check first, and if it fails then calling PyNumber_Index. This change would make the common case faster, but make the code a few lines longer. 2) Remove some of the exessive checking for NULL unless Py_DEBUG is defined Many of the routines in abstract.c check their parameters for NULL, as a sanity check, and throw a SystemError if NULL is detected. I'm tempted to suggest making these checks only when Py_DEBUG is defined, but I suspect if you wanted it that way, you would have changed it already. ;) Assuming you really want the NULL checks in production Python, there are 5 checks for NULL even though there are only two parameters. Seems like overkill? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] speeding up PyObject_GetItem
On Tue, Mar 24, 2009 at 4:30 AM, Hrvoje Niksic wrote: > Agreed, and more importantly, I have yet to be convinced that those NULL > checks introduce a measurable slowdown. Daniel, have you tried measuring > the performance difference with only the NULL checks removed? I'll play around with different permutations and report back on their impact. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] speeding up PyObject_GetItem
On Tue, Mar 24, 2009 at 7:30 AM, Daniel Stutzbach < [email protected]> wrote: > On Tue, Mar 24, 2009 at 4:30 AM, Hrvoje Niksic wrote: > >> Agreed, and more importantly, I have yet to be convinced that those NULL >> checks introduce a measurable slowdown. Daniel, have you tried measuring >> the performance difference with only the NULL checks removed? > > > I'll play around with different permutations and report back on their > impact. After playing around with it, I see that I was barking up the wrong tree. That's what I get for tracing through code visually instead of using a debugger. PyList implements tp_as_mapping, so it's mp_subscript method is called and the PySequence line is never used. I tried many variations of special casing at various points in the path and removing extra checks. It looks like function calls are the biggest expense. Calls to functions within the same module appear to be cheap (presumably because gcc inlines them). 100 nanoseconds, py3k trunk: ceval -> PyObject_GetItem (object.c) -> list_subscript (listobject.c) -> PyNumber_AsSsize_t (object.c) -> PyLong_AsSsize_t (longobject.c) 86 nanoseconds, by special-casing PyLong in list_subscript ceval -> PyObject_GetItem (object.c) -> list_subscript (listobject.c) -> PyLong_AsSsize_t (longobject.c) cost: could slow down mylist[some_PyNumber_that_is_not_a_long_yet_still_a_valid_index] (who cares?) 75 nanoseconds, by special-casing PyList and PyLong in PyObject and using PyList_GET_ITEM ceval -> PyObject_GetItem (object.c) -> PyLong_AsSsize_t (longobject.c) cost: could slow down not_a_list[x] 75 nanoseconds, by special-casing PyList and PyLong in ceval and using PyList_GET_ITEM ceval -> PyLong_AsSsize_t (longobject.c) cost: could slow down not_a_list[x] I had been hoping to find something general to speed up all uses of __getitem__, but that doesn't seem possible. Oh well. :-( -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] speeding up PyObject_GetItem
On Tue, Mar 24, 2009 at 10:13 AM, Mark Dickinson wrote: > 2009/3/24 Daniel Stutzbach : > > [...] > > 100 nanoseconds, py3k trunk: > > ceval -> PyObject_GetItem (object.c) -> list_subscript (listobject.c) -> > > PyNumber_AsSsize_t (object.c) -> PyLong_AsSsize_t (longobject.c) > > [more timings snipped] > > Does removing the PyLong_Check call in PyLong_AsSsize_t > make any noticeable difference to these timings? > Making no other changes from the trunk, removing the PyLong_Check and NULL check from PyLong_AsSsize_t shaves off 4 nanoseconds (or around 4% since the trunk is around 100 nanoseconds). Here's what I'm testing with, by the way: ./python.exe Lib/timeit.py -r 10 -s 'x = list(range(10))' 'x[5]' -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] __length_hint__
Iterators can implement a method called __length_hint__ that provides a hint to certain internal routines (such as list.extend) so they can operate more efficiently. As far as I can tell, __length_hint__ is currently undocumented. Should it be? If so, are there any constraints on what an iterator should return? I can think of 3 possible rules, each with advantages and disadvantages: 1. return your best guess 2. return your best guess that you are certain is not higher than the true value 3. return your best guess that you are certain is not lower than the true value Also, I've noticed that if a VERY large hint is returned by the iterator, list.extend will sometimes disregard the hint and try to allocate memory incrementally (correct for rule #1 or #2). However, in another code path it will throw a MemoryError immediately based on the hint (correct for rule #3). -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping bytes "support" in json
On Thu, Apr 9, 2009 at 6:01 AM, Barry Warsaw wrote:
> Anyway, aside from that decision, I haven't come up with an elegant way to
> allow /output/ in both bytes and strings (input is I think theoretically
> easier by sniffing the arguments).
>
Won't this work? (assuming dumps() always returns a string)
def dumpb(obj, encoding='utf-8', *args, **kw):
s = dumps(obj, *args, **kw)
return s.encode(encoding)
--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping bytes "support" in json
On Fri, Apr 10, 2009 at 10:06 PM, "Martin v. Löwis" wrote: > However, I really think that this question cannot be answered by > reading the RFC. It should be answered by verifying how people use > the json library in 2.x. > I use the json module in 2.6 to communicate with a C# JSON library and a JavaScript JSON library. The C# and JavaScript libraries produce and consume the equivalent of str, not the equivalent of bytes. Yes, the data eventually has to go over a socket as bytes, but that's often handled by a different layer of code. For JavaScript, data is typically received by via XMLHttpRequest(), which automatically figures out the encoding from the HTTP headers and/or other information (defaulting to UTF-8) and returns a str-like object that I pass to the JavaScript JSON library. For C#, I wrap the socket in a StreamReader object, which decodes the byte stream into a string stream (similar to Python's new TextIOWrapper class). Hope that helps, -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping bytes "support" in json
On Mon, Apr 13, 2009 at 12:19 PM, "Martin v. Löwis" wrote:
> > I use the json module in 2.6 to communicate with a C# JSON library and a
> > JavaScript JSON library. The C# and JavaScript libraries produce and
> > consume the equivalent of str, not the equivalent of bytes.
>
> I assume there is a TCP connection between the json module and the
> C#/JavaScript libraries?
>
Yes, there's a TCP connection. Sorry for not making that clear to begin
with.
I also sometimes store JSON objects in a database. In that case, I pass
strings to the database API which stores them in a TEXT field. Obviously
somewhere they get encoding to bytes, but that's handled by the database.
> If so, it doesn't matter what representation these implementations chose
> to use.
True, I can always convert from bytes to str or vise versa. Sometimes it is
illustrative to see how others have chosen to solve the same problem. The
JSON specification and other implementations serializes an object to a
string. Python's json.dumps() needs to either return a str or let the user
specify an encoding.
At least one of these two needs to work:
json.dumps({}).encode('utf-16le') # dumps() returns str
'{\x00}\x00'
json.dumps({}, encoding='utf-16le') # dumps() returns bytes
'{\x00}\x00'
In 2.6, the first one works. The second incorrectly returns '{}'.
--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping bytes "support" in json
On Mon, Apr 13, 2009 at 3:28 PM, Bob Ippolito wrote: > It's not a bug in dumps, it's a matter of not reading the > documentation. The encoding parameter of dumps decides how byte > strings should be interpreted, not what the output encoding is. > You're right; I apologize for not reading more closely. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping bytes "support" in json
On Mon, Apr 13, 2009 at 3:02 PM, "Martin v. Löwis" wrote:
> > True, I can always convert from bytes to str or vise versa.
>
> I think you are missing the point. It will not be necessary to convert.
Sometimes I want bytes and sometimes I want str. I am going to be
converting some of the time. ;-)
Below is a basic CGI application that assumes that json module works with
str, not bytes. How would you write it if the json module does not support
returning a str?
print("Content-Type: application/json; charset=utf-8")
input_object = json.loads(sys.stdin.read())
output_object = do_some_work(input_object)
print(json.dumps(output_object))
print()
The questions is: which of them is more appropriate, if, what you want,
> is bytes. I argue that the second form is better, since it saves you
> an encode invocation.
>
If what you want is bytes, encoding has to happen somewhere. If the json
module has some optimizations to do the encoding at the same time as the
serialization, great. However, based on the original post of this thread,
it sounds like that code doesn't exist or doesn't work correctly.
What's the benefit of preventing users from getting a str out if that's what
they want?
--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed: drop unnecessary "context" pointer from PyGetSetDef
On Mon, May 4, 2009 at 4:10 AM, Larry Hastings wrote: > So: you don't need it, it clutters up our code (particularly typeobject.c), > and it adds overhead. The only good reason to keep it is backwards > compatibility, which I admit is a fine reason. > If you make the change, will 3rd party code that relies on it fail in unexpected ways, or will they just get a compile error? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed: drop unnecessary "context" pointer from PyGetSetDef
On Mon, May 4, 2009 at 3:00 PM, Mark Dickinson wrote: > But I guess the bigger issue is that extensions already compiled against > 3.0 > that use PyGetSetDef (even if they don't make use of the closure field) > won't work with 3.1 without a recompile: they'll segfault, or otherwise > behave > unpredictably. > I was under the impression that binary compatibility was only guaranteed within a minor revision (e.g., 2.6.1 must run code compiled for 2.6.0, but 2.7.0 doesn't have to). I've been wrong before, though. Certainly the C extension module I maintain is sprinkled with #ifdef's so it will compile under 2.5, 2.6, and 3.0. ;-) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] using help function in Py3k
On Tue, May 5, 2009 at 5:41 AM, s|s wrote: > LookupError: unknown encoding: uft-8 > uft-8? Looks like a variation of Issue 4540 <http://bugs.python.org/issue4540> (or a duplicate? I can't tell) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com/> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] special method lookup: how much do we care?
On Fri, May 8, 2009 at 1:09 PM, Benjamin Peterson wrote: > I've know hit __enter__ and __exit__. The compiler > generates LOAD_ATTR instructions for these, so it uses the normal > lookup. The only way I can see to fix this is add a new opcode which > uses _PyObject_LookupSpecial, but I don't think we really care this > much. Opinions? > Why does this problem arise only with __enter__ and __exit__? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] special method lookup: how much do we care?
On Fri, May 8, 2009 at 6:14 PM, Benjamin Peterson wrote: > Normally special methods use slots of the PyTypeObject struct. > typeobject.c looks up all those methods on Python classes correctly. > In the case of __enter__ and __exit__, the compiler generates bytecode > to look them up, and that bytecode use PyObject_Getattr. Would this problem apply to all special methods that don't use a slot in PyTypeObject, then? I know of several other examples: __reduce__ __setstate__ __reversed__ __length_hint__ __sizeof__ (unless I misunderstand the definition of "special methods", which is possible) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Flow of control - a new way - Idea
On Tue, Jun 30, 2009 at 1:07 AM, vin vin<[email protected]> wrote: > at that point if we think to move directly to B (what error handler do > if that B has the handler defined of the error), changing the frames > instruction pointer to back to the B's position (in python code without > raising an exception) is a really useful for this web applications. If I understand you correctly, the language feature you are looking for is called a "continuation", which was originally developed in the LISP/Scheme world (as "call/cc"). I'm not familiar with the details, but I believe you can get them in Stackless Python. Hope that helps, -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com/> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3144 review.
On Tue, Sep 15, 2009 at 12:16 PM, Scott Dial
> wrote:
> >>> addr = ipaddr.IPAddress("10.1.2.3/255.255.240.0")
> ...
> ipaddr.IPAddressIPValidationError: '98.223.189.24/255.255.240.0' is not
> a valid address (hint, it's probably a network)
>
> Because, it *is* a address of a host on a network. I gave in and said:
>
10.1.2.3 is the address of a host on a network. 255.255.240.0 is a subnet
mask.
>>> net = ipaddr.IPNetwork("10.1.2.3/255.255.240.0")
>
> But then, I was dumbfounded as to how I could get the gateway IP from
> this IPNetwork object. It took me a while to figure out that you can
> iterate over IPNetwork instances:
>
Any IP address on an IP network could be the gateway, so there is no
reliable way to determine the gateway IP from the network address and subnet
mask alone.
Technicalities aside, I agree with you that IPNetwork appears to be doing
double-duty as an IPNetwork type and an IPAddressWithNetwork type, which I
find confusing.
--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3144 review.
On Wed, Sep 16, 2009 at 4:59 PM, Greg Ewing wrote: > Some people have claimed that the gateway address of a > network isn't necessarily the zero address in that network. > I'll go further: I don't think it's even legal for the gateway address to be the zero address of the network (and I used to program the embedded software in routers for a living :) ). > If that's true, then you *can't* calculate the network > address from a host address and a netmask -- there isn't > enough information. In a router or operating system kernel, an IPv4 address or netmask is stored in a 32-bit unsigned integer. Host addresses, network addresses, and netmasks satisfy the following properties, where & is the bitwise-AND operation: network address & netmask == network address host address & netmask == network address of that host A gateway is just a host that will forward packets. A gateway address has the same properties as a host address and isn't the same as the zero address. Every host has a routing table that contains a list of (network address, gateway, netmask) tuples. To see the list, just run "/sbin/route -n" on any Linux system (and most other Unixes; root is not typically required) or "route print" on a Windows system. To route a packet, the operating system essentially performs the following algorithm: def find_gateway_for_destination(destination_address): for (network_address, gateway, netmask) in list_of_routing_table_entires: if network_address == destination_address & netmask: return gateway raise DestinationUnreachable Furthermore, an IPNetwork object > needs to be able to represent a network address whose > address part contains bits that aren't in the mask. > Network addresses never contain bits that aren't in the mask (due to the rule: "network address & netmask == network address"). -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3144 review.
On Thu, Sep 17, 2009 at 9:26 AM, DrKJam wrote: > Please can we have the following RFCs added to the references section that > cover many of the aspects covered by this PEP? > > RFC 791 - Internet Protocol > RFC 1918 - Address Allocation for Private Internets > RFC 3330 - Special-Use IPv4 Addresses > RFC 4291 - IPv6 Addressing Architecture > RFC 4632 - Classless Inter-domain Routing (CIDR): The Internet Address > +1 -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fuzziness in io module specs
On Fri, Sep 18, 2009 at 2:17 PM, Pascal Chambon wrote: > - it is unclear what truncate() methods do with the file pointer, and even > if the current implementation simply moves it to the truncation point, it's > very contrary to the standard way of doing under unix, where the file > pointer is normally left unchanged. Shouldn't we specify that the file > pointer remains unmoved, and fix the _fileio module accordingly ? > +1 on having consistent, documented behavior (regardless of what that behavior is :) ). > - exceptions are not always specified, and even if most of them are > IOErrors, weirdly, in some cases, an OSError is raised instead (ie, if we > try to wrap a wrong file descriptor when instanciating a new FileIO). This > might lead to bad program crashes if some people don't "refuse the > temptation to guess" and only get prepared to catch IOErrors > I'd wager that you may also get a WindowsError in some cases, on Windows systems. Part of the reason for having several different, but similar, exceptions is that they may contain operating system specific error codes and the type of exception helps the programmer figure out how to interpret those codes. I'm not really clear on when IOError is preferred over OSError, but I know that WindowsError necessarily uses a completely different error numbering system. The careful programmer should catch EnvironmentError, which is the base class of all of these different kinds of related errors. +1 on documenting that the methods may raise an EnvironmentError. > - the doc sometimes says that when we receive an empty string from a read() > operation, without exceptions, it means the file is empty. However, with the > current implementation, if we call file.read(0), we simply receive "", even > though it doesn't mean that we're at EOF. Shouldn't we avoid this (rare, I > admit) ambiguity on the return value, by preventing read(0) ? Or at least, > note in the doc that (we receive an empty string) <-> (the file is at EOF OR > we called read with 0 as parameter) ? > Some programs may rely on read(0) and the behavior matches the behavior of POSIX, so I'm -1 on changing the behavior. +1 on documenting the exception to the rule. > Are there some arguments that I don't know, which lead to this or that > particular implementation choice ? > The original I/O PEP and some of the code was put together during a sprint at PyCon 2007. Our primary goal was to get a decent first cut of the new I/O system put together quickly. For nitty-gritty details and corner-cases like these, there's a good chance that the current undocumented behavior is simply an accident of implementation. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] POSIX [Fuzziness in io module specs]
On Sat, Sep 19, 2009 at 2:46 AM, Pascal Chambon wrote:
> This reimplementation is actually necessary to get file locking, because
> advanced win32 operations only work on real file handles, not the handles
> that are underlying the C API layer. Furthermore, some interesting features
> (like O_EXCL | O_CREAT) are not possible with the current io
> implementations. So well, reimplementation required ^^
>
> Concerning exceptions, which one is raised is not so important to me, as
> long as it's well documented and not tricky (eg. WindowsErrors are OK to me,
> because they subclass OSError, so most cross-platform programs wont even
> have to know about them).
>
If you use real Windows file handles (instead of the POSIX-ish Windows API),
won't you need to return WindowsErrors?
> I had the feeling that IOErrors were for operations on file streams
> (opening, writing/reading, closing...), whereas OSErrors were for
> manipulations on filesystems (renaming, linking, stating...) and processes.
>
If that were documented and a firm rule, that would certainly be great.
It's not too hard to find counterexamples in the current codebase. Also,
I'm not sure how one could avoid needing to raise WindowsError in some
cases.
Maybe someone with more knowledge of the history of IOError vs. OSError
could chime in.
Python 2.6:
>>> os.write(f.fileno(), 'blah')
Traceback (most recent call last):
File "", line 1, in
OSError: [Errno 9] Bad file descriptor
>>> f.write('blah')
Traceback (most recent call last):
File "", line 1, in
IOError: [Errno 9] Bad file descriptor
--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] POSIX [Fuzziness in io module specs]
On Sat, Sep 19, 2009 at 5:31 AM, Pascal Chambon wrote: > Actually, since Windows Error Codes concern any possible error (IO, file > permissions, memory problems...), I thought the best would be to convert > them to the most appropriate python standard exception, only defaulting to > WindowsError (i.e, OSError's hierarchy) when no other exception type > matches. So at the moment, I use a decorator to automatically convert all > errors on stream operations into IOErrors. Error codes are not the same as > unix ones indeed, but I don't know if it's really important (imo, most > people just want to know if the operation was successful, I don't know if > many developers scan error codes to act accordingly). > I don't often need to check the error code at runtime but seeing the corresponding message is often critical for debugging. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fuzziness in io module specs - PEP update proposition
On Sun, Sep 20, 2009 at 4:48 AM, Pascal Chambon wrote: > *RawIOBase*.readinto(b: bytes) -> int > "bytes" are immutable. The signature is: *RawIOBase*.readinto(b: bytearray) -> int Your efforts in working on clarifying these important corner cases is appreciated. :-) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3144 review.
On Sat, Sep 26, 2009 at 2:07 PM, DrKJam wrote: > The current version of the PEP and reference implementation do not mention > or deal with IPv4 classful addressing (A, B, C, D and E). It would be good > to know if any of this (admittedly older yet no less important) > functionality is going to be supported. If the library is to concentrate > solely on classless addressing (i.e. CIDR) please can this be stated in > future revisions of the PEP. > Classful addressing was deprecated more than 15 years ago! Quoting RFC 4632: "With the full deployment of CIDR on the Internet, such scenarios are no longer operationally relevant." -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> "The plumage don't enter into it. It's stone dead." ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3144 review.
On Sat, Sep 26, 2009 at 4:57 PM, DrKJam wrote:
> 2009/9/26 Daniel Stutzbach
>
>> On Sat, Sep 26, 2009 at 2:07 PM, DrKJam wrote:
>>
>>> The current version of the PEP and reference implementation do not
>>> mention or deal with IPv4 classful addressing (A, B, C, D and E). It would
>>> be good to know if any of this (admittedly older yet no less important)
>>> functionality is going to be supported. If the library is to concentrate
>>> solely on classless addressing (i.e. CIDR) please can this be stated in
>>> future revisions of the PEP.
>>>
>>
>> Classful addressing was deprecated more than 15 years ago!
>>
>> Quoting RFC 4632: "With the full deployment of CIDR on the Internet, such
>> scenarios are no longer operationally relevant."
>
>
> Interesting as evidence of classful IPv4 behaviour seems fairly pervasive
> in current IP stacks and software that supports IPv4 addressing (e.g.
> PostgreSQL inet and cidr data types).
>
PostgreSQL's inet type stores a host address and optionally a network mask.
It's cidr type stores a network address and mask. Neither of them are
classful.
> Here's an excerpt from the 'ifconfig' man page (taken from an Ubuntu 9.04
> install) :-
> netmask addr
>
> Set the IP network mask for this interface. This value defaults to the
> usual class A, B or C network mask (as derived from the interface IP
> address). but it can be set to any value.
>
Since the network mask can be set to any legal value, that means it's stored
as CIDR. It falls back on the classful network mask as a default.
Undoubtedly that was useful when ifconfig was updated for CIDR. There were
still many classful networks and it needed to retain backward
compatibility. After all, ifconfig is around 25 years old, and CIDR is only
16.
Today, all IP networks are classless (CIDR), and a system administrator
would have to do extra work to figure out if the default value happened to
be just right for the network they're setting up.
> The point being that you can't always assume /32 implicitly for all use
> cases.
>
Certainly, which is why the user can explicitly set the netmask.
> Here is how IP addresses without an explicit prefix or netmask are
> currently handled :-
>
> >>> import ipaddr
> >>> ipaddr.IPv4Network('10.0.0.1')
> IPv4Network('10.0.0.1/32')
>
> It may not be something we want to support (you could always force the end
> user to specify a prefix or netmask explicitly). This is fair enough, but
> let's indicate that it's a considered choice. Somewhere in the PEP text is
> might be a good place for treatment of this topic.
>
+1 on forcing the user to specify a netmask explicitly and documenting it.
--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3144 review.
On Mon, Sep 28, 2009 at 7:24 AM, Nick Coghlan wrote:
> I should note that I've softened my position slightly from what I posted
> yesterday. I could live with the following compromise:
>
>>>> x = IPv4Network('192.168.1.1/24')
>>>> y = IPv4Network('192.168.1.0/24')
> >>> x == y # Equality is the part I really want to see changed
>True
> >>> x.ip
>IPv4Address('192.168.1.1')
>>>> y.ip
> IPv4Address('192.168.1.0')
>
With those semantics, IPv4Network objects with distinct IP addresses (but
the same network) could no longer be stored in a dictionary or set. IMO, it
is a little counter-intuitive for objects to compare equal yet have
different properties. I don't think this is a good compromise.
--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3144 review.
On Mon, Sep 28, 2009 at 9:35 AM, R. David Murray wrote: > A little googling produced this hit: > > > http://tools.ietf.org/wg/mip4/draft-ietf-mip4-nemo-v4-base/draft-ietf-mip4-nemo-v4-base-11-from-10.diff.txt > > which while not a standard itself clearly believes that non-contiguous > netmasks are valid, and is dated 2008. Earlier RFCs (950 and 1219) > give them as valid but discouraged. > That's a draft for RFC 5177. In the RFC, all references to non-contiguous subnets have been removed. http://tools.ietf.org/html/rfc5177 -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3144 review.
On Mon, Sep 28, 2009 at 4:29 PM, "Martin v. Löwis" wrote: > I guess a close case would be rational numbers: clearly, 3÷2 == 6÷4; > would a Python library still remember (and repr) the original numerator > and denominator? > No need for a hypothetical, rational numbers were added in Python 2.6: Python 2.6.2 (r262:71600, Apr 15 2009, 07:20:39) [GCC 3.4.4 (cygming special, gdc 0.12, using dmd 0.125)] on cygwin Type "help", "copyright", "credits" or "license" for more information. >>> import fractions >>> fractions.Fraction(6,4) Fraction(3, 2) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3144 review.
.On Wed, Sep 30, 2009 at 3:33 PM, R. David Murray wrote: >1) do not add another class, just pass around (IPvXAddress, >IPVXNetwork) tuples when the address needs to be retained (or write >your own tailored trivial class, like I did in my example). > I've been puzzled by objections to forcing the user to decide what metadata to store. Every time I've ever needed to store an IP address or a network, I've always need to store a bunch of other affiliated data with it. E.g.: class GnutellaPeer: # IP address, port, timestamp, and dozen of other fields class NetworkInterface: # IP address, subnet mask, name, MAC address, etc. class Route: # interface, IP address, subnet mask, gateway address, route priority Is it such a burden for the programmer to store one extra field in the class they will inevitably write? For that matter, I do not see the advantage to an IPNetwork class that saves the programmer from having to separately store the IP address and subnet mask. Maybe I am biased by my background: I learned network programming in C, and I think of an IP address as little more than an integer type with a special string representation. People have voiced support for the IPNetwork class and for the occasional utility of an .ip field. I assume they have good use cases. It would be nice if the use cases were collected into the PEP, in a clear and articulate way. Preferably by someone other than ipaddr author, for whom the use cases are understandably a bit too obvious to explain at length with exasperation. It aught to be easy to justify the functionality of the library, if the use cases are clearly articulated. The current PEP begins by noting that many other IP address libraries are available, but doesn't describe what makes ipaddr unique or superior other than a claim to being lightweight. After downloading several of the other IP address libraries (http://bit.ly/483Yw4), ipaddr appears to be the second largest, after the huge netaddr package.<http://packages.python.org/netaddr/> I worry that this discussion has focused too much on the details of ipaddr (and the false dichotomy of "ipaddr versus nothing"), without properly tackling the question of "What use-cases for IP addresses are sufficiently universal* that they belong in the standard library?" -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Package install failures in 2.6.3
On Mon, Oct 5, 2009 at 9:32 AM, Barry Warsaw wrote: > If, as I hope, the answer to that is "yes", then I strongly support > releasing a fixed setuptools instead of reverting the change to Python. > How do your propose to get the author of setuptools to release a new version? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Weak dict iterators are fragile
On Sun, Oct 11, 2009 at 10:50 AM, Antoine Pitrou wrote:
> In py3k, the weak dict methods keys(), values() and items() have been
> changed to return iterators (they returned lists in 2.x).
> However, it turns out that it makes these methods quite fragile, because
> a GC collection can occur whenever during iterating, destroy one of the
> weakref'ed objects, and trigger a resizing of the underlying dict, which
> in turn raises an exception ("RuntimeError: dictionary changed size
> during iteration").
>
Ouch!
The iterator from __iter__ is also affected.
1. Add the safe methods listkeys(), listitems(), listvalues() which would
> behave as the keys(), etc. methods from 2.x
>
> 2. Make it so that keys(), items(), values() atomically build a list of
> items internally, which makes them more costly for large weak dicts, but
> robust.
>
-1 on 1.
+0 on 2.
It'd be nice if we could postpone the resize if there are active iterators,
but I don't think there's a clean way to track the iterators.
--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Better module shutdown procedure
On Wed, Oct 14, 2009 at 3:16 PM, Neil Schemenauer wrote: > The current shutdown code in pythonrun.c zaps module globals by > setting them to None (an attempt to break reference cycles). That > causes problems since __del__ methods can try to use the globals > after they have been set to None. > > The procedure implemented by http://bugs.python.org/issue812369 > seems to be a better idea. References to modules are replaced by > weak references and the GC is allowed to cleanup reference cycles. > Based on the description, it still resorts to zapping module globals by setting them to None. It zaps them to weakrefs first, which means that globals are more likely to be valid during __del__, but it still cannot make any guarantees and referencing globals from __del__ is still a bad idea. Is that a correct synopsis? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Better module shutdown procedure
On Wed, Oct 14, 2009 at 4:42 PM, Neil Schemenauer wrote: > Yes, it does still resort to setting globals to None. However, the > weakref step makes it much more likely that __del__ methods run > before that happens. After this change, referencing global variables > from __del__ methods is okay. > The first and last sentences seem like a contradiction to me. If I cannot guarantee that globals will be valid when __del__ is executed, then I must not reference globals from __del__. I think I'm missing something here. Is there some way the programmer can determine that referencing a global variable in __del__ will be 100% safe? (not just "likely") -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Better module shutdown procedure
On Wed, Oct 14, 2009 at 6:05 PM, Neil Schemenauer wrote: >def __del__(): >print sys.version > > the global variable reference to 'sys' is not a reference on the GC > referencing counting sense. IOW, it does not result in a a Py_INCREF > while the function is not being executed and therefore should be > safe after the proposed change. Currently, it could result in 'None' > being printed. > Currently it throws an exception since "sys" is None. :-) Here is my understanding of the proposed procedure: 1. Replace modules in sys.modules with weakrefs 2. Run the garbage collector 3. Replace globals in any remaining modules with None 4. Run the garbage collector Is it possible for a __del__ method to be called in step 4 or not? I am still unclear on this point. :-) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] SIGCHECK() in longobject.c
On Sun, Oct 18, 2009 at 3:01 PM, Antoine Pitrou wrote: > Can we remove this check, or are there people doing million-digits > calculations > they want to interrupt using Control-C ? > I sometimes do million-digits calculations that I want to interrupt using Control-C. (particularly when I didn't *intend* to do a million-digits calculation... ;) ) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] SIGCHECK() in longobject.c
On Sun, Oct 18, 2009 at 5:46 PM, Antoine Pitrou wrote: > Daniel Stutzbach stutzbachenterprises.com> writes: > > I sometimes do million-digits calculations that I want to interrupt using > Control-C.(particularly when I didn't *intend* to do a million-digits > calculation... ;) )-- > > Sure, but it's no different than doing, e.g.: >list(range(1)).sort() > That's a good point, although I can't recall the last time I accidently created a painfully large list. I can recall the last time I started a painfully large integer computation. Being able to stop the interpretter with Control-C instead of kill -9 is a minor convenience, though. I could live without it. (Although I can't speak for everyone, of course) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add const to python API - issue 6952
On Tue, Oct 20, 2009 at 4:03 PM, Barry Scott wrote:
> Checking my patch I have two functions that need to
> have output params changed to const to avoid casting.
>
>PyOS_strtoul- ptr
>PyLong_FromString - pend
>
This is a no-win situation. If the string is const in the caller, they
currently need to cast it. If you make the change, then if string is not
const in the caller then they will need to cast it. I've provided a short
example below and marked the lines that generate "an incompatible pointer
type" warning with gcc.
I suggest following POSIX's lead and omitted the const in these cases.
void test_const(const char **foo);
void test_noconst(char **foo);
int main(void) {
char *noconst_var;
const char *const_var;
test_const(&noconst_var); // generates a warning
test_const(&const_var);
test_noconst(&noconst_var);
test_noconst(&const_var); // generates a warning
return 0;
}
--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reworking the GIL
On Mon, Oct 26, 2009 at 10:58 AM, Antoine Pitrou wrote: > Er, I prefer to keep things simple. If you have lots of I/O you should > probably > use an event loop rather than separate threads. > On Windows, sometimes using a single-threaded event loop is sometimes impossible. WaitForMultipleObjects(), which is the Windows equivalent to select() or poll(), can handle a maximum of only 64 objects. Do we really need priority requests at all? They seem counter to your desire for simplicity and allowing the operating system's scheduler to do its work. That said, if a thread's time budget is merely paused during I/O rather than reset, then a thread making frequent (but short) I/O requests cannot starve the system. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] MSDN subscribers: Using Visual Studio?
On Thu, Oct 29, 2009 at 6:57 AM, Antoine Pitrou wrote: > If it's just a matter of building and testing it you don't need any MSDN > subscription, you just need Visual Studio Express which is free (as in > free beer...). I don't know if it allows you to build an installer > though. > It does. If I recall correctly, in addition to Visual Studio Express, I also needed the Windows SDK (which is also free as in beer). -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reworking the GIL
On Mon, Nov 2, 2009 at 11:27 AM, "Martin v. Löwis" wrote: > Hmm. This creates a busy wait loop; if you add larger sleep values, > then it loses accuracy. > I thought that at first, too, but then I checked the documentation for Sleep(0): "A value of zero causes the thread to relinquish the remainder of its time slice to any other thread of equal priority that is ready to run." (this is not to say that I think the solution with Sleep is worthwhile, though...) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
On Tue, Nov 3, 2009 at 3:55 AM, David Cournapeau wrote: > - are two branches are necessary or not ? If two branches are > necessary, I think we simply do not have the resources at the moment. > - how to maintain a compatible C API across 2.x and 3.x > - is it practically possible to support and maintain numpy from 2.4 > to 3.x ? For example, I don't think the python 2.6 py3k warnings are > very useful when you need to maintain compatibility with 2.4 and 2.5. > I've already ported some of my Python extension modules to Python 3. Here's how I would answer your questions based on my experience. Writing C code that compiles with Python 2.4 through 3.1 is pretty easy. Python's C API hasn't changed much and you can use #ifdef and #define for any bits that must be version-specific. It's pretty easy to make Python source that works under 2.6 and 3.x. It's basically impossible to make Python source that works under 2.4/2.5 and 3.x. You may be able to write code that works under 2.4/2.5 and works cleanly with 2to3 to produce 3.x code. I haven't tried that route, though in theory it should work. All you really need is syntax compatibility. For the rest, you can check sys.version_info. In a nutshell, I don't think you need two branches to support an extension module on Python 2 and Python 3. YMMV. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Retrieve an arbitrary element from a set withoutremoving it
On Tue, Nov 3, 2009 at 4:46 PM, Steven D'Aprano wrote: > def pick_two_cards(hand): >assert isinstance(hand, (set, frozenset)) >assert len(hand) == 5 >return (hand.pick(), hand.pick()) > Even if pick() chose random, you still might end up picking the same card twice. Is that really what you intended? FWIW, I've been working on an extension module that supplies a "sortedset" type [1]. In most ways, it's similar to a set except it's indexable like a list. The items are kept in sorted order, so index 0 is always the lowest item, index 1 is the next-to-lowest, etc. Because they're indexable, it's easy and efficient to retrieve random elements using the standard library's "random" module. With the sortedset type, that function would become: def pick_two_cards(hand): assert isinstance(hand, (set, frozenset)) assert len(hand) == 5 return random.choice(hand), random.choice(hand) or if you want to avoid duplicates: return random.sample(hand, 2) Would something like that fit your needs? [1] It's already implemented, along with sortedlist, weaksortedlist, and weaksortedset types. I'm just putting them through the paces in my own products before releasing them. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Retrieve an arbitrary element from asetwithoutremoving it
On Mon, Nov 9, 2009 at 2:42 AM, "Martin v. Löwis" wrote: > Interesting. Something goes wrong, it seems: if items get removed over > and over again, I think the set should shrink (not sure whether it > actually does). Then, I think you should end up with an amortized O(1) > for selecting an element (assuming that the underlying hashes don't > collide). > I'm not sure if Python's implementation shrinks or not, but even if it did shrink, it would still be amortized O(n). Imagine a completely full hash table that currently contains n elements in n slots (I know this cannot happen in Python's implementation but it's useful for illustrative purposes). Assume it will shrink when it gets down to n/2 elements. Here is my pathological algorithm again, for reference purposes: while s: for i in s: break # Imagine a complex algorithm here that depends on i still being in s s.remove(i) We repeatedly search through the slots sequentially and remove the first element we find. The first removal requires checking 1 slot, the second removal requires checking 2 slots, the third removal requires checking 3 slots, and so forth. The hash table will shrink after the n/2-th removal, when we have checked 1 + 2 + 3 + ... + n/2 = O(n**2) slots for n/2 removals (or amortized O(n) per removal). It's too late for shrinking to save us; we've already performed too much work. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Retrieve an arbitrary element from asetwithoutremoving it
On Mon, Nov 9, 2009 at 3:50 PM, "Martin v. Löwis" wrote: > I think for regular removal, the same logic should not apply: if a > series of removals is performed, then further (non-pop) removals > see increasing costs, as do regular lookups. So I think that a removal > should trigger shrinking (with appropriate thresholds) unless it's a > .pop. > Minor technicality, but it's the next() operation of the iterator that has increasing cost as the set/dict becomes sparse. Removals are always O(1). The removal uses the hash to find the item quickly. The iterator has to scan through the table for non-empty entries. (the above assumes a good hash function with few collisions, of course) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Too many Python accounts
On Sun, Nov 15, 2009 at 2:44 PM, "Martin v. Löwis" wrote: > But then, users can easily create as many fake accounts as they want to. > Why not do something more robust, then? For example, when a user enters an OpenID that hasn't been seen by PyPi before, make them enter a CAPTCHA. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] recursive closures - reference leak
2009/12/8 Maciej Fijalkowski > Note that disabling gc > does not mean that you will not have unpredictable pauses. Consider > for example that if you loose a reference to a very long chain of > objects, you can have arbitrarily many frees being called before > anything else can happen. > That strikes me as a *predictable* long pause. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] patch to make list.pop(0) work in O(1) time
On Mon, Jan 25, 2010 at 1:22 PM, Steve Howell wrote: > I haven't completely worked out the best strategy to eventually release the > memory taken up by the pointers of the unreleased elements, but the worst > case scenario is that the unused memory only gets wasted until the time that > the list itself gets garbage collected. > FWIW, for a long-running FIFO queue, it's critical to release some of the memory along the way, otherwise the amount of wasted memory is unbounded. Good luck :) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] patch to make list.pop(0) work in O(1) time
On Tue, Jan 26, 2010 at 1:17 AM, Steve Howell wrote: > Ok, I fixed the obvious segfaults, and I am now able to pass all the tests > on my debug build. A new diff is attached. > Just to be clear, when you say "all tests" do you mean "all of the list tests" or "the full Python test suite"? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] patch to make list.pop(0) work in O(1) time
On Tue, Jan 26, 2010 at 3:32 PM, Steve Howell wrote: > Even most tiny-ish lists are wasting memory, though, according to this > sequence: > > 4, 8, 16, 25, ... > Several operations will allocate a list of exactly the right size, wasting no memory. In particular, a fixed-size list will always be exactly the right size. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] patch to make list.pop(0) work in O(1) time
On Wed, Jan 27, 2010 at 7:13 AM, Steve Howell wrote: > My concept of Python lists is that they should have at least the same > performance characteristics as an ordinary to-do list that you make with > pencil, paper, and an eraser. > > When you complete the first task on your to-do list, you can just erase it; > no need to recopy the whole list. > I don't think your analogy works, unless you recopy your to-do lists whenever you complete a task in the middle of the list. ;-) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] patch to make list.pop(0) work in O(1) time
On Wed, Jan 27, 2010 at 9:55 AM, Steve Howell wrote: > Fair enough, but that's still wasteful of memory, keeping around a bunch of > None elements because you can't inexpensively delete them. > Even if there are many references to it, there is only one None element. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] O(1) random access to deque? (Re: patch to make list.pop(0) work in O(1) time)
On Wed, Jan 27, 2010 at 3:50 PM, Nick Coghlan wrote: > I'm actually wondering if you could apply some of the implementation > strategies discussed here to grant O(1) random access to arbitrary > elements of a deque. > > I haven't looked at the deque code in a long time, but I believe the > memory structure is already larger than that for a basic list. > The memory structure is a linked list of blocks, where each block can hold several elements. As a linked list, the current structure would have O(n) random access. A few years back, I had proposed an alternative way of implementing deque, that would allow O(1) random access: http://mail.python.org/pipermail/python-dev/2007-November/075242.html It's essentially identical to Steve's proposal for list, except that I made the array circular, so element i is at position items[(start + i) % len]. That way it doesn't have to regularly allocate and deallocate memory for an approximately-fixed-length FIFO queue (which Steve's list will need to do). Raymond's objections are here: http://mail.python.org/pipermail/python-dev/2007-November/075244.html -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] patch to make list.pop(0) work in O(1) time
On Wed, Jan 27, 2010 at 3:49 PM, Raymond Hettinger < [email protected]> wrote: > Also, am not sure if this affects psyco or the other > implementations such as Jython which may implement > lists in terms of existing Java containers. > Or Unladen Swallow. I'm -1 on mucking with any of the fundamental data structures until the Unladen Swallow patch lands (assuming it lands). -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3147: PYC Repository Directories
On Sat, Jan 30, 2010 at 8:21 PM, Vitor Bosshard wrote: > Putting the files into a separate dir also makes it much harder to > work with external tools; e.g. VCSes already ignore .pyc and .pyo > files, but not unknown directories. > Can't a VCS be configured to ignore a .pyr directory just as easily as it can be configured to ignore a .pyc file? ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mercurial repository for Python benchmarks
On Sun, Feb 21, 2010 at 2:28 PM, Collin Winter wrote: > Would it be possible for us to get a Mercurial repository on > python.org for the Unladen Swallow benchmarks? Maciej and I would like > to move the benchmark suite out of Unladen Swallow and into > python.org, where all implementations can share it and contribute to > it. PyPy has been adding some benchmarks to their copy of the Unladen > benchmarks, and we'd like to have as well, and Mercurial seems to be > an ideal solution to this. > If and when you have a benchmark repository set up, could you announce it via a reply to this thread? I'd like to check it out. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Platform extension for distutils on other interpreters than CPython
On Tue, Feb 23, 2010 at 12:50 PM, Maciej Fijalkowski wrote: > I would like to have a feature on platform module (or sys or > somewhere) that can tell distutils or distutils2 that this platform > (be it PyPy or Jython) is not able to compile any C module. The > purpose of this is to make distutils bail out in more reasonable > manner than a compilation error in case this module is not going to > work on anything but CPython. > Also, it would be nice if the package could tell distutils that the compilation is option, in order to support modules where the C version simply replaces functions for speed. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Caching function pointers in type objects
In CPython, is it safe to cache function pointers that are in type objects? For example, if I know that some_type->tp_richcompare is non-NULL, and I call it (which may execute arbitrary user code), can I assume that some_type->tp_richcompare is still non-NULL? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Caching function pointers in type objects
On Tue, Mar 2, 2010 at 9:06 PM, Reid Kleckner wrote: > I don't think this will help you solve your problem, but one thing > we've done in unladen swallow is to hack PyType_Modified to invalidate > our own descriptor caches. We may eventually want to extend that into > a callback interface, but it probably will never be considered an API > that outside code should depend on. > Thanks Reid and Benjamin for the information. I think I see a way to dramatically speed up PyObject_RichCompareBool when comparing immutable, built-in, non-container objects (int, float, str, etc.). It would speed up list.sort when the key is one of those types, as well as most operations on the ubiquitous dictionary with str keys. Is that a worthwhile avenue to pursue, or is it likely to be redundant with Unladen Swallow's optimizations? If I can find time to pursue it, would it be best for me to implement it as a patch to Unladen Swallow, CPython trunk, or CPython py3k? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Caching function pointers in type objects
On Wed, Mar 3, 2010 at 3:29 PM, Benjamin Peterson wrote: > 2010/3/3 Daniel Stutzbach : > > I think I see a way to dramatically speed up PyObject_RichCompareBool > when > > comparing immutable, built-in, non-container objects (int, float, str, > > etc.). It would speed up list.sort when the key is one of those types, > as > > well as most operations on the ubiquitous dictionary with str keys. > (correcting myself) I just noticed that CPython already optimizes dictionaries with str-only keys and skips PyObject_RichCompareBool, so no speed up there. I think it would be redundant with optimization I have in mind, so it could perhaps be taken out which would simplify the dict code a bit and save a pointer in the dict structure. > Perhaps you could explain what exactly you want to do. :) That would > help us make a judgment. > It'd actually be a pretty small patch, so I think I should just explain it by actually writing it. ;-) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Caching function pointers in type objects
On Wed, Mar 3, 2010 at 4:34 PM, Collin Winter wrote: > I would recommend patching py3k, with a backport to trunk. > After thinking it over, I'm inclined to patch trunk, so I can run the Unladen Swallow macro-benchmarks, then forward-port to py3k. I'm correct in understanding that it will be a while before the Unladen Swallow benchmarks can support Python 3, right? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [PEP 3148] futures - execute computations asynchronously
On Fri, Mar 5, 2010 at 12:03 AM, Brian Quinlan wrote: > import futures > +1 on the idea, -1 on the name. It's too similar to "from __future__ import ...". Also, the PEP should probably link to the discussions on stdlib-sig? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [PEP 3148] futures - execute computations asynchronously
On Fri, Mar 5, 2010 at 11:03 AM, Jesse Noller wrote: > http://java.sun.com/javase/6/docs/api/java/util/concurrent/Future.html > According to that link, Java has a module named "Concurrent" with an interface named "Future". You're proposing a module named "Futures" with a class named "Future". Why not name your module "concurrent"? That would eliminate the confusion with "from __future__". I don't see a problem with keeping the class name. Plus, a "concurrent" module might be useful for things other than Futures, in the future. ;-) Just my 0.02 cents, -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [PEP 3148] futures - execute computations asynchronously
On Sat, Mar 6, 2010 at 6:43 AM, Nick Coghlan wrote: > You may want to consider providing global thread and process executors > in the futures module itself. Code which just wants to say "do this in > the background" without having to manage the lifecycle of its own > executor instance is then free to do so. +1 -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [PEP 3148] futures - execute computations asynchronously
On Sat, Mar 6, 2010 at 5:38 PM, Michael Foord wrote: > On 06/03/2010 23:37, Greg Ewing wrote: > >> I've been thinking for a while that it would be a big help >> if there were one, standardised module in the stdlib for >> handling async events, and all the other gui toolkits >> etc. were made to use it. >> >> Wouldn't it have to be the Tcl event loop then? > I image he means a standardized Abstract Base Class, which each GUI toolkit would subclass. That's more or less how Twisted's "reactors" work. That way non-GUI async code can just use the ABC and not worry about what event loop is running underneath (be it TCL, GTK, or just poll()). -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] "Fixing" the new GIL
On Sat, Mar 13, 2010 at 3:46 PM, Antoine Pitrou wrote: > - the second mechanism dynamically computes the "interactiveness" of a > thread and allows interactive threads to steal the GIL quickly. In > this approach, IO methods don't have to be modified at all. > I like the second approach as well, assuming "interactiveness" can be computed cheaply. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] WeakSet in Python 2.7
On Mon, Mar 29, 2010 at 12:16 PM, Michael Foord wrote: > Python 3 includes a WeakSet implementation. Any objections to me > backporting this to 2.7? > > http://docs.python.org/py3k/library/weakref.html#weakref.WeakSet > Backporting WeakSet would also make it possible to backport the fix for this reference leak: http://bugs.python.org/issue2521 -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] WeakSet in Python 2.7
On Mon, Mar 29, 2010 at 2:21 PM, Michael Foord wrote: > It should be possible to fix it with a WeakKeyDictionary instead of > WeakSet. > True. I should have said "Backporting WeakSet would make it *easier* to backport the fix ..." :-) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] stdlib socket usage and "keepalive"
On Mon, Apr 12, 2010 at 5:34 PM, Jesus Cea wrote: > The problem is: linux doesn't uses KEEPALIVE by default. > If you believe the problem is with the Linux kernel, perhaps you should take up your case on a more appropriate mailing list? Python's socket module is a fairly low-level module, as it's just a thin wrapper around the corresponding operating system calls. Anyone using it has to be prepared to deal with a certain amount of exposed operating system details. If you want to use TCP KEEPALIVE on Linux, then just call: my_socket_object.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1) Most non-trivial applications use select() or poll() to avoid blocking calls and do their own timeout-checking at the application layer, so they don't need KEEPALIVE. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tuning Python dicts
On Sat, Apr 10, 2010 at 1:06 PM, Reid Kleckner wrote: > Looking at dictnotes.txt, I can see that people have experimented with > taking advantage of cache locality. I was wondering what benchmarks > were used to glean these lessons before I write my own. Python > obviously has very particular workloads that need to be modeled > appropriately, such as namespaces and **kwargs dicts. > I don't know what benchmarks were used to write dictnotes.txt, but moving forward I would recommend implementing your changes on trunk (i.e., Python 2.x) and running the Unladen Swallow Benchmarks, which you can get from the link below: http://code.google.com/p/unladen-swallow/wiki/Benchmarks They are macro-benchmarks on real applications. You will probably also want to write some micro-benchmarks of your own so that you can pinpoint any bottlenecks in your code and determine where you are ahead of the current dict implementation and where you are behind. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Very Strange Argument Handling Behavior
On Fri, Apr 16, 2010 at 4:11 PM, Raymond Hettinger < [email protected]> wrote: > ISTM that making it illegal costs cycles with giving any real benefit. > It is reasonably common to accept **kwds and then pass it down > to another function. Do we want to validate the keys of every > kwds dict on every call? Why do we even care? > IIRC, there's a performance hack in dictobject.c that keeps track of whether all of the keys are strings or not. The hack is designed so that lookup operations can call the string compare/hash functions directly if possible, rather than going through the slower PyObject_ functions. Consequently, validating **kwds should be cheap. I don't know if the the current validating of **kwds with Python functions already leverages that hack or not. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Very Strange Argument Handling Behavior
On Fri, Apr 16, 2010 at 4:51 PM, Benjamin Peterson wrote: > 2010/4/16 Daniel Stutzbach : > > IIRC, there's a performance hack in dictobject.c that keeps track of > whether > > all of the keys are strings or not. The hack is designed so that lookup > > operations can call the string compare/hash functions directly if > possible, > > rather than going through the slower PyObject_ functions. > > That won't work. You could put non-string keys in a dictionary and > remove them, but the dictionary would still be in the less optimized > state. > It would be enough to make the common case fast (1 pointer comparison). The error case would need to check all of the keys, true, but that's not really a performance concern. Unless you're saying you often create a dictionary, add non-string keys, remove the non-string keys, then pass it as a **kwds? ;-) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] http://bugs.python.org/ is down
On Sat, Apr 17, 2010 at 12:17 PM, Victor Stinner < [email protected]> wrote: > http://bugs.python.org/ displays "Service Temporarily Unavailable". Is it > normal? > It's working fine for me. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] bug tracker permissions request
May I have enhanced permissions on the bug tracker, so that I can perform the following tasks? - Assign issues to myself that I plan to write a patch for - Update the Stage to "patch review" after writing a patch - Occasional bug triage My login name on the tracker is "stutzbach". I only find the time to produce patches once in awhile, but when I have the time I usually produce more than one. Assigning bugs to myself will increase my motivation to write patches, as I will feel that I've made a commitment to fixing them. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bug tracker permissions request
On Tue, Apr 27, 2010 at 10:14 AM, R. David Murray wrote: > Done. I agree with Brian, Daniel has been making valuable > contributions for quite some time now. I/we will keep an eye on > his triage, of course. > Thanks. Is there a document that describes the meaning of all of the different fields in the bug tracker? I've read http://www.python.org/dev/workflow/, but it doesn't cover everything. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bug tracker permissions request
On Tue, Apr 27, 2010 at 11:16 AM, R. David Murray wrote: > I think it is on Brett's list to update that doc, but maybe we should help > him out :). Can you list what's missing? We should fill in the gaps. > Sure, here's what I've noticed: The page doesn't document the Resolution or Status fields. For the Keywords field, the page only documents the "easy" keyword. Also, some of the headings in the page are enclosed in square brackets, while others are not. It's not clear to me what the brackets are intended to designate. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] versioned .so files for Python 3.2
On Thu, Jun 24, 2010 at 10:50 AM, Barry Warsaw wrote: > The idea is to put the Python version number in the shared library file > name, > and extend .so lookup to find these extended file names. So for example, > we'd > see foo.3.2.so instead, and Python would know how to dynload both that and > the > traditional foo.so file too (for backward compatibility). > What use case does this address? PEP 3147 addresses the fact that the user may have different versions of Python installed and each wants to write a .pyc file when loading a module. .so files are not generated simply by running the Python interpreter, ergo .so files are not an issue for that use case. If you want to make it so a system can install a package in just one location to be used by multiple Python installations, then the version number isn't enough. You also need to distinguish debug builds, profiling builds, Unicode width (see issue8654), and probably several other ./configure options. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration)
On Thu, Jul 1, 2010 at 7:52 AM, anatoly techtonik wrote: > 4. Even if I make patch in my Mercurial clone - you still can't pull > it and I have to attach it to tracker. No gain. > Was there ever any discussion about hosting the central repository on a site such as bitbucket or github? I tried searching the python-dev archives but was unable to find much. Anyway... assuming there's a at least a clone of the central repository on one of those sites, you can fork it and work on your patch there. A core developer can then pull your patch to their local repository, tweak it as needed, then commit it to the central repository. > I would put accent on keeping mirror of Subversion as easy way to > contribute for those who are not yet ready for DVCS. Subversion also > provides greater interoperability. Assuming that any modern DVCS tool > may act as Subversion client, we will gain more contributors if we > won't try to force people use Python and Mercurial. > The hg-git tool allows Mercurial and git to interoperate, so that's not as much of an issue as it once was. It's geared toward using a Mercurial client to talk to a git server, but I'm told it can work the other way around with a bit of work. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mercurial migration readiness
On Sat, Jul 3, 2010 at 11:36 AM, Tres Seaver wrote: > - - Create a "pristine" clone of the trunk (one where I never commit any > changes): > > $ cd $python_repo > $ hg clone http://code.python.org/hg/trunk/ pytrunk-upstream > > - - Create a local clone from that repository: > > $ hg clone pytrunk-upstream pytrunk-work > $ ./configure && make > My question is basically the same as Terry Reedy's, but I'm going to phrase it a bit differently: This is perhaps a naive question, but why do you create a second local clone instead of just creating a branch? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] versioned .so files for Python 3.2
On Fri, Jul 16, 2010 at 5:40 AM, Matthias Klose wrote: > 2) As PEP 3147 defines a non-configurable name for .pyc files, this PEP > should define a non-configurable way for the tag. The tag should > include all information which currently makes an extension ABI > incompatible: > - the python implementation (cpython, PyPy, ...) > - the python version (3,2, 3.3, ...) > - unicode configure option (--with-wide-unicode, 16 or 32) > - platform information (necessary?) I'm not sure it's that easy to enumerate all of the ways to end up with an incompatible ABI. There are quite a lot of configure options listed with "configure --help", and it's not always obvious if an option changes the ABI. On top of that, there are compilation flags that can change the ABI, as Kristján discovered in the following thread: http://mail.python.org/pipermail/python-dev/2010-June/100583.html On the flip side, a fully enumerated ABI signature could be used to identify (in)compatible binary eggs, which is basically impossible now. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cProfile and threads
2010/8/17 Kristján Valur Jónsson > These patches are the result of work that we have done in profiling > Stackless Python server applications at runtime, but they apply just as well > to C Python. > > The first patch makes _lsprof, the engine behind cProfile, multi-stack > aware. This allows the same cProfiler.Profile() instance to be active on > multiple python threads and still meaningful information is gathered. > +1 > The second patch allows to set the trace/profile function in python * > globally*, so that all threads are affected. This is essential if you > want to take a profililng snapshot of a running application. > +1 -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Request for commits and/or privileges
Several issues that I'm involved with (listed below) are ready for commit, as far as I can tell. They have a patch, and either a core developer has positively reviewed the patch or the patch is a straightforward implementation of a core developer's suggested approach. They are all bug-fixes or optimizations. May I have commit privileges so that I can commit these patches and future patches in a similar state? If the consensus is negative for any reason, I completely respect that decision and will continue to contribute patches just as I am now (but in that case, consider this my humble request for someone to commit these changes :) ). If positive, I would start by committing just one (after writing an appropriate NEWS entry) and soliciting feedback to make sure that I had done it right. http://bugs.python.org/issue8781 - 32-bit wchar_t doesn't need to be unsigned to be usable http://bugs.python.org/issue9214 - Most Set methods of KeysView and ItemsView do not work right http://bugs.python.org/issue8750 - Many of MutableSet's methods assume that the other parameter is not self http://bugs.python.org/issue5553 - Py_LOCAL_INLINE(type) doesn't actually inline except using MSC http://bugs.python.org/issue2521 - ABC caches should use weak refs http://bugs.python.org/issue808164 - socket.close() doesn't play well with __del__ Many more in the pipeline :-) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Request for commits and/or privileges
On Sat, Aug 21, 2010 at 12:41 PM, Raymond Hettinger wrote: > +1 from me too. > > When you start, olease do leave me as the primary maintainer > for issues relating to sets and set ABCs. I see myself as the primary maintainer of nothing. :-) I would not commit anything related to sets or set ABCs unless you have signed off on it in some way. Perhaps in time there will be some piece of Python that I've modified so heavily that I become ipso facto the primary maintainer, but I'm in no hurry. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Request for commits and/or privileges
On Sat, Aug 21, 2010 at 3:47 AM, "Martin v. Löwis" wrote: > Please send me your SSH key. Done. I have also subscribed to python-committers and python-checkins. I will add my interests to Misc/maintainers.rst. Are there any other initial start-up tasks I should perform? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Shared Folders Under Vista
On Sun, Aug 22, 2010 at 9:01 PM, Steve Holden wrote: > Folders that I create with Cygwin using mkdir appear to be shared - sort > of. They are denoted with the shared folder icon, and when I use Windows > Explorer to delete them I get the warning about the folder being shared. When you say "mkdir", what exactly do you mean? /usr/bin/mkdir? If so, what does this question have to do with Python? Python's os.mkdir? If so, which version of Python are you using? What version of Windows are you running? For what it's worth, I just tried both of the above on my XP/Cygwin system and did not see the shared folder icon in Explorer. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 status
On Tue, Aug 31, 2010 at 4:54 AM, M.-A. Lemburg wrote: > Is it possible to have multiple versions of the lib C loaded > on Windows ? > Yes, and it's a pretty common situation. The fopen() that I call within a DLL may not be the same fopen() called by another DLL. When writing a DLL for Windows, the API must be designed with the assumption that anything returned by the C library cannot be passed a different C library. For example, suppose I a expose a function in my DLL that allocates some memory, populates it with useful information, and returns a pointer. I must also supply a function to free the memory. I cannot ask the caller to simply call free(), because their free() may not be using the same heap as my malloc(). Likewise, a FILE * isn't safe to pass around, unless I can guarantee that the application really is one big happy family compiled against the same version of the C library. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] r84430 - in python/branches/py3k: Include/unicodeobject.h Objects/unicodeobject.c
On Thu, Sep 2, 2010 at 6:26 PM, Victor Stinner wrote: > But I didn't found any doc for other Py_UNICODE_str*() > functions in Doc/c-api/*.rst. > http://bugs.python.org/issue8649 - Py_UNICODE_* functions are undocumented -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] new LRU cache API in Py3.2
On Sat, Sep 4, 2010 at 3:28 AM, Stefan Behnel wrote: > What about adding an intermediate namespace called "cache", so that the new > operations are available like this: > I had been thinking that the lru_cache should be a class (with a dict-like interface), so it can be used explicitly and not just as a decorator. It could provide a wrap() method to be used as a decorator (or implement __call__ to keep the current semantics, but explicit is better than implicit) widget_cache = lru_cache() widget_cache[name] = widget @lru_cache().wrap def get_thingy(name): return something(name) # get_thingy.cache is an lru_cache instance print(get_thingy.cache.hits) I have been using a similar LRU cache class to store items retrieved from a database. In my case, a decorator-paradigm wouldn't work well because I only want to cache a few of the columns from a much larger query, plus there are multiple functions that want to talk to the cache. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Garbage announcement printed on interpreter shutdown
On Fri, Sep 10, 2010 at 3:32 PM, Georg Brandl wrote: > IMO this runs contrary to the decision we made when DeprecationWarnings > were > made silent by default: it spews messages not only at developers, but also > at > users, who don't need it and probably are going to be quite confused by it, > assuming it came from their console application (imagine Mercurial printing > this). > A non-empty gc.garbage often indicates that there is a bug in the program and that it is probably leaking memory [1]. That's a little different from a DeprecationWarning which doesn't indicate a bug; it just indicates that the program might not run correctly using a future version of Python. I think a better comparison would be with exceptions throw from a __del__, which (as far as I know) are still printed to the console. +1 on adding a way to enable/disable the feature. -1 on removing the feature -0 on making it disabled by default [1] I know that some large, long-running programs periodically check gc.garbage and carefully choose where to break cycles, but those are the exception and not the rule. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
