[Python-3000] gc, timestamping
(on python-dev) Kristján V. Jónsson wrote: > We have been using gc.get_objects() but it has several problems: > 1) ... results in a list so long that it often kills the system. In Py3K, should this also return an iterator, rather than a list? In other words, is the dict.items change something specific to dicts, or something that could happen with any large sequence? > 2) There is no way to frame certain operations and get just those > objects that were created during their execution. ... storing the id()s of > all objects >... the ambiguity of id() in the face of objects being created and destroyed. Changing id() to be something other than the address probably isn't practical, but with the elimination of old-style classes, everything should inherit from object. Would it make sense to add optional timestamping (or other alternative id-like attributes)? Right now, the best solution I see is to use a debug build and a C debugger to inspect the serial number that _PyObject_DebugMalloc sticks after its FORBIDDENBYTE barrier. -jJ ___ Python-3000 mailing list [email protected] http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] gc, timestamping
On 1/24/07, Jim Jewett <[EMAIL PROTECTED]> wrote: > (on python-dev) Kristján V. Jónsson wrote: > > > We have been using gc.get_objects() but it has several problems: > > 1) ... results in a list so long that it often kills the system. > > In Py3K, should this also return an iterator, rather than a list? > > In other words, is the dict.items change something specific to dicts, > or something that could happen with any large sequence? It should be considered on a case-by-case basis, depending on how the result is most commonly used, or perhaps what's the most useful result. I don't know if it makes sense given the GC implementation to do it in this case. (For example, I looked briefly into making os.listdir() an iterator, but decided against it because you'd have a file descriptor open while iterating, and the list is never truly long.) > > 2) There is no way to frame certain operations and get just those > > objects that were created during their execution. ... storing the id()s of > > all objects > >... the ambiguity of id() in the face of objects being created and destroyed. > > Changing id() to be something other than the address probably isn't > practical, but with the elimination of old-style classes, everything > should inherit from object. Would it make sense to add optional > timestamping (or other alternative id-like attributes)? > > Right now, the best solution I see is to use a debug build and a C > debugger to inspect the serial number that _PyObject_DebugMalloc > sticks after its FORBIDDENBYTE barrier. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-3000 mailing list [email protected] http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
[Python-3000] new pickle semantics/API
i'm having great trouble in RPyC with pickling object proxies.
several users have asked for this feature, but no matter how hard
i try to "bend the truth", pickle always complains. it uses
type(obj) for the dispatching, which "uncovers" the object
is actually a proxy, rather than a real object.
recap: RPyC uses local proxies that refer to objects of a
remote interpreter (another process/machine).
if you'd noticed, every RPC framework has its own serializer.
for example banna/jelly in twisted and bunch of other
XML serializers, and what not.
for RPyC i wrote yet another serializer, but for different purposes,
so it's not relevant for the issue at hand.
what i want is a standard serialization *API*. the idea is that
any framework could make use of this API, and that it would
be generic enough to eliminate copy_reg and other misfortunes.
this also means the built in types should be familiarized with
this API.
- - - - - - - -
for example, currently the builtin types don't support __reduce__,
and require pickle to use it's own internal registry. moreover,
__reduce__ is very pickle-specific (i.e., it takes the protocol number).
what i'm after is an API for "simplifying" complex objects into
simpler parts.
here's the API i'm suggesting:
def __getstate__(self):
# return a tuple of (type(self), obj), where obj is a simplified
# version of self
@classmethod
def __setstate__(cls, state):
# return an instance of cls, with the given state
well, you may already know these two, although their
semantics are different. but wait, there's more!
the idea is of having the following simple building blocks:
* integers (int/long)
* strings (str)
* arrays (tuples)
all picklable objects should be able to express themselves
as a collection of these building blocks. of course this will be
recursive, i.e., object X could simplify itself as object Y,
where object Y might go further simplification, until we are
left with building blocks only.
for example:
* int - return self
* float - string in the format "[+-]X.YYYe[+-]EEE"
* complex - two floats
* tuple - tuple of its simplified elements
* list - tuple of its simplified elements
* dict - a tuple of (key, value) tuples
* set - a tuple of its items
* file - raises TypeError("can't be simplified")
all in all, i choose to call that *simplification* rather than *serialization*,
as serialization is more about converting the simplified objects into a
sequence of bytes. my suggestion leaves that out for the
implementers of specific serializers.
so this is how a typical serializer (e.g., pickle) would be implemented:
* define its version of a "recursive simplifier"
* optionally use a "memo" to remember objects that were already
visited (so they would be serialized by reference rather than by value)
* define its flavor of converting ints, strings, and arrays to bytes
(binary, textual, etc. etc.)
- - - - - - - -
the default implementation of __getstate__, in object.__getstate__,
will simply return self.__dict__ and any self.__slots__
this removes the need for __reduce__, __reduce_ex__, and copy_reg,
and simplifies pickle greatly. it does require, however, adding
support for simplification for all builtin types... but this doesn't
call for much code:
def PyList_GetState(self):
state = tuple(PyObject_GetState(item) for item in self)
return PyListType, state
also note that it makes the copy module much simpler:
def copy(obj):
state = obj.__getstate__()
return type(obj).__setstate__(state)
- - - - - - - -
executive summary:
simplifying object serialization and copying by revising
__getstate__ and __setstate__, so that they return a
"simplified" version of the object.
this new mechanism should become an official API to
getting or setting the "contents" of objects (either builtin or
user-defined).
having this unified mechanism, pickling proxy objects would
work as expected.
if there's interest, i'll write a pep-like document to explain
all the semantics.
-tomer
___
Python-3000 mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-3000
Unsubscribe:
http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] Pre-peps on raise and except changes (was: Warning for 2.6 and greater)
On 1/23/07, Giovanni Bajo <[EMAIL PROTECTED]> wrote: > On 22/01/2007 23.45, Collin Winter wrote: > > > target, thus eliminating the reference cycle. The source-to-source > > translation, as suggested by Phillip J. Eby [#except-translation]_ is > > :: > > > > try: > > ... > > except E as N: > > ... > > ... > > > > is translated to :: > > > > try: > > ... > > except E as N: > > ... > > N = None > > del N > > ... > > > > The mail you link (which matched my recollection of the thread) has an > additional try/finally suite in the except block translation: > > except ExcType, e: > try: > # body > finally: > e = None > del e > > which is missing in the PEP. I think you should probably explain why the > try/finally is not needed (if it really is not). The try/finally is indeed needed; I just screwed up when copying it out for the PEP : ) > Also, the second code snippet should probably use the "except E, N" form (no > "as") to make more clear that you're speaking of the Python 2.x equivalent of > the 3.x form. Thanks! I've changed the "as" to "," and included a note that the second code sample is Python 2. Collin WInter ___ Python-3000 mailing list [email protected] http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] Pre-peps on raise and except changes
On 1/23/07, Brett Cannon <[EMAIL PROTECTED]> wrote: > I just realized I misread your paragraph and took E to represent an > exception class, not a nebulous object that could be an exception > class or instance. That's what I figured. Is it clearer if E is changed to EXCEPTION? """ 2. ``raise EXCEPTION`` is used to raise a new exception. This form has two sub-variants: ``EXCEPTION`` may be either an instance of ``BaseException`` or a subclass of ``BaseException`` [#pep352]_. If ``EXCEPTION`` is a subclass, it will be called with no arguments to obtain an exception instance. """ Thanks, Collin Winter ___ Python-3000 mailing list [email protected] http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
Re: [Python-3000] new pickle semantics/API
On Wed, Jan 24, 2007, tomer filiba wrote: > > i'm having great trouble in RPyC with pickling object proxies. several > users have asked for this feature, but no matter how hard i try to > "bend the truth", pickle always complains. it uses type(obj) for the > dispatching, which "uncovers" the object is actually a proxy, rather > than a real object. This should go to python-ideas, I think -- Aahz ([EMAIL PROTECTED]) <*> http://www.pythoncraft.com/ Help a hearing-impaired person: http://rule6.info/hearing.html ___ Python-3000 mailing list [email protected] http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
