Re: struct.Struct random access
castironpi [EMAIL PROTECTED] wrote: CMIIW correct me if I'm wrong, I don't think that pack_into returns a value the way that pack does. Sorry, I was not aware that struct.pack_into and struct.unpack_from already existed (they were added in Python 2.5). I thought you were proposing them as new additions. Because of that, the rest of my post doesn't make much sense. -- Tim Roberts, [EMAIL PROTECTED] Providenza Boekelheide, Inc. -- http://mail.python.org/mailman/listinfo/python-list
Re: struct.Struct random access
On Aug 29, 10:30 pm, Tim Roberts [EMAIL PROTECTED] wrote: castironpi [EMAIL PROTECTED] wrote: CMIIW correct me if I'm wrong, I don't think that pack_into returns a value the way that pack does. Sorry, I was not aware that struct.pack_into and struct.unpack_from already existed (they were added in Python 2.5). I thought you were proposing them as new additions. Because of that, the rest of my post doesn't make much sense. -- Tim Roberts, [EMAIL PROTECTED] Providenza Boekelheide, Inc. I was a lot less enthusiastic too when I discovered 'PyObject_AsWriteBuffer'. Nevertheless I think it could be instructive as well as a benefit. The ctypes conversion, cast( buffer_p + offset ), is a bit of a kludge. With the introduction of 'namedtuple' type, I think it falls right in line with the direction Python is going in. It might suffice to merely expose the 'offset' member of the items in the array of 'formatcode'. Though, the addition of the dictionary to lookup offsets by field name could offset the benefit somewhat. -- http://mail.python.org/mailman/listinfo/python-list
Re: struct.Struct random access
castironpi [EMAIL PROTECTED] wrote: I'd like to seriously nominate this idea and get a considered opinion on it. struct.Struct lets you encode Python objects into structured memory. It accepts a format string, and optionally a buffer and offset to/from which to read/write the structure. What do you think of random access to the results? To avoid Marc's concern about meaningless anonymous record entries, model the constructor after 'namedtuple' type. (unproduced) packer= struct.Struct( 'IIIf255p', 'i1 i2 i3 afloat name' ) packer.pack_into( buf, off, 10, 20, 30, 0.5, 'abc' ) packer.unpack_from( buf, off, 'i3' ) 30 packer.unpack_from( buf, off ) ( 10, 20, 30, 0.5, 'abc' ) packer.pack_into( buf, off, 'i1', 12 ) What data type would you expect buf to be? Your design requires it to be both an input and an output parameter. Today's struct converts into and out of a string, and strings are immutable. So, to continue with that standard, you would have to pass a string into pack_into and have the modified result returned as an output: buf = packer.pack_into( None, 0, 10, 20, 30, 0.5, 'abc' ) buf = packer.pack_into( buf, 0, 'i1', 12 ) In the end, I'm not sure you are really saving anything. Even in sequential access speed benefits, by avoiding the construction of n-1 objects. This is a fairly major change, because it requires a struct.Struct object to maintain state, something that the struct module currently does not do. In my personal opinion, this is a rather large and confusing change, for very little benefit. -- Tim Roberts, [EMAIL PROTECTED] Providenza Boekelheide, Inc. -- http://mail.python.org/mailman/listinfo/python-list
Re: struct.Struct random access
On Aug 28, 1:59 am, Tim Roberts [EMAIL PROTECTED] wrote: castironpi [EMAIL PROTECTED] wrote: I'd like to seriously nominate this idea and get a considered opinion on it. struct.Struct lets you encode Python objects into structured memory. It accepts a format string, and optionally a buffer and offset to/from which to read/write the structure. What do you think of random access to the results? To avoid Marc's concern about meaningless anonymous record entries, model the constructor after 'namedtuple' type. (unproduced) packer= struct.Struct( 'IIIf255p', 'i1 i2 i3 afloat name' ) packer.pack_into( buf, off, 10, 20, 30, 0.5, 'abc' ) packer.unpack_from( buf, off, 'i3' ) 30 packer.unpack_from( buf, off ) ( 10, 20, 30, 0.5, 'abc' ) packer.pack_into( buf, off, 'i1', 12 ) What data type would you expect buf to be? Your design requires it to be both an input and an output parameter. Today's struct converts into and out of a string, and strings are immutable. So, to continue with that standard, you would have to pass a string into pack_into and have the modified result returned as an output: buf = packer.pack_into( None, 0, 10, 20, 30, 0.5, 'abc' ) buf = packer.pack_into( buf, 0, 'i1', 12 ) In the end, I'm not sure you are really saving anything. Even in sequential access speed benefits, by avoiding the construction of n-1 objects. This is a fairly major change, because it requires a struct.Struct object to maintain state, something that the struct module currently does not do. In my personal opinion, this is a rather large and confusing change, for very little benefit. -- Tim Roberts, [EMAIL PROTECTED] Providenza Boekelheide, Inc. CMIIW correct me if I'm wrong, I don't think that pack_into returns a value the way that pack does. The syntax is ambiguous in the case of two-element structs: are you packing a new struct or assigning a field? Maybe the field name needs a keyword. A field name can be a third argument in unpack_from regardless, however. Or use the field name as keyword: strt.pack_into( buf, off, i1= 32 ). I just tried an experiment with an Mmap and PyObject_AsWriteBuffer. Mmaps could work but you would not be able to use arbitrary strings. You would have to specially allocate a ctypes buffer with a size to match the packed size of the structure. -- http://mail.python.org/mailman/listinfo/python-list
struct.Struct random access
I'd like to seriously nominate this idea and get a considered opinion on it. struct.Struct lets you encode Python objects into structured memory. It accepts a format string, and optionally a buffer and offset to/from which to read/write the structure. What do you think of random access to the results? To avoid Marc's concern about meaningless anonymous record entries, model the constructor after 'namedtuple' type. (unproduced) packer= struct.Struct( 'IIIf255p', 'i1 i2 i3 afloat name' ) packer.pack_into( buf, off, 10, 20, 30, 0.5, 'abc' ) packer.unpack_from( buf, off, 'i3' ) 30 packer.unpack_from( buf, off ) ( 10, 20, 30, 0.5, 'abc' ) packer.pack_into( buf, off, 'i1', 12 ) Even in sequential access speed benefits, by avoiding the construction of n-1 objects. PS: If you don't have time or don't want to consider it seriously, or want to see more specifics, just say so. -- http://mail.python.org/mailman/listinfo/python-list
