On 5/21/05, DJTB <[EMAIL PROTECTED]> wrote:
> [posted to comp.lang.python, mailed to [EMAIL PROTECTED]
[Following up to both places.]
> I'm having problems storing large amounts of objects in a ZODB.
> After committing changes to the database, elements are not cleared from
> memory. Since the num
class ExtendedTupleTable(Persistent):
def __init__(self):
self.interning = ObjectInterning()
# This Set stores all generated ExtendedTuple objects.
self.ets = Set() # et(s): ExtendedTuple object(s)
# This dictionary stores a mapping of elements to Sets of
in RAM, my program gets killed with signal 11 or signal 9...
Below a minimal working (or actually: it doesn't work because of memory
errors)
example code with hopefully enough comments:
# This was suggested by Tim Peters in comp.lang.python thread
# 'processing a Very Large file'
# I
Mike Meyer wrote:
>
> Unknown. Python relies on the C alloc/free routines for handling
> memory. del may immediately free() the memory (I don't know), but that
> doesn't mean it gets released to the OS. None of the implementations
> of alloc/free I'm aware of ever give the memory back to the OS.
DJTB <[EMAIL PROTECTED]> writes:
> I'm not a Python memory specialist, but does del immediately release/free
> the memory to the OS? I thought it was impossible to let Python immediately
> release memory.
Unknown. Python relies on the C alloc/free routines for handling
memory. del may immediately
Tim Peters wrote:
>
>>tuple_size = int(splitres[0])+1
>>path_tuple = tuple(splitres[1:tuple_size])
>>conflicts = Set(map(int,splitres[tuple_size:-1]))
>
> Do you really mean to throw away the last value on the line? That is,
> why is the slice here [tuple_size:-1] rather
Robert Brewer wrote:
> DJTB wrote:
>> I'm trying to manually parse a dataset stored in a file. The
>> data should be converted into Python objects.
>>
>
> The first question I would ask is: what are you doing with "result", and
> can the consumption of "result" be done iteratively?
>
>
The pr
DJTB wrote:
> Hi,
>
> I'm trying to manually parse a dataset stored in a file. The data should be
> converted into Python objects.
In addition to what the others have mentioned, this sort of problem is
pretty easy to do with a C coded extension type, if you have (or can
buy/borrow) any C skills
I'm surprised you didn't recommend to use ZODB. Seems like an ideal way
to manage this large amount of data as a collection of Python objects...
--
http://mail.python.org/mailman/listinfo/python-list
DJTB wrote:
> I'm trying to manually parse a dataset stored in a file. The
> data should be converted into Python objects.
>
> Here is an example of a single line of a (small) dataset:
>
> 3 13 17 19 -626177023 -1688330994 -834622062 -409108332
> 297174549 955187488
> 589884464 -1547848504 8573
[DJTB]
> I'm trying to manually parse a dataset stored in a file. The data should be
> converted into Python objects.
>
> Here is an example of a single line of a (small) dataset:
>
> 3 13 17 19 -626177023 -1688330994 -834622062 -409108332 297174549 955187488
> 589884464 -1547848504 857311165 585
Hi,
I'm trying to manually parse a dataset stored in a file. The data should be
converted into Python objects.
Here is an example of a single line of a (small) dataset:
3 13 17 19 -626177023 -1688330994 -834622062 -409108332 297174549 955187488
589884464 -1547848504 857311165 585616830 -74991020
12 matches
Mail list logo
