matiskiva wrote:
> Just to skip the newbie replies :) I attempted to go over the "rects"
> collection and expunge all the items.
> This direct approach didn't work.
expunging is not enough. did you expire the rects collection on each
Detection object, and also remove any other references which may exist to
any of those "rects" ? this is a python reference counting issue. you
can see just how many objects are not released by using gc.collect() and
then len(gc.get_objects()). note that the memory footprint of the
cPython interpreter doesn't shrink very much once grown to a particular
size so releasing will also not really help much with the total size of
the interpreter.
>
> Mati
>
> On Jan 31, 11:01 am, matiskiva <mati.sk...@mobileye.com> wrote:
>> Hello all,
>> I have an application that loads many "Detection" objects, each of
>> them having thousands of "Rect" objects, loaded with eager loading.
>> I am performing some calculation on the rects and saves the result,
>> along with the Detection objects.
>> But than I am stuck with all the Rect objects, taking up lots of
>> memory and I can't find a way to release them.
>>
>> And for the code:
>>
>> # there is some kind of RectDB mapping and definition...
>> mapper(RectDB....
>>
>> # there is some kind of DetectionDB mapping and definition
>> mapper(DetectionDB,_detection_table,properties={
>> "rects":relation(RectDB),
>> ....
>>
>> # query the detections
>> detections = session.query(DetectionDB).filter(....).eagerload
>> ("rects").all()
>>
>> for d in detections:
>> # make calculation
>> rectsScore = calculateScore(d)
>> # save score
>> d.score = rectsScore
>> # release rects, they take up lots of space
>> releaseEagerLoading(d.rects)
>>
>> # update the detections
>> ...
>>
>> Currently I have no idea how to implement releaseEagerLoading. The
>> result is very large (over 1GB) memory footprint. I tried the same
>> without using the rects (empty score calculation) and without the
>> eager loading and it was below 100MB.
>> I think I can guess why it happens, but the question is - how can I
>> release the eager loading?
>>
>> Thank you in advance,
>> Mati
>
> --
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To post to this group, send email to sqlalch...@googlegroups.com.
> To unsubscribe from this group, send email to
> sqlalchemy+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/sqlalchemy?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to sqlalch...@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
