I'm a little puzzled on something I ran across today. Of course it is me and I
sometimes do strange things. I'm still off doing conversion stuff and I have
this script that is fairly complex, well does a lot of work. Its main purpose
is to convert an ot object that is a straight key value object and convert it
into a more structured collection, then convert the collection to json and
return it.
read a record
get the ot object from a blob in the record
convert it to a collection A
clear the ot object
filter the collection and create 5 new collections A1..A5
go through the 5 collections and create 5 other collections in a
different format B1..B5
convert the last 5 collection to json and return the json
I had about a 1000 records that I had to do this process on and called the
script with curl from Rail.
It would take about 4 seconds to make the round trip, include updating data in
Rails. So it took about an hour to convert the 1000 records.
Thought I could improve that time and I wrote a version that active4d would
loop through the 1000 records, do the stuff in the script, append to a blob and
then write the blob at the end. I then planed on parsing the file on the rails
side.
I first tried it with 10 records and it create the file in about a minute. I
then did a 100 records and can't remember how long. I finally tried it with
1125 record and that is when I learned that something was eating up memory.
After about 10 minutes the fan started to rev up (I had the script timeout set
for an hour!). I could then tell that virtual memory was doing a lot of
swapping. I let it go for about a half hour and it became painful. I opened up
activity monitor and of course 4D was the hog. I tried runtime explore and that
crashed 4d.
This long story comes down to one basic question: When I create 10 or so
collection in a loop a thousand times, does the collection memory get cleared
when I define another collection using the same name? Something ate up the
memory and that is the only thing I can think of. I guess I can do a clear
collection at the end of the loop and see if I observe the some results.
Steve
_______________________________________________
Active4D-dev mailing list
[email protected]
http://mailman.aparajitaworld.com/mailman/listinfo/active4d-dev
Archives: http://mailman.aparajitaworld.com/archive/active4d-dev/
