In <[EMAIL PROTECTED]>, rohit wrote:
> i am making a program for desktop search.
> in order to make it more effective im implementing the records in the
> form of a tree(files starting with 'a','b','c'....have different
> trees ..so 26 trees in all) in memory and writing it down in file.
> the max size file has 16000 records...now to implement the tree in
> list i'm using line no as index ..and empty child nodes are
> represented as "\n"
> all this work is going on in the memory..
> problem is the system eats up my 512 mb RAM +1gb virtual store n hangs
> cant think of an effective way to implement tree in memory(i can
> compact it on disk by writing just the index no..along with the record
> from which tree in memory can be reconstructed, but i have to
> implement tree as previous to implement random access)
I'm not quite sure what exactly you have as in-memory data structures and
how many "records" -- are you sure you don't keep references to objects
you don't really need anymore? Or maybe you have object cycles and
implemented the `__del__()` method on those objects?
Anyway… If the data doesn't fit into memory anymore it's time to put
them into a database. Either a `shelve`, an SQL database like SQLite or
maybe an object database like zodb or Durus.
Ciao,
Marc 'BlackJack' Rintsch
--
http://mail.python.org/mailman/listinfo/python-list
