On 30/08/2010 16.44, Knacktus wrote:
Hey everyone,
I have a huge number of data items coming from a database. So far
there're no restrictions about how to model the items. They can be
dicts, objects of a custom class (preferable with __slots__) or namedTuple.
Those items have references to each other using ids. Fresh from the
database the items look like this (using dicts as examples):
item_1 = {"id": 1, "name": "root", "children_ids": [2, 3]}
item_2 = {"id": 2, "name": "child_1", "children_ids": [4]}
item_3 = {"id": 3, "name": "child_2", "children_ids": [6, 7, 8]}
Now I'd like to resolve the references on demand.
Two questions and one doubt for you:
1- How many "generations" do you want to keep in a single item (call it
dictionary or list, or record, whatever)? I mean, what if some children
have children too, and some of those have more children, etc ?
2- Are you SURE that there are no circular references in your database?
In your example, what if item_3 was
item_3 = {"id": 3, "name": "child_2", "children_ids": [6, 1, 8]}? Is't
it possible that those recursion limit problems you had could come from
some circular reference in your data?
d- If the number of data items is really huge, are you sure that you
want to keep the whole family in memory at the same time? It depends on
the answer you gave to my question #1, of course, but if retrieving an
item from your database is quick as it should be, you could use a query
to resolve the references on demand, and you wouldn't need a special
structure to hold "the rest of the family". If the retrieval is slow or
difficult, then the creation of your structure could take a significant
amount of time.
Hope this helps,
Francesco
Nessun virus nel messaggio in uscita.
Controllato da AVG - www.avg.com
Versione: 9.0.851 / Database dei virus: 271.1.1/3100 - Data di rilascio:
08/29/10 08:34:00
_______________________________________________
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
