Don't really know if this will be useful but i'd try pytables: http://www.pytables.org/moin it deals very well with every kind of hierarchical data sets, doesn't matter the size. It will let you load only significant data, and you'll be able to query your data. It's built on top of HDF5 libraries but exposes a very friendly pythonic interface. Surely you'll still have to implement all of the graph logic yourself but this could be a good starting point. Hope it helps
Marco -- http://mail.python.org/mailman/listinfo/python-list
