On Fri, Oct 28, 2011 at 11:47:48AM +0200, Gilles Louppe wrote:
> > import cPickle
> > for i in range(0, 20):
> > with open("forest%d.pkl" % (i), 'r') as f:
> > start = datetime.now()
> > a = cPickle.load(f)
> > print 'loaded ', i, datetime.now() - start
> In this case, the reference to `a` is lost after each iteration, which
> means that the forest can be garbage-collected and that memory can be
> reused.
> [snip]
> In contrast, in this case, forests can no longer be garbage-collected
> and new memory need to be allocated at each iteration, the private heap
> need to be extended and so on. In the process, I suspect that objects
> are moved to one place to another, which may be the reason why it
> slows down (since the number of objects in memory keeps increasing, it
> takes longer and longer to move them).
Interesting. This hypothesis should be testable, for instance by keeping
a reference on 'a', appending it to a list. I'd be interested in the
results, if you mind trying out Brian.
Cheers,
Gaël
------------------------------------------------------------------------------
The demand for IT networking professionals continues to grow, and the
demand for specialized networking skills is growing even more rapidly.
Take a complimentary Learning@Cisco Self-Assessment and learn
about Cisco certifications, training, and career opportunities.
http://p.sf.net/sfu/cisco-dev2dev
_______________________________________________
Scikit-learn-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
