On Sun, May 8, 2011 at 1:51 AM, Jeroen Vermeulen <[email protected]> wrote: > If we block reference demand-loading, we start taking oopses for those > situations. And if we then fix those oopses by doing more eager loading, we > may end up taking an unnecessary performance hit and pay for it in extra > code complexity.
If we generate an OOPS it means that scenario wasn't tested. Thats suboptimal at best. If reference loading is appropriate (I hold strong doubts that it ever is in the webapp post traversal), then the point doing the reference based load could reenable it temporarilty. (But doing that implies that we could just use a single-item bulk load too). > Wouldn't a logging-and-monitoring approach be preferable over a > blocking-and-fixing approach here? I don't think this is an alternative. Its a useful technique that solves different problems. We probably want both. > I haven't had time to work on my Storm > profiling branch lately, but it would let us track which references lead to > how many new objects being fetched, coming from how many other objects. > With information like that we could make rational tradeoffs between eager > and on-demand fetching. In a year of close attention now, I've seen one case where eager loading was a pessimisation - with commit() being slowed down by storm pathology when 10's of K of objects are live - it has O(live) overhead rather than O(changed). This is a bug with storm though: the eager loading would still have made the script faster (and more consistent) were it not for this defect. -Rob _______________________________________________ Mailing list: https://launchpad.net/~launchpad-dev Post to : [email protected] Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
