-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
... > We have queries on it that bring back 200000 rows to do set > differences; thats 10 times the data for the fact table - and its > being joined on awkwardly; making those do a graphtraversal in the > query (SELECT WITH RECURSIVE) should get it processing a few thousand > rows most of the time - which will be a lot faster. I'm not sure if you can do set intersection via recursive queries in postgres, I would imagine it is possible, but probably runs into the same issues we've run into in bzr (race conditions in simple graph traversal can be ugly). (Stuff like X being found as an ancestor of one branch before the other, and you stop walking some of the common ancestors to find that it is really common to both.) history-db taught be a lot about possibly layering here. Adding 'generation' (greatest-distance-from-origin) can help a ton, since if you've gone far enough back, you *know* you've found everything that can be uncommon, etc. Anyway, my point is that to do accurate set intersection, postgres may still need to touch those 200k rows as the data exists today, even if it only returns 10 rows when it is done. The fact that you actually shrink to 1/3rd the size instead of grow to 3x the size is a great hint that this will work. The actual "matrix" is huge, but there are a lot of zeros, so having it as a sparse matrix works great. I don't really see people taking something with 100k bugs, and actually milestoning all of them and tagging all of them. Unless we get much better at automatic scripting for it. As long as a human is actually making the decision, I think your scaling factors are probably under control. John =:-> -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk2msrEACgkQJdeBCYSNAAOiVwCfY4qD4/gn9Jyo6P7vsyM2w79p YeEAoJQNIC6pqA7oBTy+y5k3Nl53vcZY =YDH0 -----END PGP SIGNATURE----- _______________________________________________ Mailing list: https://launchpad.net/~launchpad-dev Post to : [email protected] Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
