On Thu, 31 May 2012, Jeff Janes wrote:
No, idt_match is getting filled by multi-threaded copy() and then joined
with 4 other big tables like idt_phot. The result is then split into
partitions.
That does make things more complicated. But you could you partition
it at that level and then do the joins partition-wise?
Unfortunately the information to understand in which partition the data
needs to be put in is only available from the idt_match table. So I have
to do at least one join with idt_match. But I will experiment further
with ways to perform queries, I just stopped doing that because I saw
these problems with scalability.
I don't have much experience at data partitioning (well, I do, but the
experience is with partitioning in Perl with terabytes of flat files,
not in PG :) ) but I think that once you have your partitioning keys
you want to apply them the same way up and down the data set.
I'm not sure what you mean by "the same way up and down the data set".
Cheers,
S
*****************************************************
Sergey E. Koposov, PhD, Research Associate
Institute of Astronomy, University of Cambridge
Madingley road, CB3 0HA, Cambridge, UK
Tel: +44-1223-337-551 Web: http://www.ast.cam.ac.uk/~koposov/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
