Re: [HACKERS] parallelism and sorting

2015-11-24 Thread Robert Haas
On Tue, Nov 24, 2015 at 7:44 PM, Jim Nasby wrote: > On 11/23/15 5:47 PM, Robert Haas wrote: >> 2. In Parallel Seq Scan, the determination of what page to scan next >> isn't dependent on the contents of any page previously scanned. In >> Parallel Index Scan, it is. Therefore, the amount of effect

Re: [HACKERS] parallelism and sorting

2015-11-24 Thread Jim Nasby
On 11/24/15 7:10 AM, Ants Aasma wrote: The use case I have in mind is a table containing multiple years worth of (approximately) time series data, where overwhelming majority of queries are explicitly interested in recent data. Having a partial index with WHERE tstamp > $some_recent_tstamp cuttin

Re: [HACKERS] parallelism and sorting

2015-11-24 Thread Jim Nasby
On 11/23/15 5:47 PM, Robert Haas wrote: 2. In Parallel Seq Scan, the determination of what page to scan next isn't dependent on the contents of any page previously scanned. In Parallel Index Scan, it is. Therefore, the amount of effective parallelism is likely to be less. This doesn't mean tha

Re: [HACKERS] parallelism and sorting

2015-11-24 Thread Amit Kapila
On Tue, Nov 24, 2015 at 7:53 PM, Robert Haas wrote: > > On Tue, Nov 24, 2015 at 7:59 AM, Amit Kapila wrote: > > On Tue, Nov 24, 2015 at 8:59 AM, Robert Haas wrote: > >> One idea about parallel sort is that perhaps if multiple workers feed > >> data into the sort, they can each just sort what the

Re: [HACKERS] parallelism and sorting

2015-11-24 Thread Robert Haas
On Tue, Nov 24, 2015 at 7:59 AM, Amit Kapila wrote: > On Tue, Nov 24, 2015 at 8:59 AM, Robert Haas wrote: >> One idea about parallel sort is that perhaps if multiple workers feed >> data into the sort, they can each just sort what they have and then >> merge the results. > > Sounds like a good ap

Re: [HACKERS] parallelism and sorting

2015-11-24 Thread Ants Aasma
On Tue, Nov 24, 2015 at 5:29 AM, Robert Haas wrote: > On Mon, Nov 23, 2015 at 8:45 PM, Peter Geoghegan wrote: >> Beyond that, CREATE INDEX and CLUSTER utility >> cases will also need to be parallelized without all this executor >> infrastructure. > > Or, alternatively, CREATE INDEX and CLUSTER co

Re: [HACKERS] parallelism and sorting

2015-11-24 Thread Amit Kapila
On Tue, Nov 24, 2015 at 8:59 AM, Robert Haas wrote: > > > One idea about parallel sort is that perhaps if multiple workers feed > data into the sort, they can each just sort what they have and then > merge the results. Sounds like a good approach for parallel sorting, however small extension to

Re: [HACKERS] parallelism and sorting

2015-11-23 Thread Robert Haas
On Mon, Nov 23, 2015 at 8:45 PM, Peter Geoghegan wrote: >> 2. Within parallel query, there are two reasons to care about data >> that is in sorted order. First, we might need to deliver the results >> to the user in a particular order, because they've specified ORDER BY >> whatever. Second, the

Re: [HACKERS] parallelism and sorting

2015-11-23 Thread Peter Geoghegan
On Mon, Nov 23, 2015 at 2:01 PM, Robert Haas wrote: > I've been thinking about how parallelism interacts with sorting over > the last few days and I wanted to share a few preliminary thoughts. I > definitely don't have all the answers worked out here yet, so thoughts > are welcome. I think it's

Re: [HACKERS] parallelism and sorting

2015-11-23 Thread Robert Haas
On Mon, Nov 23, 2015 at 5:38 PM, David Fetter wrote: > That sounds like a very handy thing to have. Any idea whether it's > possible for 9.6? Is there any of the Parallel Seq Scan code that > looks like it could be reused or slightly generalized for the > implementation? I think it would be a g

Re: [HACKERS] parallelism and sorting

2015-11-23 Thread David Fetter
On Mon, Nov 23, 2015 at 05:01:43PM -0500, Robert Haas wrote: > Hi, > > [snip] > > If we had a Parallel Index Scan which worked like a Parallel Seq > Scan, That sounds like a very handy thing to have. Any idea whether it's possible for 9.6? Is there any of the Parallel Seq Scan code that looks