Re: [HACKERS] parallelism and sorting

On Tue, Nov 24, 2015 at 7:53 PM, Robert Haas wrote: > > On Tue, Nov 24, 2015 at 7:59 AM, Amit Kapila wrote: > > On Tue, Nov 24, 2015 at 8:59 AM, Robert Haas wrote: > >> One idea about parallel sort is that perhaps if

Re: [HACKERS] parallelism and sorting

On 11/23/15 5:47 PM, Robert Haas wrote: 2. In Parallel Seq Scan, the determination of what page to scan next isn't dependent on the contents of any page previously scanned. In Parallel Index Scan, it is. Therefore, the amount of effective parallelism is likely to be less. This doesn't mean

Re: [HACKERS] parallelism and sorting

On 11/24/15 7:10 AM, Ants Aasma wrote: The use case I have in mind is a table containing multiple years worth of (approximately) time series data, where overwhelming majority of queries are explicitly interested in recent data. Having a partial index with WHERE tstamp > $some_recent_tstamp

Re: [HACKERS] parallelism and sorting

On Tue, Nov 24, 2015 at 7:44 PM, Jim Nasby wrote: > On 11/23/15 5:47 PM, Robert Haas wrote: >> 2. In Parallel Seq Scan, the determination of what page to scan next >> isn't dependent on the contents of any page previously scanned. In >> Parallel Index Scan, it is.

Re: [HACKERS] parallelism and sorting

On Tue, Nov 24, 2015 at 7:59 AM, Amit Kapila wrote: > On Tue, Nov 24, 2015 at 8:59 AM, Robert Haas wrote: >> One idea about parallel sort is that perhaps if multiple workers feed >> data into the sort, they can each just sort what they have and

Re: [HACKERS] parallelism and sorting

On Tue, Nov 24, 2015 at 8:59 AM, Robert Haas wrote: > > > One idea about parallel sort is that perhaps if multiple workers feed > data into the sort, they can each just sort what they have and then > merge the results. Sounds like a good approach for parallel sorting,

Re: [HACKERS] parallelism and sorting

On Tue, Nov 24, 2015 at 5:29 AM, Robert Haas wrote: > On Mon, Nov 23, 2015 at 8:45 PM, Peter Geoghegan wrote: >> Beyond that, CREATE INDEX and CLUSTER utility >> cases will also need to be parallelized without all this executor >> infrastructure. > > Or,

[HACKERS] parallelism and sorting

Hi, I've been thinking about how parallelism interacts with sorting over the last few days and I wanted to share a few preliminary thoughts. I definitely don't have all the answers worked out here yet, so thoughts are welcome. But here are a few observations: 1. Parallel sort is useful but

Re: [HACKERS] parallelism and sorting

On Mon, Nov 23, 2015 at 05:01:43PM -0500, Robert Haas wrote: > Hi, > > [snip] > > If we had a Parallel Index Scan which worked like a Parallel Seq > Scan, That sounds like a very handy thing to have. Any idea whether it's possible for 9.6? Is there any of the Parallel Seq Scan code that looks

Re: [HACKERS] parallelism and sorting

On Mon, Nov 23, 2015 at 5:38 PM, David Fetter wrote: > That sounds like a very handy thing to have. Any idea whether it's > possible for 9.6? Is there any of the Parallel Seq Scan code that > looks like it could be reused or slightly generalized for the > implementation? I

Re: [HACKERS] parallelism and sorting

On Mon, Nov 23, 2015 at 2:01 PM, Robert Haas wrote: > I've been thinking about how parallelism interacts with sorting over > the last few days and I wanted to share a few preliminary thoughts. I > definitely don't have all the answers worked out here yet, so thoughts > are

Re: [HACKERS] parallelism and sorting

On Mon, Nov 23, 2015 at 8:45 PM, Peter Geoghegan wrote: >> 2. Within parallel query, there are two reasons to care about data >> that is in sorted order. First, we might need to deliver the results >> to the user in a particular order, because they've specified ORDER BY >>