Re: [HACKERS] Parallel query and temp_file_limit

2016-07-07 Thread Robert Haas
On Tue, Jul 5, 2016 at 3:59 PM, Peter Geoghegan wrote: > On Tue, Jul 5, 2016 at 12:58 PM, Tom Lane wrote: >> Perhaps we could change the wording of temp_file_limit's description >> from "space that a session can use" to "space that a process can use" >> to help clarify this? > > That's all that I

Re: [HACKERS] Parallel query and temp_file_limit

2016-07-05 Thread Tom Lane
Peter Geoghegan writes: > On Tue, Jul 5, 2016 at 12:00 PM, Robert Haas wrote: >> I think that it is not worth mentioning specifically for >> temp_file_limit; to me that seems to be a hole with no bottom. We'll >> end up arguing about which GUCs should mention it specifically and >> there will be

Re: [HACKERS] Parallel query and temp_file_limit

2016-07-05 Thread Peter Geoghegan
On Tue, Jul 5, 2016 at 12:58 PM, Tom Lane wrote: > Perhaps we could change the wording of temp_file_limit's description > from "space that a session can use" to "space that a process can use" > to help clarify this? That's all that I was looking for, really. -- Peter Geoghegan -- Sent via pg

Re: [HACKERS] Parallel query and temp_file_limit

2016-07-05 Thread Peter Geoghegan
On Tue, Jul 5, 2016 at 12:00 PM, Robert Haas wrote: > I think that it is not worth mentioning specifically for > temp_file_limit; to me that seems to be a hole with no bottom. We'll > end up arguing about which GUCs should mention it specifically and > there will be no end to it. I don't think t

Re: [HACKERS] Parallel query and temp_file_limit

2016-07-05 Thread Robert Haas
On Tue, Jul 5, 2016 at 1:58 PM, Peter Geoghegan wrote: > On Tue, Jul 5, 2016 at 7:45 AM, Robert Haas wrote: >> Since Peter doesn't seem in a hurry to produce a patch for this issue, >> I wrote one. It is attached. I'll commit this in a day or two if >> nobody objects. > > Sorry about the delay.

Re: [HACKERS] Parallel query and temp_file_limit

2016-07-05 Thread Peter Geoghegan
On Tue, Jul 5, 2016 at 7:45 AM, Robert Haas wrote: > Since Peter doesn't seem in a hurry to produce a patch for this issue, > I wrote one. It is attached. I'll commit this in a day or two if > nobody objects. Sorry about the delay. Your patch seems reasonable, but I thought we'd also want to c

Re: [HACKERS] Parallel query and temp_file_limit

2016-07-05 Thread Robert Haas
On Tue, Jun 21, 2016 at 8:15 AM, Robert Haas wrote: > On Mon, Jun 20, 2016 at 11:01 PM, Tom Lane wrote: >> Peter Geoghegan writes: >>> On Wed, May 18, 2016 at 3:40 AM, Robert Haas wrote: What I'm tempted to do is trying to document that, as a point of policy, parallel query in 9.6 use

Re: [HACKERS] Parallel query and temp_file_limit

2016-06-21 Thread Robert Haas
On Mon, Jun 20, 2016 at 11:01 PM, Tom Lane wrote: > Peter Geoghegan writes: >> On Wed, May 18, 2016 at 3:40 AM, Robert Haas wrote: >>> What I'm tempted to do is trying to document that, as a point of >>> policy, parallel query in 9.6 uses up to (workers + 1) times the >>> resources that a single

Re: [HACKERS] Parallel query and temp_file_limit

2016-06-20 Thread Tom Lane
Peter Geoghegan writes: > On Wed, May 18, 2016 at 3:40 AM, Robert Haas wrote: >> What I'm tempted to do is trying to document that, as a point of >> policy, parallel query in 9.6 uses up to (workers + 1) times the >> resources that a single session might use. That includes not only CPU >> but al

Re: [HACKERS] Parallel query and temp_file_limit

2016-06-20 Thread Peter Geoghegan
On Wed, May 18, 2016 at 3:40 AM, Robert Haas wrote: > What I'm tempted to do is trying to document that, as a point of > policy, parallel query in 9.6 uses up to (workers + 1) times the > resources that a single session might use. That includes not only CPU > but also things like work_mem and tem

Re: [HACKERS] Parallel query and temp_file_limit

2016-06-07 Thread Peter Geoghegan
On Tue, Jun 7, 2016 at 8:32 AM, Robert Haas wrote: > You previously offered to write a patch for this. Are you still > planning to do that? OK, I'll get to that in the next few days. I'm slightly concerned that I might have missed a real problem in the code. I'll need to examine the issue more

Re: [HACKERS] Parallel query and temp_file_limit

2016-06-07 Thread Robert Haas
On Sun, Jun 5, 2016 at 4:32 PM, Peter Geoghegan wrote: > On Wed, May 18, 2016 at 12:01 PM, Peter Geoghegan wrote: >>> I think for 9.6 we just have to document this issue. In the next >>> release, we could (and might well want to) try to do something more >>> clever. >> >> Works for me. You may w

Re: [HACKERS] Parallel query and temp_file_limit

2016-06-05 Thread Peter Geoghegan
On Wed, May 18, 2016 at 12:01 PM, Peter Geoghegan wrote: >> I think for 9.6 we just have to document this issue. In the next >> release, we could (and might well want to) try to do something more >> clever. > > Works for me. You may wish to update comments within fd.c at the same time. I've crea

Re: [HACKERS] Parallel query and temp_file_limit

2016-05-18 Thread Peter Geoghegan
On Wed, May 18, 2016 at 3:40 AM, Robert Haas wrote: >> I'll write a patch to fix the issue, if there is a consensus on a solution. > > I think for 9.6 we just have to document this issue. In the next > release, we could (and might well want to) try to do something more > clever. Works for me. Yo

Re: [HACKERS] Parallel query and temp_file_limit

2016-05-18 Thread David Rowley
On 18 May 2016 at 22:40, Robert Haas wrote: > On Tue, May 17, 2016 at 6:40 PM, Peter Geoghegan wrote: > > On Tue, May 17, 2016 at 3:33 PM, Peter Geoghegan wrote: > >> Fundamentally, since temporary_files_size enforcement simply > >> piggy-backs on low-level fd.c file management, without any > >

Re: [HACKERS] Parallel query and temp_file_limit

2016-05-18 Thread Robert Haas
On Tue, May 17, 2016 at 6:40 PM, Peter Geoghegan wrote: > On Tue, May 17, 2016 at 3:33 PM, Peter Geoghegan wrote: >> Fundamentally, since temporary_files_size enforcement simply >> piggy-backs on low-level fd.c file management, without any >> consideration of what the temp files contain, it'll be

Re: [HACKERS] Parallel query and temp_file_limit

2016-05-17 Thread Peter Geoghegan
On Tue, May 17, 2016 at 3:33 PM, Peter Geoghegan wrote: > Fundamentally, since temporary_files_size enforcement simply > piggy-backs on low-level fd.c file management, without any > consideration of what the temp files contain, it'll be hard to be sure > that parallel workers will not have issues.

Re: [HACKERS] Parallel query and temp_file_limit

2016-05-17 Thread Peter Geoghegan
On Tue, May 17, 2016 at 1:53 PM, Amit Kapila wrote: > What kind of special treatment are you expecting for temporary_files_size, > also why do you think it is required? Currently neither we build hash in > parallel nor there is any form of parallel sort work. I expect only that temporary_files_s

Re: [HACKERS] Parallel query and temp_file_limit

2016-05-17 Thread Amit Kapila
On Wed, May 18, 2016 at 12:55 AM, Peter Geoghegan wrote: > > temp_file_limit "specifies the maximum amount of disk space that a > session can use for temporary files, such as sort and hash temporary > files", according to the documentation. That's not true when parallel > query is in use, since th