Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-21 Thread Kyotaro HORIGUCHI
At Mon, 19 Dec 2016 12:24:38 -0500, Robert Haas wrote in > Interesting idea. My bet is that nobody cares about dtrace very much. FWIW, I just had an inquiry about system tap for PostgreSQL but he probed using function names. I mean, at least few people care it, not nobody, but... regards, --

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-19 Thread Robert Haas
On Mon, Dec 19, 2016 at 6:35 PM, Thomas Munro wrote: > On Tue, Dec 20, 2016 at 11:12 AM, Robert Haas wrote: >> On Thu, Dec 1, 2016 at 6:35 AM, Thomas Munro >> wrote: >>> On Sat, Nov 26, 2016 at 1:55 AM, Thomas Munro >>> wrote: Here's a new version to apply on top of dsa-v7.patch. >>> >>> H

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-19 Thread Thomas Munro
On Tue, Dec 20, 2016 at 11:12 AM, Robert Haas wrote: > On Thu, Dec 1, 2016 at 6:35 AM, Thomas Munro > wrote: >> On Sat, Nov 26, 2016 at 1:55 AM, Thomas Munro >> wrote: >>> Here's a new version to apply on top of dsa-v7.patch. >> >> Here's a version to go with dsa-v8.patch. > > All right, so I've

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-19 Thread Robert Haas
On Thu, Dec 1, 2016 at 6:35 AM, Thomas Munro wrote: > On Sat, Nov 26, 2016 at 1:55 AM, Thomas Munro > wrote: >> Here's a new version to apply on top of dsa-v7.patch. > > Here's a version to go with dsa-v8.patch. All right, so I've committed this, but not before (1) renaming some things that you

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-19 Thread Robert Haas
On Sun, Dec 18, 2016 at 10:33 PM, Thomas Munro wrote: > On Sat, Dec 17, 2016 at 5:41 AM, Robert Haas wrote: >> On Wed, Dec 14, 2016 at 3:25 PM, Robert Haas wrote: >>> Thoughts? >> >> Hearing no objections, I've gone ahead and committed this. If that >> makes somebody really unhappy I can revert

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-18 Thread Thomas Munro
On Sat, Dec 17, 2016 at 5:41 AM, Robert Haas wrote: > On Wed, Dec 14, 2016 at 3:25 PM, Robert Haas wrote: >> Thoughts? > > Hearing no objections, I've gone ahead and committed this. If that > makes somebody really unhappy I can revert it, but I am betting that > the real story is that nobody car

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-16 Thread Alvaro Herrera
Robert Haas wrote: > I am not sure the issue was time so much as the ability to foresee all > the problems we'd want to solve. I think all that movement is okay. It's not like we're breaking things to no purpose. The amount of effort that has to go into making extensions compile with changed AP

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-16 Thread Robert Haas
On Fri, Dec 16, 2016 at 12:36 PM, Alvaro Herrera wrote: > Robert Haas wrote: >> On Wed, Dec 14, 2016 at 3:25 PM, Robert Haas wrote: >> > Thoughts? >> >> Hearing no objections, I've gone ahead and committed this. If that >> makes somebody really unhappy I can revert it, but I am betting that >> t

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-16 Thread Robert Haas
On Fri, Dec 16, 2016 at 12:37 PM, Andres Freund wrote: > Yea, I don't think that's good either. I'm all for evolving APIs when > necessary, but constantly breaking the same API release after release > seems indicative of needing to spend a bit more time on it in the first > round. I am not sure

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-16 Thread Andres Freund
On 2016-12-16 12:33:11 -0500, Robert Haas wrote: > On Fri, Dec 16, 2016 at 12:32 PM, Robert Haas wrote: > > On Fri, Dec 16, 2016 at 12:28 PM, Andres Freund wrote: > >> On 2016-12-16 11:41:49 -0500, Robert Haas wrote: > >>> On Wed, Dec 14, 2016 at 3:25 PM, Robert Haas > >>> wrote: > >>> > Though

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-16 Thread Andres Freund
On 2016-12-16 12:32:49 -0500, Robert Haas wrote: > On Fri, Dec 16, 2016 at 12:28 PM, Andres Freund wrote: > > On 2016-12-16 11:41:49 -0500, Robert Haas wrote: > >> On Wed, Dec 14, 2016 at 3:25 PM, Robert Haas wrote: > >> > Thoughts? > >> > >> Hearing no objections, I've gone ahead and committed t

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-16 Thread Alvaro Herrera
Robert Haas wrote: > On Wed, Dec 14, 2016 at 3:25 PM, Robert Haas wrote: > > Thoughts? > > Hearing no objections, I've gone ahead and committed this. If that > makes somebody really unhappy I can revert it, but I am betting that > the real story is that nobody cares about preserving T_ID(). AFA

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-16 Thread Robert Haas
On Fri, Dec 16, 2016 at 12:32 PM, Robert Haas wrote: > On Fri, Dec 16, 2016 at 12:28 PM, Andres Freund wrote: >> On 2016-12-16 11:41:49 -0500, Robert Haas wrote: >>> On Wed, Dec 14, 2016 at 3:25 PM, Robert Haas wrote: >>> > Thoughts? >>> >>> Hearing no objections, I've gone ahead and committed t

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-16 Thread Robert Haas
On Fri, Dec 16, 2016 at 12:28 PM, Andres Freund wrote: > On 2016-12-16 11:41:49 -0500, Robert Haas wrote: >> On Wed, Dec 14, 2016 at 3:25 PM, Robert Haas wrote: >> > Thoughts? >> >> Hearing no objections, I've gone ahead and committed this. If that >> makes somebody really unhappy I can revert i

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-16 Thread Andres Freund
On 2016-12-16 11:41:49 -0500, Robert Haas wrote: > On Wed, Dec 14, 2016 at 3:25 PM, Robert Haas wrote: > > Thoughts? > > Hearing no objections, I've gone ahead and committed this. If that > makes somebody really unhappy I can revert it, but I am betting that > the real story is that nobody cares

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-16 Thread Robert Haas
On Wed, Dec 14, 2016 at 3:25 PM, Robert Haas wrote: > Thoughts? Hearing no objections, I've gone ahead and committed this. If that makes somebody really unhappy I can revert it, but I am betting that the real story is that nobody cares about preserving T_ID(). -- Robert Haas EnterpriseDB: http

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-14 Thread Robert Haas
On Mon, Dec 5, 2016 at 3:12 PM, Robert Haas wrote: > On Thu, Dec 1, 2016 at 6:35 AM, Thomas Munro > wrote: >> On Sat, Nov 26, 2016 at 1:55 AM, Thomas Munro >> wrote: >>> Here's a new version to apply on top of dsa-v7.patch. >> >> Here's a version to go with dsa-v8.patch. > > Thomas has spent a f

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-05 Thread Robert Haas
On Thu, Dec 1, 2016 at 6:35 AM, Thomas Munro wrote: > On Sat, Nov 26, 2016 at 1:55 AM, Thomas Munro > wrote: >> Here's a new version to apply on top of dsa-v7.patch. > > Here's a version to go with dsa-v8.patch. Thomas has spent a fair amount of time beating me up off-list about the fact that we

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-04 Thread Haribabu Kommi
On Thu, Dec 1, 2016 at 10:35 PM, Thomas Munro wrote: > On Sat, Nov 26, 2016 at 1:55 AM, Thomas Munro > wrote: > > Here's a new version to apply on top of dsa-v7.patch. > > Here's a version to go with dsa-v8.patch. Moved to next CF with "needs review" status. Regards, Hari Babu Fujitsu Austral

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-01 Thread Thomas Munro
On Sat, Nov 26, 2016 at 1:55 AM, Thomas Munro wrote: > Here's a new version to apply on top of dsa-v7.patch. Here's a version to go with dsa-v8.patch. -- Thomas Munro http://www.enterprisedb.com dsa-area-for-executor-v4.patch Description: Binary data -- Sent via pgsql-hackers mailing list (

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-11-25 Thread Thomas Munro
On Fri, Nov 25, 2016 at 4:32 AM, Dilip Kumar wrote: > I have one more question, > > In V1 we were calling dsa_detach in ExecParallelCleanup and in > ParallelQueryMain, but it's removed in v2. > > Any specific reason ? > Does this need to be used differently ? > > ExecParallelCleanup(ParallelExecu

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-11-24 Thread Dilip Kumar
I have one more question, In V1 we were calling dsa_detach in ExecParallelCleanup and in ParallelQueryMain, but it's removed in v2. Any specific reason ? Does this need to be used differently ? ExecParallelCleanup(ParallelExecutorInfo *pei) { + if (pei->area != NULL) + { + dsa_detach(pei->area

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-11-24 Thread Dilip Kumar
On Wed, Nov 23, 2016 at 5:42 PM, Thomas Munro wrote: > ... or we could allow DSA areas to be constructed inside existing > shmem, as in the attached patch which requires dsa_create_in_place, > from the patch at > https://www.postgresql.org/message-id/CAEepm%3D0-pbokaQdCXhtYn%3Dw64MmdJz4hYW7qcSU235

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-11-23 Thread Thomas Munro
On Wed, Oct 5, 2016 at 10:32 AM, Thomas Munro wrote: > One obvious problem is that this patch results in at least *two* DSM > segments being created for every parallel query execution: the main > segment used for parallel execution, and then the initial segment > managed by the DSA area. One thou

[HACKERS] Creating a DSA area to provide work space for parallel execution

2016-10-04 Thread Thomas Munro
Hi hackers, A couple of months ago I proposed dynamic shared areas[1]. DSA areas are dynamically sized shared memory heaps that backends can use to share data, building on top of the existing DSM infrastructure. One target use case for DSA areas is to provide work space for parallel query execut