Re: Copy data to DSA area

2020-02-01 Thread Tomas Vondra
I've marked this patch as returned with feedback. It's been sitting in the CF for a long time without any update (and does not apply anymore). That does not mean we don't want the feature (it seems useful) so feel free to resubmit an updated patch. regards -- Tomas Vondra

Re: Copy data to DSA area

2019-11-30 Thread Michael Paquier
On Fri, Oct 18, 2019 at 12:53:16PM +, ideriha.take...@fujitsu.com wrote: > I added ShmZoneContext to my patch. > I haven't added detailed comments and test set, so let me explain how to > use it here. I followed Thomas' suggestion. The latest patch sent fails to apply. Could you please send

RE: Copy data to DSA area

2019-10-18 Thread ideriha.take...@fujitsu.com
>>ShmZoneContext for SharedPlan and SharedRelCache is not implemented but >>I'm going to do it following your points. > >After looking into existing code, I'm thinking Generation Memory Context seems >to >have the similar purpose. So I'll implement ShmZoneContext by reference it. >Generation

RE: Copy data to DSA area

2019-10-16 Thread ideriha.take...@fujitsu.com
Hi, >ShmZoneContext for SharedPlan and SharedRelCache is not implemented but I'm >going to do it following your points. After looking into existing code, I'm thinking Generation Memory Context seems to have the similar purpose. So I'll implement ShmZoneContext by reference it. Generation

RE: Copy data to DSA area

2019-10-15 Thread ideriha.take...@fujitsu.com
Hi, Sorry for waiting. >Thomas Munro wrote: >>What do you think about the following? Even though I know you want to >>start with much simpler kinds of cache, I'm looking ahead to the lofty >>end-goal of having a shared plan cache. No doubt, that involves >>solving many other problems that

Re: Copy data to DSA area

2019-09-25 Thread Alvaro Herrera
Should now this be closed as Returned with Feedback, or are we really waiting on an updated patch in the short term? -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

RE: Copy data to DSA area

2019-07-12 Thread Ideriha, Takeshi
Hi, thank you for the previous two emails. Thomas Munro wrote: >What do you think about the following? Even though I know you want to start >with >much simpler kinds of cache, I'm looking ahead to the lofty end-goal of having >a shared >plan cache. No doubt, that involves solving many other

Re: Copy data to DSA area

2019-07-10 Thread Thomas Munro
On Wed, Jul 10, 2019 at 6:03 PM Thomas Munro wrote: > Hmm. I wonder if we should just make ShmContextFree() do nothing! And > make ShmContextAlloc() allocate (say) 8KB chunks (or larger if needed > for larger allocation) and then hand out small pieces from the > 'current' chunk as needed. Then

Re: Copy data to DSA area

2019-07-10 Thread Thomas Munro
On Tue, Jun 25, 2019 at 12:53 AM Ideriha, Takeshi wrote: > I've rebased the patch to fit the core code rather than extension. > Regarding shared memory context (ShmContext), I added three > APIs: > - CreatePermShmContext > create "permanent" context located in shared memory > -

RE: Copy data to DSA area

2019-06-24 Thread Ideriha, Takeshi
>From: Ideriha, Takeshi [mailto:ideriha.take...@jp.fujitsu.com] >Sent: Friday, April 26, 2019 11:50 PM >To: 'Kyotaro HORIGUCHI' ; >thomas.mu...@enterprisedb.com; robertmh...@gmail.com >Cc: pgsql-hack...@postgresql.org >Subject: RE: Copy data to DSA area > >Hi, I've updated

RE: Copy data to DSA area

2019-05-14 Thread Ideriha, Takeshi
Hi, >>From: Thomas Munro [mailto:thomas.mu...@gmail.com] >>Subject: Re: Copy data to DSA area >> >>On Wed, May 8, 2019 at 5:29 PM Ideriha, Takeshi >> >>wrote: >>> >From: Ideriha, Takeshi [mailto:ideriha.take...@jp.fujitsu.com] >>> >Sent

RE: Copy data to DSA area

2019-05-08 Thread Ideriha, Takeshi
Hi, Thomas >-Original Message- >From: Thomas Munro [mailto:thomas.mu...@gmail.com] >Subject: Re: Copy data to DSA area > >On Wed, May 8, 2019 at 5:29 PM Ideriha, Takeshi > >wrote: >> >From: Ideriha, Takeshi [mailto:ideriha.take...@jp.fujitsu.com] >>

Re: Copy data to DSA area

2019-05-08 Thread Thomas Munro
On Wed, May 8, 2019 at 5:29 PM Ideriha, Takeshi wrote: > >From: Ideriha, Takeshi [mailto:ideriha.take...@jp.fujitsu.com] > >Sent: Friday, April 26, 2019 11:50 PM > >Well, after developing PoC, I realized that this PoC doesn't solve the local > >process is > >crashed before the context becomes

RE: Copy data to DSA area

2019-05-07 Thread Ideriha, Takeshi
Hi, >From: Ideriha, Takeshi [mailto:ideriha.take...@jp.fujitsu.com] >Sent: Friday, April 26, 2019 11:50 PM >Well, after developing PoC, I realized that this PoC doesn't solve the local >process is >crashed before the context becomes shared because local process keeps track of >pointer to

RE: Copy data to DSA area

2019-04-26 Thread Ideriha, Takeshi
Hi, I've updated Thomas's quick PoC. >From: Ideriha, Takeshi [mailto:ideriha.take...@jp.fujitsu.com] >Sent: Wednesday, April 17, 2019 2:07 PM >>From: Ideriha, Takeshi [mailto:ideriha.take...@jp.fujitsu.com] >>Sent: Wednesday, December 5, 2018 2:42 PM >>Subject: RE: Copy da

RE: Copy data to DSA area

2019-04-16 Thread Ideriha, Takeshi
>From: Ideriha, Takeshi [mailto:ideriha.take...@jp.fujitsu.com] >Sent: Wednesday, December 5, 2018 2:42 PM >Subject: RE: Copy data to DSA area Hi It's been a long while since we discussed this topic. Let me recap first and I'll give some thoughts. It seems things we got consensus is

RE: Copy data to DSA area

2018-12-04 Thread Ideriha, Takeshi
Hi, thank you for the comment. >-Original Message- >From: Kyotaro HORIGUCHI [mailto:horiguchi.kyot...@lab.ntt.co.jp] >Sent: Wednesday, November 28, 2018 5:09 PM > >Hello. > >At Wed, 28 Nov 2018 05:13:26 +, "Ideriha, Takeshi" > wrote in

Re: Copy data to DSA area

2018-11-28 Thread Kyotaro HORIGUCHI
Hello. At Wed, 28 Nov 2018 05:13:26 +, "Ideriha, Takeshi" wrote in <4E72940DA2BF16479384A86D54D0988A6F3BD73A@G01JPEXMBKW04> > Hi > > >From: Thomas Munro [mailto:thomas.mu...@enterprisedb.com] > >Here we need something better, because once this new kind of memory fills up > >with >

RE: Copy data to DSA area

2018-11-27 Thread Ideriha, Takeshi
Hi >From: Thomas Munro [mailto:thomas.mu...@enterprisedb.com] >Sent: Wednesday, November 14, 2018 9:50 AM >To: Ideriha, Takeshi/出利葉 健 > >On Tue, Nov 13, 2018 at 10:59 PM Ideriha, Takeshi > >wrote: >> Can I check my understanding? >> The situation you are talking about is the following: >> Data

Re: Copy data to DSA area

2018-11-13 Thread Thomas Munro
On Tue, Nov 13, 2018 at 10:59 PM Ideriha, Takeshi wrote: > Can I check my understanding? > The situation you are talking about is the following: > Data structure A and B will be allocated on DSA. Data A has a pointer to B. > Another data X is already allocated on some area (might be on local

RE: Copy data to DSA area

2018-11-13 Thread Ideriha, Takeshi
From: Thomas Munro [mailto:thomas.mu...@enterprisedb.com] > >On Tue, Nov 13, 2018 at 9:45 AM Robert Haas wrote: >> On Thu, Nov 8, 2018 at 9:05 PM Thomas Munro >> wrote: >> > * I had some ideas about some kind of "allocation rollback" interface: >> > you begin an "allocation transaction",

RE: Copy data to DSA area

2018-11-13 Thread Ideriha, Takeshi
Thank you for the comment. From: Thomas Munro [mailto:thomas.mu...@enterprisedb.com] >> I'm thinking to go with plan 1. No need to think about address >> translation seems tempting. Plan 2 (as well as plan 3) looks a big project. > >The existing function dsa_create_in_place() interface was

Re: Copy data to DSA area

2018-11-12 Thread Thomas Munro
On Tue, Nov 13, 2018 at 9:45 AM Robert Haas wrote: > On Thu, Nov 8, 2018 at 9:05 PM Thomas Munro > wrote: > > * I had some ideas about some kind of "allocation rollback" interface: > > you begin an "allocation transaction", allocate a bunch of stuff > > (perhaps indirectly, by calling some API

Re: Copy data to DSA area

2018-11-12 Thread Robert Haas
On Thu, Nov 8, 2018 at 9:05 PM Thomas Munro wrote: > * I had some ideas about some kind of "allocation rollback" interface: > you begin an "allocation transaction", allocate a bunch of stuff > (perhaps indirectly, by calling some API that makes query plans or > whatever and is totally unaware of

Re: Copy data to DSA area

2018-11-08 Thread Thomas Munro
On Fri, Nov 9, 2018 at 2:19 PM Ideriha, Takeshi wrote: > From: Thomas Munro [mailto:thomas.mu...@enterprisedb.com] > >I know of 3 ideas that would make your idea C work, so that you could share > >something as complicated as a query plan directly without having to > >deserialise it to > >use it:

RE: Copy data to DSA area

2018-11-08 Thread Ideriha, Takeshi
Hi, thank you for all the comment. It's really helpful. >From: Thomas Munro [mailto:thomas.mu...@enterprisedb.com] >Sent: Wednesday, November 7, 2018 1:35 PM > >On Wed, Nov 7, 2018 at 3:34 PM Ideriha, Takeshi > >wrote: >> Related to my development (putting relcache and catcache onto shared >>

Re: Copy data to DSA area

2018-11-06 Thread Thomas Munro
On Wed, Nov 7, 2018 at 3:34 PM Ideriha, Takeshi wrote: > Related to my development (putting relcache and catcache onto shared > memory)[1], > > I have some questions about how to copy variables into shared memory, > especially DSA area, and its implementation way. > > Under the current