Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan [take-2]

2017-03-24 Thread David Steele
Hi Kaigai, On 3/21/17 1:11 PM, David Steele wrote: On 3/13/17 3:25 AM, Jeevan Chalke wrote: I have reviewed this patch further and here are my comments: This thread has been idle for over a week. Please respond and/or post a new patch by 2017-03-24 00:00 AoE (UTC-12) or this submission will

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan [take-2]

2017-03-21 Thread David Steele
Hi, On 3/13/17 3:25 AM, Jeevan Chalke wrote: I have reviewed this patch further and here are my comments: This thread has been idle for over a week. Please respond and/or post a new patch by 2017-03-24 00:00 AoE (UTC-12) or this submission will be marked "Returned with Feedback". Thanks,

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan [take-2]

2017-03-13 Thread Jeevan Chalke
Hi, I have reviewed this patch further and here are my comments: 1. Will it be better to use compare_pathkeys() instead of equal(root->query_pathkeys, pathkeys)? 2. I think it will be good if we add a simple test-case in postgres_fdw.sql which shows that LIMIT is passed to remote server. We migh

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan [take-2]

2017-03-12 Thread Kouhei Kaigai
> Hello, > > On Wed, March 1, 2017 7:21 pm, Kouhei Kaigai wrote: > >> I've looked at the patch, and as I'm not that familiar with the > >> pg-sourcecode, customs and so on, this isn't a review, more like food > >> for thought and all should be taken with a grain of salt. :) > >> > >> So here are a

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan [take-2]

2017-03-10 Thread Tels
Hello, On Wed, March 1, 2017 7:21 pm, Kouhei Kaigai wrote: >> I've looked at the patch, and as I'm not that familiar with the >> pg-sourcecode, >> customs and so on, this isn't a review, more like food for thought and >> all >> should be taken with a grain of salt. :) >> >> So here are a few quest

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan [take-2]

2017-03-01 Thread Kouhei Kaigai
> Hello all, > > as this is my first mail to pgsql-hackers, please be gentle :) > Welcome to pgsql-hackers, > I've looked at the patch, and as I'm not that familiar with the pg-sourcecode, > customs and so on, this isn't a review, more like food for thought and all > should be taken with a grain

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan [take-2]

2017-03-01 Thread Tels
Hello all, as this is my first mail to pgsql-hackers, please be gentle :) I've looked at the patch, and as I'm not that familiar with the pg-sourcecode, customs and so on, this isn't a review, more like food for thought and all should be taken with a grain of salt. :) So here are a few questions

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan [take-2]

2017-02-28 Thread Kouhei Kaigai
Of Kohei KaiGai > Sent: Tuesday, January 03, 2017 12:07 PM > To: Kaigai Kouhei(海外 浩平) > Cc: Jeevan Chalke ; Robert Haas > ; pgsql-hackers@postgresql.org; Etsuro Fujita > ; Andres Freund > Subject: Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan > [take-2] > &

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan [take-2]

2017-01-04 Thread Michael Paquier
On Thu, Jan 5, 2017 at 8:58 AM, Kouhei Kaigai wrote: > Unfortunately, it was not possible. Probably, administrator privilege will be > needed for this operation. Adding a patch to a CF in progress indeed requires administrator privileges, > May I add it to the CF:2017-03? I can notice that the

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan [take-2]

2017-01-04 Thread Kouhei Kaigai
ostgresql.org; Etsuro > Fujita ; Andres Freund > Subject: Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan > [take-2] > > On Mon, Jan 2, 2017 at 10:07 PM, Kohei KaiGai wrote: > > Oops, I oversight this patch was marked as "returned with feedback", > > n

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan [take-2]

2017-01-04 Thread Robert Haas
On Mon, Jan 2, 2017 at 10:07 PM, Kohei KaiGai wrote: > Oops, I oversight this patch was marked as "returned with feedback", > not "moved to the next CF". > > Its status has not been changed since the last update. (Code was > revised according to the last > comment by Jeevan, but CF-Nov was time up

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan [take-2]

2017-01-02 Thread Kohei KaiGai
Oops, I oversight this patch was marked as "returned with feedback", not "moved to the next CF". Its status has not been changed since the last update. (Code was revised according to the last comment by Jeevan, but CF-Nov was time up at that time.) How do I handle the patch? 2016-12-05 16:49 GMT

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan [take-2]

2016-12-04 Thread Kouhei Kaigai
Hello, Sorry for my late response. The attached patch reflects your comments. > Here are few comments on latest patch: > > > 1. > make/make check is fine, however I am getting regression failure in > postgres_fdw contrib module (attached regression.diff). > Please investigate and fix. > It was

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan [take-2]

2016-12-01 Thread Haribabu Kommi
On Thu, Nov 10, 2016 at 10:59 AM, Kouhei Kaigai wrote: > > On Tue, Nov 8, 2016 at 6:54 AM, Jeevan Chalke > > wrote: > > > 1. ps_numTuples is declared as long, however offset and count members > in > > > LimitState struct and bound member in SortState struct is int64. > However > > > long on 32 b

[HACKERS] PassDownLimitBound for ForeignScan/CustomScan [take-2]

2016-11-24 Thread Jeevan Chalke
On Mon, Nov 21, 2016 at 1:59 PM, Kouhei Kaigai wrote: > Hello, > > The attached patch is a revised version of pass-down LIMIT to FDW/CSP. > > Below is the updates from the last version. > > 'ps_numTuples' of PlanState was declared as uint64, instead of long > to avoid problems on 32bits machine w

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan [take-2]

2016-11-21 Thread Kouhei Kaigai
Hello, The attached patch is a revised version of pass-down LIMIT to FDW/CSP. Below is the updates from the last version. 'ps_numTuples' of PlanState was declared as uint64, instead of long to avoid problems on 32bits machine when a large LIMIT clause is supplied. 'ps_numTuples' is re-interpret

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan [take-2]

2016-11-09 Thread Kouhei Kaigai
> On Tue, Nov 8, 2016 at 6:54 AM, Jeevan Chalke > wrote: > > 1. ps_numTuples is declared as long, however offset and count members in > > LimitState struct and bound member in SortState struct is int64. However > > long on 32 bit machine may be 32 bits and thus I think tuples_needed which > > is

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan [take-2]

2016-11-09 Thread Kouhei Kaigai
ita; Andres Freund > Subject: Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan [take-2] > > On Mon, Oct 31, 2016 at 10:20 AM, Kouhei Kaigai wrote: > > As an example, I enhanced postgres_fdw to understand the ps_numTuples > > if it is set. If and when remote ORDER BY is

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan [take-2]

2016-11-09 Thread Robert Haas
On Tue, Nov 8, 2016 at 6:54 AM, Jeevan Chalke wrote: > 1. ps_numTuples is declared as long, however offset and count members in > LimitState struct and bound member in SortState struct is int64. However > long on 32 bit machine may be 32 bits and thus I think tuples_needed which > is long may hav

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan [take-2]

2016-11-09 Thread Robert Haas
On Mon, Oct 31, 2016 at 10:20 AM, Kouhei Kaigai wrote: > As an example, I enhanced postgres_fdw to understand the ps_numTuples > if it is set. If and when remote ORDER BY is pushed down, the latest > code tries to sort the entire remote table because it does not know > how many rows to be returned

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan [take-2]

2016-11-08 Thread Jeevan Chalke
Hi, I have reviewed the patch. Patch applies cleanly. make/"make install"/"make check" all are fine. I too see a good performance in execution time when LIMIT is pushed with cursor query in postgres_fdw at execution time However here are few comments on the changes: 1. ps_numTuples is declared

[HACKERS] PassDownLimitBound for ForeignScan/CustomScan [take-2]

2016-10-31 Thread Kouhei Kaigai
aigai Kouhei(海外 浩平) > Cc: Jeevan Chalke; pgsql-hackers@postgresql.org; Etsuro Fujita; Andres Freund > Subject: ##freemail## Re: [HACKERS] PassDownLimitBound for > ForeignScan/CustomScan > > On Tue, Sep 13, 2016 at 9:07 PM, Kouhei Kaigai wrote: > > In the current implementatio

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan

2016-09-28 Thread Robert Haas
On Tue, Sep 13, 2016 at 9:07 PM, Kouhei Kaigai wrote: > It is because of just my time pressure around the patch submission days. > I'll try to enhance postgres_fdw as a usage of this run-time optimization. Time has (pretty much) expired for this CommitFest. In any case, this will amount to a who

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan

2016-09-15 Thread Robert Haas
On Tue, Sep 13, 2016 at 9:07 PM, Kouhei Kaigai wrote: > In the current implementation calls recompute_limits() on the first > invocation of ExecLimit and ExecReScanLimit. Do we expect the > ps->numTuples will be also passed down to the child nodes on the same > timing? Sure, unless we find some r

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan

2016-09-13 Thread Kouhei Kaigai
> On Tue, Sep 13, 2016 at 3:48 AM, Kouhei Kaigai wrote: > > Sorry for my late. > > > > The attached patch fixed the wording problems on SGML part. > > I agree that we should have some way for foreign data wrappers and > custom scans and perhaps also other executor nodes to find out whether > ther

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan

2016-09-13 Thread Tom Lane
Robert Haas writes: > I agree that we should have some way for foreign data wrappers and > custom scans and perhaps also other executor nodes to find out whether > there's a known limit to the number of tuples that they might need to > produce, but I wonder if we should be doing something more gen

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan

2016-09-13 Thread Robert Haas
On Tue, Sep 13, 2016 at 3:48 AM, Kouhei Kaigai wrote: > Sorry for my late. > > The attached patch fixed the wording problems on SGML part. I agree that we should have some way for foreign data wrappers and custom scans and perhaps also other executor nodes to find out whether there's a known limi

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan

2016-09-13 Thread Kouhei Kaigai
11:22 PM > To: Kaigai Kouhei(海外 浩平) > Cc: pgsql-hackers@postgresql.org; Etsuro Fujita > Subject: Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan > > Hi, > > Changes look good to me. > > However there are couple of minor issues need to be fixed. >

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan

2016-09-06 Thread Jeevan Chalke
Hi, Changes look good to me. However there are couple of minor issues need to be fixed. 1. "under" repeated on second line. Please remove. +if and when CustomScanState is located under +under LimitState; which implies the underlying node is not 2. Typo: dicsussion => discussion Please f

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan

2016-09-05 Thread Kouhei Kaigai
Subject: Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan > > > On Mon, Aug 29, 2016 at 7:25 AM, Kouhei Kaigai wrote: > > > > > > Hello, > > > > The attached patch adds an optional callback to support special > > optimization > >

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan

2016-09-04 Thread Kouhei Kaigai
> On Mon, Aug 29, 2016 at 7:25 AM, Kouhei Kaigai wrote: > > > Hello, > > The attached patch adds an optional callback to support special > optimization > if ForeignScan/CustomScan are located under the Limit node in plan-tree. > > Our sort node wisely switches the beha

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan

2016-09-02 Thread Jeevan Chalke
Hi, On Mon, Aug 29, 2016 at 7:25 AM, Kouhei Kaigai wrote: > Hello, > > The attached patch adds an optional callback to support special > optimization > if ForeignScan/CustomScan are located under the Limit node in plan-tree. > > Our sort node wisely switches the behavior when we can preliminary

[HACKERS] PassDownLimitBound for ForeignScan/CustomScan

2016-08-28 Thread Kouhei Kaigai
Hello, The attached patch adds an optional callback to support special optimization if ForeignScan/CustomScan are located under the Limit node in plan-tree. Our sort node wisely switches the behavior when we can preliminary know exact number of rows to be produced, because all the Sort node has t