On Fri, Feb 8, 2019 at 1:34 AM, I wrote:
> On Wed, Feb 6, 2019 at 2:04 AM, Tsunakawa, Takayuki wrote:
> > Can you compare the performance of auto and force_custom_plan again
> > with the attached patch? It uses PGPROC's LOCALLOCK list instead of
> > the hash table.
>
> Thanks for the patch, but i
Amit-san
Sorry for my late reply. I had another work to do.
On Fri, Feb 8, 2019 at 9:13 AM, Amit Langote wrote:
> On 2019/02/08 13:44, Imai, Yoshikazu wrote:
> > 3.
> > 0001: line 1919-1920
> >
> > - case CONSTRAINT_EXCLUSION_ON:
&g
On Mon, Feb 18, 2019 at 5:28 PM, Tom Lane wrote:
> Frankly, that code is just horrid. Having a function with side effects
> in an if-test is questionable at the best of times, and having it be the
> second of three conditions (which the third condition silently depends
> on) is unreadable and unma
Hi Amit-san.
On Fri, Feb 22, 2019 at 5:55 PM, Amit Langote wrote:
>
> Please find attached updated patches. I've made a few updates in last
> couple of hours such as improving comments, fixing a few thinkos in
> inheritance_planner changes, etc.
Thanks for the patch.
While doing code review o
Hi,
On Tue, Feb 19, 2019 at 8:20 PM, Euler Taveira wrote:
> Em seg, 18 de fev de 2019 às 19:21, Tom Lane escreveu:
> >
> > Euler Taveira writes:
> > > Since no one has stepped up, I took a stab at it. It will prohibit
> > > standard output unless '-f -' be specified. -l option also has the
> > >
Hosoya-san
On Wed, Feb 27, 2019 at 6:51 AM, Yuzuko Hosoya wrote:
> > From: Amit Langote [mailto:langote_amit...@lab.ntt.co.jp]
> > Sent: Wednesday, February 27, 2019 11:22 AM
> >
> > Hosoya-san,
> >
> > On 2019/02/22 17:14, Yuzuko Hosoya wrote:
> > > Hi,
> > >
> > > I found the bug of default part
Amit-san,
On Fri, Mar 1, 2019 at 1:02 PM, Amit Langote wrote:
> Thanks for testing and sorry it took me a while to reply.
Thanks for working about this late at night. I know you have a lot of things to
do.
> On 2019/02/25 15:24, Imai, Yoshikazu wrote:
> > [update_pt_with_joining_an
Amit-san,
On Tue, Mar 5, 2019 at 0:51 AM, Amit Langote wrote:
> Hi Jesper,
>
> Thanks for the review. I've made all of the changes per your comments
> in my local repository. I'll post the updated patches after diagnosing
> what I'm suspecting a memory over-allocation bug in one of the patches.
Amit-san,
On Tue, Mar 5, 2019 at 10:24 AM, Amit Langote wrote:
> On 2019/03/05 9:50, Amit Langote wrote:
> > I'll post the updated patches after diagnosing what I'm suspecting a
> > memory over-allocation bug in one of the patches. If you configure
> > build with --enable-cassert, you'll see that
On Wed, Mar 6, 2019 at 2:10 AM, Imai, Yoshikazu wrote:
> > and Imai-san's review. I haven't been able to pin down the bug (or
> > whatever) that makes throughput go down as the partition count
> > increases, when tested with a --enable-cassert build.
>
> I di
Amit-san,
On Wed, Mar 6, 2019 at 5:38 AM, Amit Langote wrote:
> On 2019/03/06 11:09, Imai, Yoshikazu wrote:
> > [0004 or 0005]
> > There are redundant process in add_appendrel_other_rels (or
> expand_xxx_rtentry()?).
> > I modified add_appendrel_other_rels like belo
Amit-san,
On Wed, Mar 6, 2019 at 5:38 AM, Amit Langote wrote:
...
> > I didn't investigate that problem, but there is another memory
> > increase
> issue, which is because of 0003 patch I think. I'll try to solve the latter
> issue.
>
> Interested in details as it seems to be a separate problem.
Amit-san,
On Fri, Mar 8, 2019 at 9:18 AM, Amit Langote wrote:
> On 2019/03/08 16:16, Imai, Yoshikazu wrote:
> > So I modified the code and did test to confirm memory increasement don't
> happen. The test and results are below.
> >
> > [test]
> > * Create par
Amit-san,
On Tue, Mar 12, 2019 at 2:34 PM, Amit Langote wrote:
> Thanks for the heads up.
>
> On Tue, Mar 12, 2019 at 10:23 PM Jesper Pedersen
> wrote:
> > After applying 0004 check-world fails with the attached. CFBot agrees
> [1].
>
> Fixed. I had forgotten to re-run postgres_fdw tests after
Amit-san,
I have done code review of v31 patches from 0004 to 0008.
0004:
* s/childern/children
0005:
* This seems reasonable for not using a lot of memory in specific case,
although it needs special looking of planner experts.
0006:
* The codes initializing/setting RelOptInfo's part_rels look
), we do ChangeVarNodes() only for orig_rtable,
> so the codes concatenating lists of append_rel_list may be able to be
> moved before doing ChangeVarNodes() and then the codes concatenating
> lists of rowmarks, rtable and append_rel_list can be in one block (or
> one bunch).
>
> Ye
Hi, David
On Thu, Mar 14, 2019 at 9:04 AM, David Rowley wrote:
> On Thu, 14 Mar 2019 at 21:35, Imai, Yoshikazu
> wrote:
> > 0007:
> > * This changes some processes using "for loop" to using
> "while(bms_next_member())" which speeds up processing when
Amit-san,
I have another little comments about v31-patches.
* We don't need is_first_child in inheritance_planner().
On Fri, Mar 8, 2019 at 9:18 AM, Amit Langote wrote:
> On 2019/03/08 16:16, Imai, Yoshikazu wrote:
> > I attached the diff of modification for v26-0003 patch which a
Hi Jose,
Sorry for my late reply.
On Wed, Mar 6, 2019 at 10:58 AM, José Arthur Benetasso Villanova wrote:
> On Thu, 28 Feb 2019, Imai, Yoshikazu wrote:
>
> > Is there no need to rewrite the Description in the Doc to state we should
> specify either -d or -f option?
> >
On Sat, Mar 16, 2019 at 10:55 PM, Euler Taveira wrote:
> > Is there no need to rewrite the Description in the Doc to state we should
> specify either -d or -f option?
> > (and also it might be better to write if -l option is given, neither
> > -d nor -f option isn't necessarily needed.)
> >
> I don
On Fri, Mar 15, 2019 at 6:20 AM, Imai, Yoshikazu wrote:
> Upon committing this, we have to care this patch break backwards
> compatibility, but I haven't seen any complaints so far. If there are
> no objections, I will set this patch to ready for committer.
Jose had set thi
Amit-san,
On Mon, Mar 18, 2019 at 9:56 AM, Amit Langote wrote:
> On 2019/03/15 14:40, Imai, Yoshikazu wrote:
> > Amit-san,
> >
> > I have another little comments about v31-patches.
> >
> > * We don't need is_first_child in inheritance_planner().
> &
Hi Tsunakawa-san, Peter
On Tue, Mar 19, 2019 at 7:53 AM, Tsunakawa, Takayuki wrote:
> From: Peter Eisentraut [mailto:peter.eisentr...@2ndquadrant.com]
> > You posted a link to some performance numbers, but I didn't see the
> > test setup explained there. I'd like to get some more information on
>
Amit-san,
On Tue, Mar 19, 2019 at 6:53 AM, Amit Langote wrote:
> On 2019/03/15 9:33, Imai, Yoshikazu wrote:
> > On Thu, Mar 14, 2019 at 9:04 AM, Amit Langote wrote:
> >>> * In inheritance_planner(), we do ChangeVarNodes() only for
> >>> orig_rtable,
> &g
Amit-san,
On Wed, Mar 20, 2019 at 0:42 AM, Amit Langote wrote:
> On 2019/03/19 20:13, Imai, Yoshikazu wrote:
> > Thanks for new patches.
> > I looked over them and there are little comments.
> >
> > ...
> >
> > I have no more comments about codes ot
Amit-san,
On Wed, Mar 20, 2019 at 2:34 AM, Amit Langote wrote:
> On 2019/03/20 11:21, Imai, Yoshikazu wrote:
> > (4)
> > We expect the performance does not depend on the number of partitions
> after applying all patches, if possible.
> >
> > num of partTPS
&
Amit-san,
On Wed, Mar 20, 2019 at 3:01 PM, Amit Langote wrote:
> > On Wed, Mar 20, 2019 at 2:34 AM, Amit Langote wrote:
> >> On 2019/03/20 11:21, Imai, Yoshikazu wrote:
> >>> (4)
> >>> We expect the performance does not depend on the number of
> >>
Amit-san,
On Wed, Mar 20, 2019 at 8:21 AM, Amit Langote wrote:
> On 2019/03/20 12:15, Imai, Yoshikazu wrote:
> > Here the details.
> >
> > [creating partitioned tables (with 1024 partitions)] drop table if
> > exists rt; create table rt (a int, b int, c int) partitio
Amit-san,
On Wed, Mar 20, 2019 at 9:07 AM, Amit Langote wrote:
> On 2019/03/20 17:36, Imai, Yoshikazu wrote:
> > On Wed, Mar 20, 2019 at 8:21 AM, Amit Langote wrote:
> >> On 2019/03/20 12:15, Imai, Yoshikazu wrote:
> >>> [select1024.sql]
> >>> \set a ran
Hi, Amit,
On Fri, Dec 7, 2018 at 0:57 AM, Imai, Yoshikazu wrote:
> OK. I will continue the review of 0001 before/after your supplying of
> next patch with keeping those in mind.
Here's the continuation of the review. Almost all of below comments are
little fixes.
---
0001: line 76-7
Hi,
I ran the performance tests for no prepared query and for prepared query with
plan_cache_mode='auto' and plan_cache_mode='force_custom_plan'. I also changed
number of partitions as 256 or 4096. I ran the tests on master and v9-patched.
[settings]
plan_cache_mode = 'auto' or 'force_custom_plan
Hi Amit,
On Mon, Jan 7, 2019 at 6:30 PM, Amit Langote wrote:
> On 2018/12/27 20:25, Amit Langote wrote:
> > Here's the v10 of the patch. I didn't get chance to do more changes
> > than those described above and address Imai-san's review comments
> yesterday.
> >
> > Have a wonderful new year! I'
Hi David,
On Thu, Jan 10, 2019 at 4:02 PM, David Rowley wrote:
> 3. I wonder if there's a better way to handle what
> build_dummy_partition_rel() does. I see the child's relid to the
> parent's relid and makes up a fake AppendRelInfo and puts it in the
> parent's slot. What's going to happen when
On Thu, Jan 10, 2019 at 6:10 PM, Imai, Yoshikazu wrote:
> > Does that not mean that the if (parent->top_parent_relids) will always
> > be false in build_dummy_partition_rel() and it'll only ever get
> > rtekind == RTE_RELATION?
>
> At least, I checked if (parent-&g
Hi Amit-san,
On Wed, Jan 17, 2019 at 10:25 AM, Amit Langote wrote:
> Thank you Imai-san for testing. Sorry it totally slipped my mind to reply to
> this email.
Thanks for replying and sorry for my late reply. I've been undertaking
on-the-job training last week.
> Are you saying that, when usi
Tsunakawa-san
On Thu, Jan 18, 2019 at 5:29 AM, Amit Langote wrote:
> On 2019/01/18 14:12, Tsunakawa, Takayuki wrote:
...
> > Isn't CheckCachedPlan() (and AcquireExecutorLocks() therein) called
> in every EXECUTE after 6th one due to some unknow issue?
>
> CheckCachedPlan is only called if choose_
I measured the latency of queries executed before and after creating generic
plan with master + v15-patch.
In this test, table is partitioned into 4k partitions.
I executed 400,0001 queries by pgbench.
I changed the timing of creating generic plan at 1st, 10,001st, 20,001st,
50,001st, ..., 390,0
Tsunakawa-san
Thanks for giving the information.
I didn't use it yet, but I just used perf to clarify the difference of before
and after the creation of the generic plan, and I noticed that usage of
hash_seq_search() is increased about 3% in EXECUTE queries after the creation
of the generic pla
On Wed, Jan 23, 2019 at 1:35 AM, Amit Langote wrote:
> Rebased due to the heap_open/close() -> table_open/close() change.
Maybe there are not many things I can point out through reviewing the patch, so
I ran the performance test against v17 patches instead of reviewing codes.
There are already a
On Thu, Jan 24, 2019 at 6:10 AM, Imai, Yoshikazu wrote:
> updating partkey case:
>
> part-num master 0001 0002 0003 0004
> 18215.34 7924.99 7931.15 8407.40 8475.65
> 27137.49 7026.45 7128.84 7583.08 7593.73
> 45880.54 5896.47
Amit-san,
On Tue, Jan 29, 2019 at 10:11 AM, Amit Langote wrote:
> On 2019/01/28 10:44, Imai, Yoshikazu wrote:
> > On Thu, Jan 24, 2019 at 6:10 AM, Imai, Yoshikazu wrote:
> >> updating partkey case:
> >>
> >> part-num master 0001 0002 0003
On Wed, Jan 30, 2019 at 3:26 PM, Alvaro Herrera wrote:
> On 2019-Jan-30, Imai, Yoshikazu wrote:
>
> > Why I did these tests is that I wanted to confirm that even if we
> > apply each patch one by one, there's no performance problem. Because
> > patches are quite l
Tsunakawa-san,
On Wed, Feb 6, 2019 at 2:04 AM, Tsunakawa, Takayuki wrote:
> Can you compare the performance of auto and force_custom_plan again with
> the attached patch? It uses PGPROC's LOCALLOCK list instead of the hash
> table.
Thanks for the patch, but it seems to have some problems.
When I
Amit-san,
On Thu, Feb 7, 2019 at 10:22 AM, Amit Langote wrote:
> Rebased over bdd9a99aac.
I did code review of 0001 and I have some suggestions. Could you check them?
1.
0001: line 418
+* add_inherit_target_child_root() would've added only those
that are
add_inherit_target_chil
On Mon, June 11, 2018 at 4:31 PM, Alexander Korotkov wrote:
> On Mon, Jun 11, 2018 at 1:06 PM Simon Riggs wrote:
> > On 5 June 2018 at 14:45, Alexander Korotkov
> > wrote:
> > > Currently _bt_search() always locks leaf buffer in shared mode (aka
> > > BT_READ), while caller can relock it later.
On Mon, July 9, 2018 at 4:44 PM, Alexander Korotkov wrote:
> > Firstly, I did performance tests on 72-cores machines(AWS c5.18xlarge) same
> > as you did.
>
> OK. But not that c5.18xlarge is 72-VCPU machine, which AFAIK is
> close to the performance of 36 physical cores.
Thanks for pointing tha
On Mon, July 9, 2018 at 5:25 PM, Simon Riggs wrote:
> Please can you check insertion with the index on 2 keys
> 1st key has 10,000 values
> 2nd key has monotonically increasing value from last 1st key value
>
> So each session picks one 1st key value
> Then each new INSERTion is a higher value of
On Mon, July 9, 2018 at 3:19 AM, Imai, Yoshikazu wrote:
> I'm planning to do code review and send it in the next mail.
Sorry for delaying the code review.
I did the code review, and I think there are no logical wrongs
with B-Tree.
I tested integrity of B-Tree with amcheck just in
Hi.
On Wed, July 25, 2018 at 0:28 AM, Alexander Korotkov wrote:
> Hi!
>
> On Tue, Jul 10, 2018 at 4:39 PM 今井 良一 wrote:
> > On 2018/07/10 20:36, Alexander Korotkov wrote:
> > > Thank you for the experiments! It seems that there is real regression
> > > here... BTW, which script were you using in
On Wed, July 25, 2018 at 0:30 AM, Alexander Korotkov wrote:
> > Lefted tasks in my review is doing the regression tests.
>
> Cool, thank you for review!
>
I did "make check-world" in patched version and all tests were passed
successfully!
Yoshikazu Imai
Hi,
> At 2018-05-15 01:49:41, "Tom Lane" wrote:
> >=?GBK?B?19S8ug==?= writes:
> >> i run test using pg10.0 on my machine, and the program crashed on
> >> _bt_getbuf.
> >> And i found the following code:
> >> the routine _bt_page_recyclable say maybe the page is all-zero page,
> >> if so then th
Hi,
On Thu, Oct 25, 2018 at 10:38 PM, Amit Langote wrote:
> And here is the latest set of patches. Sorry it took me a while.
Thanks for revising the patch!
> I didn't get time today to repeat the performance tests, but I'm planning
> to next week. In the meantime, please feel free to test them
About inheritance_make_rel_from_joinlist(), I considered how it processes
joins for sub-partitioned-table.
sub-partitioned-table image:
part
sub1
leaf1
leaf2
inheritance_make_rel_from_joinlist() translates join_list and join_info_list
for each leafs(leaf1, leaf2 in above image). To tran
On Mon, Oct 22, 2018 at 8:31 PM, David Rowley wrote:
> On 18 October 2018 at 16:13, Imai, Yoshikazu
> wrote:
> > The patch improves the performance about 1.3% which is less than
> > David's result, but it seems still improves the performance.
>
> Thanks for doi
Hi Amit,
On Thu, Nov 8, 2018 at 8:26 PM, Amit Langote wrote:
> On 2018/11/07 10:00, Imai, Yoshikazu wrote:
> > About inheritance_make_rel_from_joinlist(), I considered how it processes
> > joins for sub-partitioned-table.
> >
> > sub-partitioned-table image:
> &
Hi Amit,
On Tue, Nov 13, 2018 at 10:29 PM, Amit Langote wrote:
> On 2018/11/12 13:35, Imai, Yoshikazu wrote:
> > adjust_appendrel_attrs_multilevel for leaf1: root -> sub1 -> leaf1
> > adjust_appendrel_attrs_multilevel for leaf2: root -> sub1 -> leaf2
>
> Ah, I
Hi Amit,
On Tue, Nov 20, 2018 at 10:24 PM, Amit Langote wrote:
> Attached v8 patches.
Thanks for the patch. I took a look 0003, 0005, 0006 of v8 patch.
1.
0003: line 267-268
+* Child relation may have be marked dummy if
build_append_child_rel
+* found self-contra
Hi, Amit
Thanks for the quick modification.
On Wed, Dec 5, 2018 at 8:26 PM, Amit Langote wrote:
> > 1.
...
> > 5.
> > 0003: line 1620-1621
> >
> > + * After creating the RelOptInfo for the given child RT index, it goes on
> > to
> > + * initialize some of its fields base on the parent RelOptInf
Hi, Amit!
On Thu, Sept 13, 2018 at 10:29 PM, Amit Langote wrote:
> Attached is what I have at the moment.
I also do the code review of the patch.
I could only see a v3-0001.patch so far, so below are all about v3-0001.patch.
I am new to inheritance/partitioning codes and code review, so my revie
Hi, David.
Thanks for the patch!
On Mon, Oct 8, 2018 at 1:00 AM, David Rowley wrote:
> I was looking at this again and I realised that we can completely skip
> the re-sequence of the subplan map when we're not going to perform any
> further pruning during execution.
I checked codes and I think so
On Tue, Oct 9, 2018 at 2:02 AM, David Rowley wrote:
> Yeah, so subplan_map is just an array that stores the List index of
> the Append or MergeAppend's subplans. When we perform run-time pruning
> during executor initialisation, if we prune away some of these
> subplans then we don't initialise tho
Hi, all.
I have a wonder about the behaviour of creating table which has a constraint
contradiction.
I created below table.
bugtest=# create table ct (a int, CHECK(a is not null and a >= 0 and a < 100
and a >= 200 and a < 300));
bugtest=# \d+ ct
Table "public
Thanks for replying!
On Tue, Oct 9, 2018 at 5:58 PM, Corey Huinker wrote:
> On Wed, Oct 10, 2018 at 1:44 AM David G. Johnston
> mailto:david.g.johns...@gmail.com> >
> wrote:
>
>
> On Tuesday, October 9, 2018, Imai, Yoshikazu
> mailto:imai.yoshik..
On Tue, Oct 9, 2018 at 6:01 PM, Amit Langote wrote:
> On 2018/10/10 14:25, Imai, Yoshikazu wrote:
> > Hi, all.
> >
> > I have a wonder about the behaviour of creating table which has a
> > constraint contradiction.
> >
> > I created below table.
> >
>
On Tue, Oct 9, 2018 at 1:24 AM, I wrote:
> On Mon, Oct 8, 2018 at 1:00 AM, David Rowley wrote:
> > I've attached an updated patch which skips the re-sequence work when
> > doing that is not required for anything.
>
>
>
> I saw the patch and it seems good to me about the codes.
> I still couldn't
Sorry for the delay in replying.
On Wed, Oct 10, 2018 at 4:05 PM, David Rowley wrote:
> > It seems to me that there is no problem in this patch as far.
> > Is there another thing I have to do for the review?
>
> There's a checklist in [1]. Perhaps there's something mentioned there
> that you've m
Hello
On 2019/09/06 23:19, Tomas Vondra wrote:
> On Wed, Sep 04, 2019 at 07:19:47PM +0300, Sergei Kornilov wrote:
>>
>> ...
>>
>> Results:
>>
>> test | mode | average_tps | degradation_perc
>> --+--+-+--
>> head_no_pgss
Hi Jesper,
On 2019/03/20 23:25, Jesper Pedersen wrote:> Hi,
>
> On 3/19/19 11:15 PM, Imai, Yoshikazu wrote:
>> Here the details.
>>
>> [creating partitioned tables (with 1024 partitions)]
>> drop table if exists rt;
>> create table rt (a int, b int
On Fri, Mar 22, 2019 at 9:07 PM, Tom Lane wrote:
> BTW, it strikes me that we could take advantage of the fact that baserels
> must all appear before otherrels in the rel array, by having loops over
> that array stop early if they're only interested in baserels. We could
> either save the index of
Amit-san,
On Tue, Mar 26, 2019 at 7:17 AM, Amit Langote wrote:
> Rebased patches attached.
I could only do the code review of v36-0001.
On Mon, Mar 25, 2019 at 11:35 AM, Amit Langote wrote:
> On 2019/03/23 6:07, Tom Lane wrote:
> > Amit Langote writes:
> >> [ v34 patch set ]
> >
> > I had a bi
46
512: 6708 12
1024:6063 3
2048:5894 1
4096: 5374 OOM
8192:4572OOM
The performance for hash is also improved, though drop rate of performance with
large partitions seems higher than that of range partitioning.
Thanks
--
Imai Yoshikazu
6932 2440
> 32: 6897 1243
> 64: 6897309
> 128: 6753120
> 256: 6727 46
> 512: 6708 12
> 1024: 6063 3
> 2048:5894 1
> 4096:5374OOM
> 8192:4572OOM
4572/6942 = 65.9 %
Certainly, using bitma
On Fri, Apr 5, 2019 at 1:31 AM, Amit Langote wrote:
> On 2019/04/05 5:42, Peter Eisentraut wrote:
> > On 2019-04-04 06:58, Amit Langote wrote:
> >> Also, since the "speed up partition planning" patch went in
> >> (428b260f8), it might be possible to see the performance boost even
> >> with the part
On Fri, Apr 5, 2019 at 0:05 AM, Tsunakawa, Takayuki wrote:
> From: Peter Eisentraut [mailto:peter.eisentr...@2ndquadrant.com]
> > I can't detect any performance improvement with the patch applied to
> > current master, using the test case from Yoshikazu Imai (2019-03-19).
>
> That's strange... Pe
Hi Fabien,
On Fri, Mar 15, 2019 at 4:17 PM, Fabien COELHO wrote:
> >> echo 'select 1' > select.sql
> >>
> >> while /bin/true; do
> >> pgbench -n -f select.sql -R 1000 -j 8 -c 8 -T 1 > /dev/null 2>&1;
> >> date;
> >> done;
> >
> > Indeed. I'll look at it over the weekend.
> >
> >> So I guess thi
On Wed, July 24, 2019 at 7:02 PM, Fabien COELHO wrote:
> > but I have one question. Is it better adding any check like if(maxsock
> > != -1) before the select?
> >
> > else/* no explicit delay, select without timeout */
> > {
> >nsocks = select(maxsock + 1, &input_mask, NULL, NU
On 2020/01/13 4:11, Pavel Stehule wrote:
> The following review has been posted through the commitfest application:
> make installcheck-world: tested, passed
> Implements feature: tested, passed
> Spec compliant: not tested
> Documentation:tested, passed
>
> I like thi
77 matches
Mail list logo
