Re: Is Recovery actually paused?

2021-01-29 Thread Yugo NAGATA
On Fri, 29 Jan 2021 16:33:32 +0530 Dilip Kumar wrote: > On Fri, Jan 29, 2021 at 3:25 PM Yugo NAGATA wrote: > > > > On Thu, 28 Jan 2021 09:55:42 +0530 > > Dilip Kumar wrote: > > > > > On Wed, Jan 27, 2021 at 2:28 PM Dilip Kumar wrote: > > > > >

Re: Is Recovery actually paused?

2021-01-29 Thread Yugo NAGATA
On Thu, 28 Jan 2021 09:55:42 +0530 Dilip Kumar wrote: > On Wed, Jan 27, 2021 at 2:28 PM Dilip Kumar wrote: > > > > On Wed, Jan 27, 2021 at 2:06 PM Yugo NAGATA wrote: > > > > > > On Wed, 27 Jan 2021 13:29:23 +0530 > > > Dilip Kumar wrote: > > >

Re: [PATCH] Add extra statistics to explain for Nested Loop

2021-01-28 Thread Yugo NAGATA
it a/src/test/regress/expected/timetz.out b/src/test/regress/expected/timetz.out index 038bb5fa094..5294179aa45 100644 Regards, Yugo Nagata -- Yugo NAGATA

Re: Is Recovery actually paused?

2021-01-27 Thread Yugo NAGATA
with the name you suggest that returns text. > > > That would create less burden for tool authors. > > > > +1 > > > > Yeah, we can do that, I will send an updated patch soon. This means pg_is_wal_replay_paused is left without any change and this returns whether pause is requested or not? If so, it seems good to modify the documentation of this function in order to note that this could not return the actual pause state. Regards, Yugo Nagata -- Yugo NAGATA

Re: Columns correlation and adaptive query optimization

2021-01-26 Thread Yugo NAGATA
ctually task of adaptive query optimization is much bigger. > We have separate AQO extension which tries to use machine learning to > correctly adjust estimations. > This my patch is much simpler and use existed mechanism (extended > statistics) to improve estimations. Well, this patch provide a kind of AQO as auto_explain feature, but this is independent of the AQO extension. Is it right? Anyway, I'm interested in the AQO extension, so I'll look into this, too. Regards, Yugo Nagata -- Yugo NAGATA

Re: Is Recovery actually paused?

2021-01-25 Thread Yugo NAGATA
Here(), but this seems redundant. (2) - while (RecoveryIsPaused()) + while (GetRecoveryPauseState() != RECOVERY_NOT_PAUSED) { + HandleStartupProcInterrupts(); Though it is trivial, I think the new line after the brace is unnecessary. Regards, Yugo Nagata -- Yugo NAGATA

Re: Implementing Incremental View Maintenance

2021-01-22 Thread Yugo NAGATA
Hi, Attached is a revised patch (v22) rebased for the latest master head. Regards, Yugo Nagata -- Yugo NAGATA IVM_patches_v22.tar.gz Description: application/gzip

Re: Is Recovery actually paused?

2021-01-21 Thread Yugo NAGATA
On Tue, 19 Jan 2021 21:32:31 +0530 Dilip Kumar wrote: > On Tue, Jan 19, 2021 at 8:34 AM Dilip Kumar wrote: > > > > On Tue, 19 Jan 2021 at 8:12 AM, Yugo NAGATA wrote: > >> > >> On Sun, 17 Jan 2021 11:33:52 +0530 > >> Dilip Kumar wrote: > >>

Re: Columns correlation and adaptive query optimization

2021-01-21 Thread Yugo NAGATA
and is it useful? (10) To achieve adaptive query optimization (AQO) in PostgreSQL, this patch proposes to use auto_explain for getting feedback from actual results. So, could auto_explain be a infrastructure of AQO in future? Or, do you have any plan or idea to make built-in infrastructure for AQO? Regards, Yugo Nagata -- Yugo NAGATA

Re: Is Recovery actually paused?

2021-01-18 Thread Yugo NAGATA
On Sun, 17 Jan 2021 11:33:52 +0530 Dilip Kumar wrote: > On Thu, Jan 14, 2021 at 6:49 PM Yugo NAGATA wrote: > > > > On Wed, 13 Jan 2021 17:49:43 +0530 > > Dilip Kumar wrote: > > > > > On Wed, Jan 13, 2021 at 3:35 PM Dilip Kumar wrote: > > > > >

Re: Is Recovery actually paused?

2021-01-14 Thread Yugo NAGATA
On Wed, 13 Jan 2021 17:49:43 +0530 Dilip Kumar wrote: > On Wed, Jan 13, 2021 at 3:35 PM Dilip Kumar wrote: > > > > On Wed, Jan 13, 2021 at 3:27 PM Yugo NAGATA wrote: > > > > > > On Thu, 10 Dec 2020 11:25:23 +0530 > > > Dilip Kumar wrote: > > &g

Re: Is Recovery actually paused?

2021-01-13 Thread Yugo NAGATA
ke user wait for a long time, I don't care either. > > > As another comment, while pg_is_wal_replay_paused is blocking, I can not > > > cancel > > > the query. I think CHECK_FOR_INTERRUPTS() is necessary in the waiting > > > loop. How about this fix? I think users may want to cancel pg_is_wal_replay_paused() during this is blocking. Regards, Yugo Nagata -- Yugo NAGATA

Re: Implementing Incremental View Maintenance

2021-01-12 Thread Yugo NAGATA
, Yugo Nagata On Tue, 22 Dec 2020 22:24:22 +0900 Yugo NAGATA wrote: > Hi hackers, > > I heard the opinion that this patch is too big and hard to review. > So, I wander that we should downsize the patch by eliminating some > features and leaving other basic features. > > If th

Re: Implementing Incremental View Maintenance

2020-12-22 Thread Yugo NAGATA
it. If so, we plan to support only selection, projection, inner-join, and some aggregates in the first release and leave sub-queries, outer-join, and CTE supports to the next release. Regards, Yugo Nagata On Tue, 22 Dec 2020 21:51:36 +0900 Yugo NAGATA wrote: > Hi, > > Attached is the revi

Re: Implementing Incremental View Maintenance

2020-12-22 Thread Yugo NAGATA
he effect of the optimization. Regards, Yugo Nagata -- Yugo NAGATA IVM_patches_v20.tar.gz Description: application/gzip

Re: Is Recovery actually paused?

2020-11-29 Thread Yugo NAGATA
s can use the old format for calling this and the backward compatibility can be maintained. As another comment, while pg_is_wal_replay_paused is blocking, I can not cancel the query. I think CHECK_FOR_INTERRUPTS() is necessary in the waiting loop. + errhint("Recovery control functions can only be executed during recovery."))); There are a few tabs at the end of this line. Regards, Yugo Nagata -- Yugo NAGATA

Re: Implementing Incremental View Maintenance

2020-11-29 Thread Yugo NAGATA
On Wed, 25 Nov 2020 18:00:16 +0300 Konstantin Knizhnik wrote: > > > On 25.11.2020 16:06, Yugo NAGATA wrote: > > On Wed, 25 Nov 2020 15:16:05 +0300 > > Konstantin Knizhnik wrote: > > > >> > >> On 24.11.2020 13:11, Yugo NAGATA wrote: > >>

Re: Implementing Incremental View Maintenance

2020-11-25 Thread Yugo NAGATA
On Wed, 25 Nov 2020 15:16:05 +0300 Konstantin Knizhnik wrote: > > > On 24.11.2020 13:11, Yugo NAGATA wrote: > > > >> I wonder if it is possible to somehow use predicate locking mechanism of > >> Postgres to avoid this anomalies without global lock? > &g

Re: Implementing Incremental View Maintenance

2020-11-24 Thread Yugo NAGATA
On Tue, 24 Nov 2020 12:46:57 +0300 Konstantin Knizhnik wrote: > > > On 24.11.2020 12:21, Yugo NAGATA wrote: > > > >> I replaced it with RowExlusiveLock and ... got 1437 TPS with 10 > >> connections. > >> It is still about 7 times slower than performa

Re: Implementing Incremental View Maintenance

2020-11-24 Thread Yugo NAGATA
teller_avgs; > REFRESH MATERIALIZED VIEW > Time: 55500.381 ms (00:55.500) Hmm, interesting... I would like to investigate this issue, too. Regards, Yugo Nagata -- Yugo NAGATA

Re: Implementing Incremental View Maintenance

2020-11-24 Thread Yugo NAGATA
fied. Therefore, this is not suitable to OLTP workload where there are frequent updates of tables. For suppressing maintenance overhead in such workload, we have to implement "deferred maintenance" which collects table change logs and updates the view in another transaction afterward. Regards, Yugo Nagata -- Yugo NAGATA

Re: Implementing Incremental View Maintenance

2020-11-12 Thread Yugo NAGATA
On Thu, 5 Nov 2020 22:58:25 -0600 Justin Pryzby wrote: > On Mon, Oct 05, 2020 at 06:16:18PM +0900, Yugo NAGATA wrote: > This needs to be rebased again - the last version doesn't apply anymore. > http://cfbot.cputube.org/yugo-nagata.html I attached the rebased patch (v19). > I l

Re: Implementing Incremental View Maintenance

2020-10-28 Thread Yugo NAGATA
On Wed, 28 Oct 2020 12:01:58 +0300 Anastasia Lubennikova wrote: > ср, 28 окт. 2020 г. в 08:02, Yugo NAGATA : > > > Hi Anastasia Lubennikova, > > > > I am writing this to you because I would like to ask the commitfest > > manager something. > > > > The s

Re: Implementing Incremental View Maintenance

2020-10-28 Thread Yugo NAGATA
s where immediate view maintenance is useful. I am happy because I found concrete use cases of immediate IVM. However, unfortunately, the view definitions in your cases are complex, and the current implementation of the patch doesn't support it. We would like to improve the feature in future so

Re: Implementing Incremental View Maintenance

2020-10-27 Thread Yugo NAGATA
I updated the patch, so can I change the status back to "Ready for Committer"? Regards, Yugo Nagata On Mon, 5 Oct 2020 18:16:18 +0900 Yugo NAGATA wrote: > Hi, > > Attached is the rebased patch (v18) to add support for Incremental > Materialized View Maintenance (IVM). It

Re: Implementing Incremental View Maintenance

2020-10-23 Thread Yugo NAGATA
provides the exact this feature and I think this will meet your needs. > As soon as my coworker gets that example built up I'll send a followup with > it attached. Great! We are looking forward to it. Regards, Yugo Nagata -- Yugo NAGATA

Re: Implementing Incremental View Maintenance

2020-10-21 Thread Yugo NAGATA
u could tell us what your use cases of materialized view is and whether our implementation suits your needs or not. Regards, Yugo Nagata -- Yugo NAGATA

Re: Implementing Incremental View Maintenance

2020-10-16 Thread Yugo NAGATA
e greatly appreciated. [1] https://wiki.postgresql.org/wiki/Incremental_View_Maintenance [2] https://www.postgresql.org/message-id/20191129173328.e5a0e9f81e369a3769c4fd0c%40sraoss.co.jp [3] https://www.postgresql.org/message-id/flat/CAEepm%3D3ZHh%3Dp0nEEnVbs1Dig_UShPzHUcMNAqvDQUgYgcDo-pA%40mail.gmail.com Regard, Yugo Nagata -- Yugo NAGATA

Re: Implementing Incremental View Maintenance

2020-10-05 Thread Yugo NAGATA
* FROM mv; r | x +- 20 | 200 (1 row) cte=# INSERT INTO r VALUES (3,30); INSERT 0 1 cte=# INSERT INTO s VALUES (1,100); INSERT 0 1 cte=# SELECT * FROM mv; r | x +- 20 | 200 30 | 300 10 | 100 (3 rows) == Regards, Yugo Nagata -- Yugo NAGATA IVM_patches_v1

Re: Implementing Incremental View Maintenance

2020-09-30 Thread Yugo NAGATA
ZED > VIEW > > Typo: lager should be larger ? > > +postgres=# SELECt * FROM m; -- automatically updated > > Typo: SELECt should be SELECT ? > > Regards, > > -- > Fujii Masao > Advanced Computing Technology Center > Research and Development Headquarters > NTT DATA CORPORATION -- Yugo NAGATA

Re: Implementing Incremental View Maintenance

2020-09-08 Thread Yugo NAGATA
ed views are Graefe's name for MVs, and > apparently this paper has a section on MVCC systems which sounds > interesting for us. > > [1] https://dsf.berkeley.edu/cs286/papers/mv-fntdb2012.pdf > [2] http://pages.cs.wisc.edu/~gangluo/latch.pdf Thanks for your information! I will also check references regarding with IVM and concurrency control. Regards, Yugo Nagata -- Yugo NAGATA

Re: Implementing Incremental View Maintenance

2020-09-08 Thread Yugo NAGATA
ads to ideas like, I dunno, update chains > containing differential update trees to be compacted later... egad!) I am interested in papers you mentioned! Are they literatures in context of incremental view maintenance? Regards, Yugo Nagata -- Yugo NAGATA

Re: Implementing Incremental View Maintenance

2020-08-30 Thread Yugo NAGATA
Hi, I updated the wiki page. https://wiki.postgresql.org/wiki/Incremental_View_Maintenance On Fri, 21 Aug 2020 21:40:50 +0900 (JST) Tatsuo Ishii wrote: > From: Yugo NAGATA > Subject: Re: Implementing Incremental View Maintenance > Date: Fri, 21 Aug 2020 17:23:20 +0900 >

Re: Implementing Incremental View Maintenance

2020-08-21 Thread Yugo NAGATA
ck_aggregate_supports_ivm > + * > + * Check if the given aggregate function is supporting Regards, Yugo Nagata -- Yugo NAGATA IVM_patches_v17.tar.gz Description: application/gzip

Re: Implementing Incremental View Maintenance

2020-08-18 Thread Yugo NAGATA
description about which situations IVM is effective or not in - Improve hint in log messages - Reorganize include directives in codes Regards, Yugo Nagata -- Yugo NAGATA IVM_patches_v16.tar.gz Description: application/gzip

Re: Implementing Incremental View Maintenance

2020-04-12 Thread Yugo NAGATA
On Fri, 10 Apr 2020 23:26:58 +0900 Yugo NAGATA wrote: > Hi, > > Attached is the latest patch (v15) to add support for Incremental Materialized > View Maintenance (IVM). It is possible to apply to current latest master > branch. I found a mistake of splitting patch, so I att

Re: Implementing Incremental View Maintenance

2020-04-10 Thread Yugo NAGATA
. Best Regards, Takuma Hoshiai -- Yugo NAGATA IVM_patches_v15.tar.gz Description: application/gzip

Re: Allow auto_explain to log plans before queries are executed

2020-02-26 Thread Yugo NAGATA
On Thu, 27 Feb 2020 06:27:24 +0100 Pavel Stehule wrote: > čt 27. 2. 2020 v 6:16 odesílatel Kyotaro Horiguchi > napsal: > > > Hello. > > > > At Thu, 27 Feb 2020 10:18:16 +0900, Yugo NAGATA > > wrote in > > > On Wed, 26 Feb 2020 18:51:21 +0100 > &g

Re: Allow auto_explain to log plans before queries are executed

2020-02-26 Thread Yugo NAGATA
On Thu, 27 Feb 2020 14:14:41 +0900 (JST) Kyotaro Horiguchi wrote: > Hello. > > At Thu, 27 Feb 2020 10:18:16 +0900, Yugo NAGATA wrote > in > > On Wed, 26 Feb 2020 18:51:21 +0100 > > Julien Rouhaud wrote: > > > > > On Thu, Feb 27, 2020 at 02:35:18

Re: Allow auto_explain to log plans before queries are executed

2020-02-26 Thread Yugo NAGATA
On Wed, 26 Feb 2020 18:51:21 +0100 Julien Rouhaud wrote: > On Thu, Feb 27, 2020 at 02:35:18AM +0900, Yugo NAGATA wrote: > > Hi, > > > > Attached is a patch for allowing auto_explain to log plans before > > queries are executed. > > > > Currently, a

Allow auto_explain to log plans before queries are executed

2020-02-26 Thread Yugo NAGATA
of auto_explain.log_min_duration unless this is set -1 to disable logging. If log_before_query is enabled, only duration time is logged after query execution as in the case of when both log_statement and log_min_duration_statement are enabled. Regards, Yugo Nagata -- Yugo NAGATA >F

Re: Implementing Incremental View Maintenance

2020-02-18 Thread Yugo NAGATA
> > SELECT 'table_y' AS name, * FROM table_y > > ; > > CREATE VIEW > > TABLE xy_union_v; > > name | id |data > > -+----+ > > table_y | 2 | 0.0528458947672874 > > table_x | 2 | 0.0222670808201144 > > tabl

Re: Implementing Incremental View Maintenance

2020-02-12 Thread Yugo NAGATA
T {WITH | WITHOUT} INCREMENTAL REFRESH although this is just a idea and we will need discussion on it. Regards, Yugo Nagata -- Yugo NAGATA

Re: Implementing Incremental View Maintenance

2020-02-09 Thread Yugo NAGATA
Nagata On Fri, 20 Dec 2019 14:02:32 +0900 Yugo Nagata wrote: > IVM is a way to make materialized views up-to-date in which only > incremental changes are computed and applied on views rather than > recomputing the contents from scratch as REFRESH MATERIALIZED VIEW > does. IV

Re: Implementing Incremental View Maintenance

2020-02-09 Thread Yugo NAGATA
ns including UNION is concurrently unsupported, although this is not checked at definition time and not documented either. Now we are thoroughly investigating unsupported queries, and will add checks and documentations for them. Regards, Yugo Nagata > > (commit id = 23151be7be8d8f8f

Re: Implementing Incremental View Maintenance

2020-02-05 Thread Yugo NAGATA
note to the document (rules.sgml). Thank you for your suggestion! We'll add some description about this to the documentation. Regards, Yugo Nagata > > execute log. > > ``` > [ec2-user@ip-10-0-1-10 rls]$ psql testdb -e -f rls.sql > CREATE USER user_a; > CREATE ROLE > CREA

Re: Implementing Incremental View Maintenance

2020-01-19 Thread Yugo NAGATA
On Fri, 17 Jan 2020 14:10:32 -0700 (MST) legrand legrand wrote: > Hello, > > It seems that patch v11 doesn't apply any more. > Problem with "scanRTEForColumn" maybe because of change: Thank you for your reporting! We will fix this in the next update. Regards, Y

Re: Incremental View Maintenance: ERROR: out of shared memory

2020-01-17 Thread Yugo NAGATA
, as well as performance problems above > and beyond the one described here. We realized that there is also other problems in using temp tables as pointed out in another thread. So, we are now working on rewrite our patch not to use temp tables. Regards, Yugo Nagata -- Yugo NAGATA

Re: Implementing Incremental View Maintenance

2020-01-17 Thread Yugo NAGATA
g into the view > source table (including the user-defined type column). > ``` > ERROR: operator does not exist Thank you for your reporting. I think this error occurs because pg_catalog.= is used also for user-defined types. I will fix this. Regards, Yugo Nagata > ``` > > An

Re: Implementing Incremental View Maintenance

2020-01-17 Thread Yugo NAGATA
ollowing error message doesn't look good because built-in min is supported, so I will improve it. ERROR: aggregate function min is not supported Regards, Yugo Nagata > > An execution example is shown below. > > ``` > [ec2-user@ip-10-0-1-10 ivm]$ cat extension-agg.sql > -

Re: Implementing Incremental View Maintenance

2019-12-25 Thread Yugo Nagata
On Tue, 24 Dec 2019 07:07:35 + "tsunakawa.ta...@fujitsu.com" wrote: > From: Yugo Nagata > > On Mon, 23 Dec 2019 08:08:53 + > > "tsunakawa.ta...@fujitsu.com" wrote: > > > How about unlogged tables ? I thought the point of using a te

Re: Implementing Incremental View Maintenance

2019-12-23 Thread Yugo Nagata
rg/message-id/flat/157703426606.1198.2452090605041230054.pgcf%40coridan.postgresql.org#331e8344bbae904350af161fb43a0aa6 Although I have not looked into this thread, this may be help if this is implemented. However, it would be still necessary to truncate the table before the view maintenance because such tables always exist and can be accessed and modified by any users. -- Yugo Nagata

Re: Implementing Incremental View Maintenance

2019-12-23 Thread Yugo Nagata
On Mon, 23 Dec 2019 08:08:53 + "tsunakawa.ta...@fujitsu.com" wrote: > From: Yugo Nagata > > 1. Create a temporary table only once at the first view maintenance in > > this session. This is possible if we store names or oid of temporary > > tables used for e

Re: Implementing Incremental View Maintenance

2019-12-22 Thread Yugo Nagata
rs. So, it seems worth to consider the way to reduce use of temptable. Regards, Yugo Nagata -- Yugo Nagata

Re: Implementing Incremental View Maintenance

2019-12-22 Thread Yugo Nagata
he view returns incorrect results. > To prevent this, we propose that the same error occur when a non-IMMUTABLE > expression is specified in the "CREATE INDEX" statement. Thank you for pointing out this. That makes sense. The check of not-IMMUTABLE epressions is missing at cre

Re: Implementing Incremental View Maintenance

2019-12-19 Thread Yugo Nagata
be easy for inner joins or aggregate views, but there is some difficult with outer joins. Regards, Yugo Nagata -- Yugo Nagata IVM_patches_v10.tar.gz Description: application/gzip

Re: Implementing Incremental View Maintenance

2019-12-04 Thread Yugo Nagata
table will fail. > > Error message. > ``` > ERROR: could not open relation with OID 0 Thank you for your pointing out this issue! This error occurs because the view's OID is retrieved using the view name. Considering that the name can be changed, this is obviously wrong. We'll fix it.

Re: Implementing Incremental View Maintenance

2019-12-02 Thread Yugo Nagata
On Mon, 2 Dec 2019 13:48:40 -0300 Alvaro Herrera wrote: > On 2019-Dec-02, Yugo Nagata wrote: > > > On Mon, 02 Dec 2019 10:36:36 +0900 (JST) > > Tatsuo Ishii wrote: > > > > > >> One thing pending in this development line is how to catalogue > &g

Re: Implementing Incremental View Maintenance

2019-12-01 Thread Yugo Nagata
> > to handle correctly. Supported aggregates can be identified using their > > OIDs. > > User-defined aggregates are not supported. I think this is the simplest and > > easiest way. > > I think this is enough for the first cut of IVM. So +1. If there is no objection, I will add the check of aggregate functions by this way. Thanks. Regards, Yugo Nagata -- Yugo Nagata

Re: Implementing Incremental View Maintenance

2019-11-29 Thread Yugo Nagata
On Fri, 29 Nov 2019 18:16:00 +0900 Yugo Nagata wrote: > On Fri, 29 Nov 2019 15:34:52 +0900 > Amit Langote wrote: > > > Thanks a lot for working on this. It's a great (and big!) feature and > > I can see that a lot of work has been put into writing this patch.

Re: Implementing Incremental View Maintenance

2019-11-29 Thread Yugo Nagata
y to attract more eyeballs. > > * Someone well versed in trigger infrastructure can help fine tune the > patch for (2) > > and so on. > > So, please consider giving some thought to this. Agreed. Although I am not sure we will do it as above way, we will consider to split the patch, anyway. Thanks. Regards, Yugo Nagata -- Yugo Nagata

Re: Implementing Incremental View Maintenance

2019-11-29 Thread Yugo Nagata
functions to calculate new aggregate values in IVM when tuples are inserted into a table. However, in the context of IVM, we also need other function used when tuples are deleted from a table, so we can not use partial aggregation for IVM in the current implementation. This might be another option to i

Re: To Suggest a "DROP INCREMENTAL MATERIALIZED VIEW" in psql, but the syntax error when you run.

2019-11-28 Thread Yugo Nagata
Hello nuko-san, Thank you for your review! As Michael commentted, we would like to discuss this on the thread of the patch, so I quote your review in the following post. https://www.postgresql.org/message-id/20191129154513.943f4ef05896d7b9d3fed69f%40sraoss.co.jp Regards, Yugo Nagata On Thu

Re: Implementing Incremental View Maintenance

2019-11-28 Thread Yugo Nagata
The following review on our patch was posted on another thread, so I quote here. The tab completion is Hoshiai-san's work, so he will handle this issue. Regards, Yugo Nagata. On Thu, 28 Nov 2019 13:00:05 +0900 nuko yokohama wrote: > Hi. > > I'm using the "Incremental Mat

Re: Implementing Incremental View Maintenance

2019-11-28 Thread Yugo Nagata
; (IMMs) would be good. So, how about using this for now? When other better opinions are raised, let's discuss again Regards, Yugo Nagata > > Best regards, > -- > Tatsuo Ishii > SRA OSS, Inc. Japan > English: http://www.sraoss.co.jp/index_en.php > Japanese:http://www.sraoss.co.jp -- Yugo Nagata

Re: Implementing Incremental View Maintenance

2019-11-28 Thread Yugo Nagata
> Best regards, > -- > Tatsuo Ishii > SRA OSS, Inc. Japan > English: http://www.sraoss.co.jp/index_en.php > Japanese:http://www.sraoss.co.jp -- Yugo Nagata

Re: Implementing Incremental View Maintenance

2019-11-28 Thread Yugo Nagata
VM, so I'll think about that a little more Regards, Yugo Nagata > > Best regards, > -- > Tatsuo Ishii > SRA OSS, Inc. Japan > English: http://www.sraoss.co.jp/index_en.php > Japanese:http://www.sraoss.co.jp -- Yugo Nagata

Re: Implementing Incremental View Maintenance

2019-11-25 Thread Yugo Nagata
iew.c so that ones which have relationship will be located closely. Moreover, I added more comments. Regards, Yugo Nagata -- Yugo Nagata IVM_v8.patch.gz Description: application/gzip

Re: Implementing Incremental View Maintenance

2019-09-30 Thread Yugo Nagata
Results - Comparison with the normal view SELECT * FROM mv ORDER BY v1; v1 | v2 --+- 10 | 100 11 | 100 1020 | 200 1020 | 222 (4 rows) SELECT * FROM v ORDER BY v1; v1 | v2 --+- 10 | 100 11 | 100 1020 | 200 1020 | 222 (4 rows) Best Regards, Yugo

Re: Implementing Incremental View Maintenance

2019-09-27 Thread Yugo Nagata
tually, post-update state is available in AFTER trigger, and pre-update state can be calculated by using delta tables (transition tables) and cmin/xmin system columns (or snapshot). This is the approach my implementation adopts. > > On Tue, Sep 17, 2019 at 8:50 AM Yugo Nagata wrote: > >

Re: Implementing Incremental View Maintenance

2019-09-17 Thread Yugo Nagata
to collect information of modified table and its changes (= transition tables), and then the only last trigger updates the view. This will avoid the double-counting. I think this implementation also would be a base of deferred approach implementation in future where "logs" are used instead of transition tables. Regards, Yugo Nagata -- Yugo Nagata

Re: Implementing Incremental View Maintenance

2019-07-31 Thread Yugo Nagata
values are contained in views as hidden columns and use them to calculate new avg value instead of using old avg values. Regards, On Fri, 26 Jul 2019 11:35:53 +0900 Yugo Nagata wrote: > Hi, > > I've updated the wiki page of Incremental View Maintenance. > > https://wiki.post

Re: Implementing Incremental View Maintenance

2019-07-25 Thread Yugo Nagata
Hi, I've updated the wiki page of Incremental View Maintenance. https://wiki.postgresql.org/wiki/Incremental_View_Maintenance On Thu, 11 Jul 2019 13:28:04 +0900 Yugo Nagata wrote: > Hi Thomas, > > Thank you for your review and discussion on this patch! > > > > 2019

Re: Implementing Incremental View Maintenance

2019-07-10 Thread Yugo Nagata
Hi Thomas, Thank you for your review and discussion on this patch! > > 2019年7月8日(月) 15:32 Thomas Munro : > > > > > On Fri, Jun 28, 2019 at 10:56 PM Yugo Nagata wrote: > > > > Attached is a WIP patch of IVM which supports some aggregate functions. > >

Re: Implementing Incremental View Maintenance

2019-06-28 Thread Yugo Nagata
Hi Greg, On Wed, 3 Apr 2019 17:41:36 -0400 Greg Stark wrote: > On Sun, 31 Mar 2019 at 23:22, Yugo Nagata wrote: > > > > Firstly, this will handle simple definition views which includes only > > selection, projection, and join. Standard aggregations (count, s

Re: Implementing Incremental View Maintenance

2019-06-28 Thread Yugo Nagata
mv_ivm_group ORDER BY 1; sum - 150 (1 row) INSERT INTO mv_base_a VALUES(6,20); SELECT * FROM mv_ivm_group ORDER BY 1; sum - 170 (1 row) === On Thu, 20 Jun 2019 16:44:10 +0900 Yugo Nagata wrote: > Hi hackers, > &

Re: Implementing Incremental View Maintenance

2019-06-28 Thread Yugo Nagata
feu2018/schedule/session/2195-implementing-incremental-view-maintenance-on-postgresql/ I agree that capturing the deltas using logical decoding will be faster than using a trigger although we haven't yet consider this well. Best regadrds, Yugo Nagata -- Yugo Nagata

Re: Implementing Incremental View Maintenance

2019-05-14 Thread Yugo Nagata
On Mon, 1 Apr 2019 12:11:22 +0900 Yugo Nagata wrote: > On Thu, 27 Dec 2018 21:57:26 +0900 > Yugo Nagata wrote: > > > Hi, > > > > I would like to implement Incremental View Maintenance (IVM) on PostgreSQL. > > > > I am now working on an initial p

Re: Implementing Incremental View Maintenance

2019-03-31 Thread Yugo Nagata
On Thu, 27 Dec 2018 21:57:26 +0900 Yugo Nagata wrote: > Hi, > > I would like to implement Incremental View Maintenance (IVM) on PostgreSQL. I am now working on an initial patch for implementing IVM on PostgreSQL. This enables materialized views to be updated incrementally

Re: Improvement of installation document

2019-03-26 Thread Yugo Nagata
e libmemcached-devel is correct instead of libmemcached? > > I don't think so. "libmemcached-devel" is just a package name in a > cetain Linux distribution. "libmemcached" is a more geneal and non > distribution dependent term. Thanks for your explaination. I understood it. > Best regards, > -- > Tatsuo Ishii > SRA OSS, Inc. Japan > English: http://www.sraoss.co.jp/index_en.php > Japanese:http://www.sraoss.co.jp -- Yugo Nagata

Re: Improvement of installation document

2019-03-26 Thread Yugo Nagata
Hi, I apologize that I accidentally sent the following email to this list. Please disregard this. I am sorry for making a lot of noise. Regard, On Tue, 26 Mar 2019 20:38:31 +0900 Yugo Nagata wrote: > Hi, > > One of our clients suggested that the installation document[1] lacks >

Improvement of installation document

2019-03-26 Thread Yugo Nagata
cache. You have to install libmemcached. , but maybe libmemcached-devel is correct instead of libmemcached? Regards, -- Yugo Nagata -- Yugo Nagata

Re: Proposal to suppress errors thrown by to_reg*()

2019-03-19 Thread Yugo Nagata
mespace, but I think we can make this more simpler by using SET SESSION AUTHORIZATION and RESET AUTHORIZATION. Regards, -- Yugo Nagata

Re: Implementing Incremental View Maintenance

2019-01-31 Thread Yugo Nagata
uot;bar" and we > compute an update for the materialized view again. If we have not > applied any of those updates (because we are "lazy") now the > previously computed update can be discarded. We could still apply > both, but it would not be efficient. In our PoC implementation, I handled this situation by removing old contents from NEW delata table. In your example, when the base table is updated from "foo" to "bar", the "foo" tuple is removed from and the "bar" tuple is inserted in NEW delta and the delta of MV is computed using the final NEW delta. Regards, -- Yugo Nagata

Re: Implementing Incremental View Maintenance

2019-01-31 Thread Yugo Nagata
t is, if concurrent transactions update different two tables in the join view definition, is MV updated sucessfully? [1] https://www.postgresql.org/message-id/flat/1368561126.64093.YahooMailNeo%40web162904.mail.bf1.yahoo.com Regards, -- Yugo Nagata

Implementing Incremental View Maintenance

2018-12-27 Thread Yugo Nagata
ailNeo%40web162904.mail.bf1.yahoo.com [5] https://dl.acm.org/citation.cfm?id=170066 Regards, -- Yugo Nagata

Re: Unused argument from execute_sql_string()

2018-09-18 Thread Yugo Nagata
unused argument. > > Same opinion and arguments here, so I have committed the patch. Thanks! -- Yugo Nagata

Unused argument from execute_sql_string()

2018-09-07 Thread Yugo Nagata
Hi, I found that a argument "filename" is not used in execute_sql_string() although the comment says "filename is used only to report errors.", so I think we can remove this argument as done in the patch I attached. Regards, -- Yugo Nagata diff --git a/src/backend/comman

Re: pg_verify_checksums -d option (was: Re: pg_verify_checksums -r option)

2018-09-03 Thread Yugo Nagata
a database by OID using -d or > > --dboid option. > > Also, when -g or --global-only is used only shared relations are scaned. > > It seems that the patch does not apply anymore. Could you rebase it? I attached the rebased patch. Regards, -- Yugo Nagata >From f2141a28d1c1

Re: pg_verify_checksums -d option (was: Re: pg_verify_checksums -r option)

2018-08-31 Thread Yugo Nagata
On Mon, 27 Aug 2018 21:05:33 +0900 Yugo Nagata wrote: > On Mon, 27 Aug 2018 13:34:12 +0200 > Michael Banck wrote: > > > Hi, > > > > On Mon, Aug 27, 2018 at 07:53:36PM +0900, Yugo Nagata wrote: > > > On Fri, 24 Aug 2018 18:01:09 +0200 > > > Peter

Re: pg_verify_checksums -d option (was: Re: pg_verify_checksums -r option)

2018-08-30 Thread Yugo Nagata
On Thu, 30 Aug 2018 07:18:13 -0300 Alvaro Herrera wrote: > On 2018-Aug-30, Yugo Nagata wrote: > > > On Thu, 30 Aug 2018 06:52:58 -0300 > > Alvaro Herrera wrote: > > > > This should have been listed in the pg11 open items. Please list there > > > everyth

Re: pg_verify_checksums -d option (was: Re: pg_verify_checksums -r option)

2018-08-30 Thread Yugo Nagata
ore release, so > that they get fixed (or at least considered) before we release. > > https://wiki.postgresql.org/wiki/Open_Items I don't have the editor privilege now, so I'll add this discussion to the wiki (Fixed issues or Resolve issues?) after I get the privilege. Regards, -- Yugo Nagata

Re: pg_verify_checksums -d option (was: Re: pg_verify_checksums -r option)

2018-08-30 Thread Yugo Nagata
On Thu, 30 Aug 2018 10:54:08 +0200 Michael Banck wrote: > Hi, > > On Thu, Aug 30, 2018 at 05:48:24PM +0900, Yugo Nagata wrote: > > Oops, It's my mistake. I updated the patch. > > Looks good to me now. > > One could argue that the message could be 'checksums verif

Re: pg_verify_checksums -d option (was: Re: pg_verify_checksums -r option)

2018-08-30 Thread Yugo Nagata
On Thu, 30 Aug 2018 10:34:06 +0200 Michael Banck wrote: > Hi, > > On Thu, Aug 30, 2018 at 05:35:09PM +0900, Yugo Nagata wrote: > > --- a/doc/src/sgml/ref/pg_verify_checksums.sgml > > +++ b/doc/src/sgml/ref/pg_verify_checksums.sgml > > @@ -61,11 +61,11

Re: pg_verify_checksums -d option (was: Re: pg_verify_checksums -r option)

2018-08-30 Thread Yugo Nagata
On Thu, 30 Aug 2018 10:13:31 +0200 Michael Banck wrote: > Hi, > > thanks for fixing this up! > > On Wed, Aug 29, 2018 at 11:25:28PM +0900, Yugo Nagata wrote: > > diff --git a/doc/src/sgml/ref/pg_verify_checksums.sgml > > b/doc/src/sgml/ref/pg_verify_checksum

Re: pg_verify_checksums failure with hash indexes

2018-08-30 Thread Yugo Nagata
On Thu, 30 Aug 2018 15:01:24 +0900 (Tokyo Standard Time) Kyotaro HORIGUCHI wrote: > At Wed, 29 Aug 2018 20:10:15 +0900, Yugo Nagata wrote > in <20180829201015.d9d4fde2748910e86a13c...@sraoss.co.jp> > > On Wed, 29 Aug 2018 16:01:53 +0530 > > Amit Kapila wrote: > >

Re: Fix comments of IndexInfo

2018-08-30 Thread Yugo Nagata
On Thu, 30 Aug 2018 09:14:26 +0300 Heikki Linnakangas wrote: > On 30/08/18 07:48, Yugo Nagata wrote: > > Attached is a patch to fix comments of IndexInfo. ii_KeyAttrNumbers was > > renamed to ii_IndexAttrNumbers and ii_Am was added but these are > > not reflected to the

Fix comments of IndexInfo

2018-08-29 Thread Yugo Nagata
Hi, Attached is a patch to fix comments of IndexInfo. ii_KeyAttrNumbers was renamed to ii_IndexAttrNumbers and ii_Am was added but these are not reflected to the comment. Regards, -- Yugo Nagata diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h index 41fa2052a2

Re: pg_verify_checksums -d option (was: Re: pg_verify_checksums -r option)

2018-08-29 Thread Yugo Nagata
On Wed, 29 Aug 2018 21:09:03 +0900 Yugo Nagata wrote: > On Wed, 29 Aug 2018 13:46:38 +0200 > Magnus Hagander wrote: > > > On Wed, Aug 29, 2018 at 1:37 PM, Michael Banck > > wrote: > > > > > Hi, > > > > > > On Wed, Aug 29, 2018 at 08:33

Re: pg_verify_checksums -d option (was: Re: pg_verify_checksums -r option)

2018-08-29 Thread Yugo Nagata
On Wed, 29 Aug 2018 13:46:38 +0200 Magnus Hagander wrote: > On Wed, Aug 29, 2018 at 1:37 PM, Michael Banck > wrote: > > > Hi, > > > > On Wed, Aug 29, 2018 at 08:33:43PM +0900, Yugo Nagata wrote: > > > On Wed, 29 Aug 2018 10:28:33 +0200 > > > Danie

<    1   2   3   4   5   >