Re: Incremental View Maintenance, take 2

2024-08-20 Thread Kirill Reshke
On Tue, 20 Aug 2024 at 02:14, Kirill Reshke wrote: > > > == Major suggestions. > > 1) At first glance, working with this IVM/IMMV infrastructure feels > really unintuitive about what servers actually do for query execution. > I do think It will be much better for user experience to add more > EXPL

Re: Incremental View Maintenance, take 2

2024-08-19 Thread Kirill Reshke
On Tue, 20 Aug 2024 at 02:14, Kirill Reshke wrote: > == Other thoughts > > In OLAP databases (see [2]), IVM opens the door for 'view > exploitation' feature. That is, use IVM (which is always up-to-date) > for query execution. But current IVM implementation is not compatible > with Cloudberry App

Re: Incremental View Maintenance, take 2

2024-08-19 Thread Kirill Reshke
On Tue, 30 Jul 2024 at 10:24, Yugo NAGATA wrote: > > Hi, > > On Tue, 30 Jul 2024 03:32:19 +0500 > Kirill Reshke wrote: > > > On Sat, 27 Jul 2024 at 13:26, Kirill Reshke wrote: > > > > > > Hi! > > > Cloudberry DB (Greenplum fork) uses IMMV feature for AQUMV (auto query > > > use matview) feature,

Re: Incremental View Maintenance, take 2

2024-08-08 Thread Kirill Reshke
On Wed, 31 May 2023 at 20:14, Yugo NAGATA wrote: > > Hello hackers, > > Here's a rebased version of the patch-set adding Incremental View > Maintenance support for PostgreSQL. That was discussed in [1]. > > The patch-set consists of the following eleven patches. > > - 0001: Add a syntax to create

Re: Incremental View Maintenance, take 2

2024-08-08 Thread Kirill Reshke
I am really sorry for splitting my review comments into multiple emails. I'll try to do a better review in a future, all-in-one. On Thu, 11 Jul 2024 at 09:24, Yugo NAGATA wrote: > > On Tue, 2 Jul 2024 17:03:11 +0900 > Yugo NAGATA wrote: > > > On Sun, 31 Mar 2024 22:59:31 +0900 > > Yugo NAGATA w

Re: Incremental View Maintenance, take 2

2024-08-06 Thread Kirill Reshke
On Thu, 11 Jul 2024 at 09:24, Yugo NAGATA wrote: > > On Tue, 2 Jul 2024 17:03:11 +0900 > Yugo NAGATA wrote: > > > On Sun, 31 Mar 2024 22:59:31 +0900 > > Yugo NAGATA wrote: > > > > > > > > Also, I added a comment on RelationIsIVM() macro persuggestion from > > > > jian he. > > > > In addition, I

Re: Incremental View Maintenance, take 2

2024-07-30 Thread Kirill Reshke
On Tue, 30 Jul 2024 at 03:32, Kirill Reshke wrote: > > On Sat, 27 Jul 2024 at 13:26, Kirill Reshke wrote: > > > > Hi! > > Cloudberry DB (Greenplum fork) uses IMMV feature for AQUMV (auto query > > use matview) feature, so i got interested in how it is implemented. > > > > On Thu, 11 Jul 2024 at 0

Re: Incremental View Maintenance, take 2

2024-07-29 Thread Yugo NAGATA
Hi, On Tue, 30 Jul 2024 03:32:19 +0500 Kirill Reshke wrote: > On Sat, 27 Jul 2024 at 13:26, Kirill Reshke wrote: > > > > Hi! > > Cloudberry DB (Greenplum fork) uses IMMV feature for AQUMV (auto query > > use matview) feature, so i got interested in how it is implemented. Thank you so much for

Re: Incremental View Maintenance, take 2

2024-07-29 Thread Kirill Reshke
On Sat, 27 Jul 2024 at 13:26, Kirill Reshke wrote: > > Hi! > Cloudberry DB (Greenplum fork) uses IMMV feature for AQUMV (auto query > use matview) feature, so i got interested in how it is implemented. > > On Thu, 11 Jul 2024 at 09:24, Yugo NAGATA wrote: > > > > I updated the patch to bump up the

Re: Incremental View Maintenance, take 2

2024-07-27 Thread Kirill Reshke
Hi! Cloudberry DB (Greenplum fork) uses IMMV feature for AQUMV (auto query use matview) feature, so i got interested in how it is implemented. On Thu, 11 Jul 2024 at 09:24, Yugo NAGATA wrote: > > I updated the patch to bump up the version numbers in psql and pg_dump codes > from 17 to 18. Few su

Re: Incremental View Maintenance, take 2

2024-03-03 Thread Yugo NAGATA
On Mon, 4 Sep 2023 16:48:02 +0800 jian he wrote: > other ideas based on v29. > > src/include/utils/rel.h > 680: #define RelationIsIVM(relation) ((relation)->rd_rel->relisivm) > I guess it would be better to add some comments to address the usage. > Since all peer macros all have some comments. O

Re: Incremental View Maintenance, take 2

2024-03-03 Thread Yugo NAGATA
On Fri, 1 Sep 2023 15:42:17 +0800 jian he wrote: I apologize for this late reply. > I added a new function append_update_set_caluse, and deleted > functions: {append_set_clause_for_count, append_set_clause_for_sum, > append_set_clause_for_avg, append_set_clause_for_minmax} > > I guess this wa

Re: Incremental View Maintenance, take 2

2024-01-22 Thread Yugo NAGATA
On Mon, 22 Jan 2024 13:51:08 +1100 Peter Smith wrote: > 2024-01 Commitfest. > > Hi, This patch has a CF status of "Needs Review" [1], but it seems > like there was some CFbot test failure last time it was run [2]. > Please have a look and post an updated version if necessary. Thank you for poin

Re: Incremental View Maintenance, take 2

2024-01-21 Thread Peter Smith
2024-01 Commitfest. Hi, This patch has a CF status of "Needs Review" [1], but it seems like there was some CFbot test failure last time it was run [2]. Please have a look and post an updated version if necessary. == [1] https://commitfest.postgresql.org/46/4337/ [2] https://cirrus-ci.com/task

Re: Incremental View Maintenance, take 2

2023-09-04 Thread jian he
On Sat, Sep 2, 2023 at 7:46 PM Tatsuo Ishii wrote: > > > attached is my refactor. there is some whitespace errors in the > > patches, you need use > > git apply --reject --whitespace=fix > > basedon_v29_matview_c_refactor_update_set_clause.patch > > > > Also you patch cannot use git apply, i final

Re: Incremental View Maintenance, take 2

2023-09-02 Thread Tatsuo Ishii
> attached is my refactor. there is some whitespace errors in the > patches, you need use > git apply --reject --whitespace=fix > basedon_v29_matview_c_refactor_update_set_clause.patch > > Also you patch cannot use git apply, i finally found out bulk apply I have no problem with applying Yugo's v

Re: Incremental View Maintenance, take 2

2023-09-01 Thread jian he
hi based on v29. based on https://stackoverflow.com/a/4014981/1560347: I added a new function append_update_set_caluse, and deleted functions: {append_set_clause_for_count, append_set_clause_for_sum, append_set_clause_for_avg, append_set_clause_for_minmax} I guess this way is more extensible/gene

Re: Incremental View Maintenance, take 2

2023-08-27 Thread Yugo NAGATA
On Sun, 2 Jul 2023 10:38:20 +0800 jian he wrote: > ok. Now I really found a small bug. > > this works as intended: > BEGIN; > CREATE INCREMENTAL MATERIALIZED VIEW test_ivm AS SELECT i, MIN(j) as > min_j FROM mv_base_a group by 1; > INSERT INTO mv_base_a select 1,-2 where false; > rollback; > >

Re: Incremental View Maintenance, take 2

2023-08-27 Thread Yugo NAGATA
On Sun, 2 Jul 2023 08:25:12 +0800 jian he wrote: > This is probably not trivial. > In function apply_new_delta_with_count. > > appendStringInfo(&querybuf, > "WITH updt AS (" /* update a tuple if this exists in the view */ > "UPDATE %s AS mv SET %s = mv.%s OPERATOR(pg_catalog.+) diff.%s " > "%s

Re: Incremental View Maintenance, take 2

2023-08-27 Thread Yugo NAGATA
On Fri, 30 Jun 2023 08:00:00 +0800 jian he wrote: > Hi there. > in v28-0005-Add-Incremental-View-Maintenance-support-to-psql.patch > I don't know how to set psql to get the output > "Incremental view maintenance: yes" This information will appear when you use "d+" command for an incrementally m

Re: Incremental View Maintenance, take 2

2023-08-27 Thread Yugo NAGATA
On Thu, 29 Jun 2023 18:51:06 +0800 jian he wrote: > I cannot build the doc. > git clean -fdx > git am ~/Desktop/tmp/*.patch > > Applying: Add a syntax to create Incrementally Maintainable Materialized Views > Applying: Add relisivm column to pg_class system catalog > Applying: Allow to prolong

Re: Incremental View Maintenance, take 2

2023-08-27 Thread Yugo NAGATA
On Thu, 29 Jun 2023 18:20:32 +0800 jian he wrote: > On Thu, Jun 29, 2023 at 12:40 AM jian he wrote: > > > > On Wed, Jun 28, 2023 at 4:06 PM Yugo NAGATA wrote: > > > > > > On Wed, 28 Jun 2023 00:01:02 +0800 > > > jian he wrote: > > > > > > > On Thu, Jun 1, 2023 at 2:47 AM Yugo NAGATA wrote: >

Re: Incremental View Maintenance, take 2

2023-08-27 Thread Yugo NAGATA
On Thu, 29 Jun 2023 00:40:45 +0800 jian he wrote: > On Wed, Jun 28, 2023 at 4:06 PM Yugo NAGATA wrote: > > > > On Wed, 28 Jun 2023 00:01:02 +0800 > > jian he wrote: > > > > > On Thu, Jun 1, 2023 at 2:47 AM Yugo NAGATA wrote: > > > > > > > > On Thu, 1 Jun 2023 23:59:09 +0900 > > > > Yugo NAGATA

Re: Incremental View Maintenance, take 2

2023-07-01 Thread jian he
ok. Now I really found a small bug. this works as intended: BEGIN; CREATE INCREMENTAL MATERIALIZED VIEW test_ivm AS SELECT i, MIN(j) as min_j FROM mv_base_a group by 1; INSERT INTO mv_base_a select 1,-2 where false; rollback; however the following one: BEGIN; CREATE INCREMENTAL MATERIALIZED VIEW

Re: Incremental View Maintenance, take 2

2023-07-01 Thread jian he
This is probably not trivial. In function apply_new_delta_with_count. appendStringInfo(&querybuf, "WITH updt AS (" /* update a tuple if this exists in the view */ "UPDATE %s AS mv SET %s = mv.%s OPERATOR(pg_catalog.+) diff.%s " "%s " /* SET clauses for aggregates */ "FROM %s AS diff " "WHERE %s

Re: Incremental View Maintenance, take 2

2023-06-29 Thread jian he
Hi there. in v28-0005-Add-Incremental-View-Maintenance-support-to-psql.patch I don't know how to set psql to get the output "Incremental view maintenance: yes"

Re: Incremental View Maintenance, take 2

2023-06-29 Thread jian he
On Thu, Jun 29, 2023 at 6:51 PM jian he wrote: > > I cannot build the doc. > git clean -fdx > git am ~/Desktop/tmp/*.patch > > Applying: Add a syntax to create Incrementally Maintainable Materialized Views > Applying: Add relisivm column to pg_class system catalog > Applying: Allow to prolong lif

Re: Incremental View Maintenance, take 2

2023-06-29 Thread jian he
I cannot build the doc. git clean -fdx git am ~/Desktop/tmp/*.patch Applying: Add a syntax to create Incrementally Maintainable Materialized Views Applying: Add relisivm column to pg_class system catalog Applying: Allow to prolong life span of transition tables until transaction end Applying: Add

Re: Incremental View Maintenance, take 2

2023-06-29 Thread jian he
On Thu, Jun 29, 2023 at 12:40 AM jian he wrote: > > On Wed, Jun 28, 2023 at 4:06 PM Yugo NAGATA wrote: > > > > On Wed, 28 Jun 2023 00:01:02 +0800 > > jian he wrote: > > > > > On Thu, Jun 1, 2023 at 2:47 AM Yugo NAGATA wrote: > > > > > > > > On Thu, 1 Jun 2023 23:59:09 +0900 > > > > Yugo NAGATA

Re: Incremental View Maintenance, take 2

2023-06-28 Thread jian he
On Wed, Jun 28, 2023 at 4:06 PM Yugo NAGATA wrote: > > On Wed, 28 Jun 2023 00:01:02 +0800 > jian he wrote: > > > On Thu, Jun 1, 2023 at 2:47 AM Yugo NAGATA wrote: > > > > > > On Thu, 1 Jun 2023 23:59:09 +0900 > > > Yugo NAGATA wrote: > > > > > > > Hello hackers, > > > > > > > > Here's a rebased

Re: Incremental View Maintenance, take 2

2023-06-28 Thread Yugo NAGATA
On Wed, 28 Jun 2023 00:01:02 +0800 jian he wrote: > On Thu, Jun 1, 2023 at 2:47 AM Yugo NAGATA wrote: > > > > On Thu, 1 Jun 2023 23:59:09 +0900 > > Yugo NAGATA wrote: > > > > > Hello hackers, > > > > > > Here's a rebased version of the patch-set adding Incremental View > > > Maintenance support

Re: Incremental View Maintenance, take 2

2023-06-27 Thread jian he
On Thu, Jun 1, 2023 at 2:47 AM Yugo NAGATA wrote: > > On Thu, 1 Jun 2023 23:59:09 +0900 > Yugo NAGATA wrote: > > > Hello hackers, > > > > Here's a rebased version of the patch-set adding Incremental View > > Maintenance support for PostgreSQL. That was discussed in [1]. > > > [1] > > https://www

Re: Incremental View Maintenance, take 2

2023-05-31 Thread Yugo NAGATA
On Thu, 1 Jun 2023 23:59:09 +0900 Yugo NAGATA wrote: > Hello hackers, > > Here's a rebased version of the patch-set adding Incremental View > Maintenance support for PostgreSQL. That was discussed in [1]. > [1] > https://www.postgresql.org/message-id/flat/20181227215726.4d166b4874f8983a641123f