Re: grouping pushdown

2023-01-04 Thread Antonin Houska
David Rowley wrote: > On Wed, 4 Jan 2023 at 23:21, Spring Zhong wrote: > > The plan is apparently inefficient, since the hash aggregate goes after the > > Cartesian product. We could expect the query's performance get much > > improved if the HashAggregate node can be pushed down to the SCAN

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-04 Thread Ankit Kumar Pandey
On 05/01/23 12:53, David Rowley wrote: We *can* reuse Sorts where a more strict or equivalent sort order is available. The question is how do we get the final WindowClause to do something slightly more strict to save having to do anything for the ORDER BY. One way you might think would be

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-04 Thread David Rowley
On Thu, 5 Jan 2023 at 16:12, Tom Lane wrote: > > David Rowley writes: > > Additionally, it's also not that clear to me that sorting by more > > columns in the sort below the WindowAgg would always be a win over > > doing the final sort for the ORDER BY. What if the WHERE clause (that > > could

Re: Resolve UNKNOWN type to relevant type instead of text type while bulk update using values

2023-01-04 Thread David G. Johnston
Please don’t top-post On Wednesday, January 4, 2023, Sayyid Ali Sajjad Rizavi wrote: > Breaking working queries for this is not acceptable. > > > Good point, let's exclude Option 2. > > >> This happens when possible so any remaining cases are not possible. Or, >> at least apparently not worth

Re: An oversight in ExecInitAgg for grouping sets

2023-01-04 Thread Richard Guo
On Thu, Jan 5, 2023 at 6:18 AM David Rowley wrote: > On Tue, 3 Jan 2023 at 10:25, Tom Lane wrote: > > The thing that I find really distressing here is that it's been > > like this for years and none of our automated testing caught it. > > You'd have expected valgrind testing to do so ... but it

Re: pgsql: Delay commit status checks until freezing executes.

2023-01-04 Thread Amit Kapila
On Wed, Jan 4, 2023 at 11:30 PM Peter Geoghegan wrote: > > On Wed, Jan 4, 2023 at 7:03 AM Robert Haas wrote: > > But that having been said, I'm kind of astonished that you didn't know > > about this already. The freezing behavior is in general extremely hard > > to get right, and I guess I feel

Re: Resolve UNKNOWN type to relevant type instead of text type while bulk update using values

2023-01-04 Thread Sayyid Ali Sajjad Rizavi
> > Breaking working queries for this is not acceptable. Good point, let's exclude Option 2. > This happens when possible so any remaining cases are not possible. Or, > at least apparently not worth the effort it would take to make work. Actually this doesn't happen when all of the values

Re: Fix showing XID of a spectoken lock in an incorrect field of pg_locks view.

2023-01-04 Thread Masahiko Sawada
On Wed, Jan 4, 2023 at 6:42 PM Amit Kapila wrote: > > On Wed, Jan 4, 2023 at 12:16 PM Masahiko Sawada wrote: > > > > It seems to be confusing and the user won't get the result even if > > they search it by transactionid = 741. So I've attached the patch to > > fix it. With the patch, the

Re: Resolve UNKNOWN type to relevant type instead of text type while bulk update using values

2023-01-04 Thread David G. Johnston
On Wednesday, January 4, 2023, Sayyid Ali Sajjad Rizavi wrote: > > > *Option 1:* Cast to the relevant column type in that position (to > `integer` in this case), whenever we have an unknown type. > This happens when possible so any remaining cases are not possible. Or, at least apparently not

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-04 Thread Ankit Kumar Pandey
On 05/01/23 07:48, Vik Fearing wrote: On 1/4/23 13:07, Ankit Kumar Pandey wrote: Also, one thing, consider the following query: explain analyze select row_number() over (order by a,b),count(*) over (order by a) from abcd order by a,b,c; In this case, sorting is done on (a,b) followed by

Resolve UNKNOWN type to relevant type instead of text type while bulk update using values

2023-01-04 Thread Sayyid Ali Sajjad Rizavi
Hi ! I discovered an interesting behavior in PostgreSQL bulk update query using `from (values %s)` syntax. Let's see an example; ``` update persons p set age = t.age from ( values ('uuid1', null), ('uuid2', null) ) as t(id, age) where p.id = t.id; ``` The `age` column is of

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-04 Thread Amit Kapila
On Thu, Jan 5, 2023 at 10:16 AM Nathan Bossart wrote: > > On Wed, Jan 04, 2023 at 08:12:37PM -0800, Nathan Bossart wrote: > > On Thu, Jan 05, 2023 at 09:09:12AM +0530, Amit Kapila wrote: > >> But there doesn't appear to be any guarantee that the result for > >> AllTablesyncsReady() will change

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-04 Thread Amit Kapila
On Thu, Jan 5, 2023 at 6:19 AM Nathan Bossart wrote: > > On Wed, Jan 04, 2023 at 10:12:19AM -0800, Nathan Bossart wrote: > > From the discussion thus far, it sounds like the alternatives are to 1) add > > a global flag that causes wal_retrieve_retry_interval to be bypassed for > > all workers or

Re: fix and document CLUSTER privileges

2023-01-04 Thread Nathan Bossart
On Wed, Jan 04, 2023 at 11:27:05PM +0100, Gilles Darold wrote: > Got it, this is patch add_cluster_skip_messages.patch . IMHO this patch > should be part of this commitfest as it is directly based on this one. You > could create a second patch here that adds the warning message so that >

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-04 Thread Nathan Bossart
On Wed, Jan 04, 2023 at 08:12:37PM -0800, Nathan Bossart wrote: > On Thu, Jan 05, 2023 at 09:09:12AM +0530, Amit Kapila wrote: >> But there doesn't appear to be any guarantee that the result for >> AllTablesyncsReady() will change between the time it is invoked >> earlier in the function and at

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-04 Thread Nathan Bossart
On Thu, Jan 05, 2023 at 09:09:12AM +0530, Amit Kapila wrote: > On Wed, Jan 4, 2023 at 11:03 PM Nathan Bossart > wrote: >> On Wed, Jan 04, 2023 at 09:41:47AM +0530, Amit Kapila wrote: >> > If so, we probably also need to >> > ensure that table_states_valid is marked false probably via >> >

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-04 Thread Amit Kapila
On Wed, Jan 4, 2023 at 11:03 PM Nathan Bossart wrote: > > On Wed, Jan 04, 2023 at 09:41:47AM +0530, Amit Kapila wrote: > > I am not sure if I understand the problem you are trying to solve with > > this part of the patch. Are you worried that after we mark some of the > > relation's state as

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-04 Thread Tom Lane
David Rowley writes: > Additionally, it's also not that clear to me that sorting by more > columns in the sort below the WindowAgg would always be a win over > doing the final sort for the ORDER BY. What if the WHERE clause (that > could not be pushed down before a join) filtered out the vast

Re: pg_upgrade test failure

2023-01-04 Thread Thomas Munro
On Wed, Dec 7, 2022 at 7:15 AM Andres Freund wrote: > On 2022-11-08 01:16:09 +1300, Thomas Munro wrote: > > So [1] on its own didn't fix this. My next guess is that the attached > > might help. > What is our plan here? This afaict is the most common "false positive" for > cfbot in the last

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-04 Thread David Rowley
On Thu, 5 Jan 2023 at 15:18, Vik Fearing wrote: > > On 1/4/23 13:07, Ankit Kumar Pandey wrote: > > Also, one thing, consider the following query: > > > > explain analyze select row_number() over (order by a,b),count(*) over > > (order by a) from abcd order by a,b,c; > > > > In this case, sorting

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-04 Thread Tom Lane
Vik Fearing writes: > On 1/4/23 13:07, Ankit Kumar Pandey wrote: >> Also, one thing, consider the following query: >> explain analyze select row_number() over (order by a,b),count(*) over >> (order by a) from abcd order by a,b,c; >> In this case, sorting is done on (a,b) followed by incremental

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-04 Thread Vik Fearing
On 1/4/23 13:07, Ankit Kumar Pandey wrote: Also, one thing, consider the following query: explain analyze select row_number() over (order by a,b),count(*) over (order by a) from abcd order by a,b,c; In this case, sorting is done on (a,b) followed by incremental sort on c at final stage.

Re: New strategies for freezing, advancing relfrozenxid early

2023-01-04 Thread Matthias van de Meent
On Tue, 3 Jan 2023 at 21:30, Peter Geoghegan wrote: > > Attached is v14. Some reviews (untested; only code review so far) on these versions of the patches: > [PATCH v14 1/3] Add eager and lazy freezing strategies to VACUUM. > +/* > + * Threshold cutoff point (expressed in # of physical

Re: [PATCH] Simple code cleanup in tuplesort.c.

2023-01-04 Thread John Naylor
On Fri, Sep 16, 2022 at 1:43 PM Richard Guo wrote: > > > On Wed, Jul 27, 2022 at 5:10 PM Xing Guo wrote: >> >> The bounded heap sorting status flag is set twice in sort_bounded_heap() and tuplesort_performsort(). This patch helps remove one of them. > > > Revisiting this patch I think maybe it's

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2023-01-04 Thread Tom Lane
Thomas Munro writes: > On Thu, Jan 5, 2023 at 11:55 AM Tom Lane wrote: >> ... But if that is the direction >> we're going to go in, we should probably revise these APIs to make them >> less odd. I'm not sure why we'd keep the REG_CANCEL error code at all. > Ah, OK. I had the impression from

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-04 Thread Nathan Bossart
On Wed, Jan 04, 2023 at 10:12:19AM -0800, Nathan Bossart wrote: > From the discussion thus far, it sounds like the alternatives are to 1) add > a global flag that causes wal_retrieve_retry_interval to be bypassed for > all workers or to 2) add a hash map in the launcher and a > restart_immediately

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2023-01-04 Thread Thomas Munro
On Thu, Jan 5, 2023 at 11:55 AM Tom Lane wrote: > Andres Freund writes: > > Hm. Seems confusing for this to continue being called rcancelrequested() and > > to be called via if(CANCEL_REQUESTED()), if we're not even documenting that > > it's intended to be usable that way? > > Yeah. I'm not

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2023-01-04 Thread Andres Freund
Hi, On 2023-01-05 13:21:54 +1300, Thomas Munro wrote: > Right, I contemplated variations on that theme. I'd be willing to > code something like that to kick the tyres, but it seems like it would > make back-patching more painful? We're trying to fix bugs here... I think we need to accept that

Re: Split index and table statistics into different types of stats

2023-01-04 Thread Andres Freund
Hi, On 2023-01-03 15:19:18 +0100, Drouvot, Bertrand wrote: > diff --git a/src/backend/access/common/relation.c > b/src/backend/access/common/relation.c > index 4017e175e3..fca166a063 100644 > --- a/src/backend/access/common/relation.c > +++ b/src/backend/access/common/relation.c > @@ -73,7

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2023-01-04 Thread Thomas Munro
On Thu, Jan 5, 2023 at 12:33 PM Andres Freund wrote: > What about using a version of errsave() that can save FATALs too? We could > have something roughly like the ProcessInterrupts() in the proposed patch that > is used from within rcancelrequested(). But instead of actually throwing the >

Re: Some compiling warnings

2023-01-04 Thread David Rowley
On Wed, 4 Jan 2023 at 20:11, Richard Guo wrote: > > When trying Valgrind I came across some compiling warnings with > USE_VALGRIND defined and --enable-cassert not configured. This is > mainly because in this case we have MEMORY_CONTEXT_CHECKING defined > while USE_ASSERT_CHECKING not defined.

Re: CI and test improvements

2023-01-04 Thread Justin Pryzby
On Tue, Nov 22, 2022 at 04:57:44PM -0600, Justin Pryzby wrote: > I shuffled my branch around and sending now the current "docs" patches, > but I suppose this is waiting on the "convert CompilerWarnings task to > meson" patch. In case it's not, here's a version to do that now. >From

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2023-01-04 Thread Andres Freund
Hi, On 2023-01-04 17:55:43 -0500, Tom Lane wrote: > I'm not very happy with this line of development at all, because I think we > are painting ourselves into a corner by not allowing code to detect whether > a cancel is pending without having it happen immediately. (That is, I do > not believe

Re: CI and test improvements

2023-01-04 Thread Justin Pryzby
On Mon, Nov 21, 2022 at 02:45:42PM -0800, Andres Freund wrote: > On 2022-11-13 17:53:04 -0600, Justin Pryzby wrote: > > > > From: Justin Pryzby > > > > Date: Tue, 26 Jul 2022 20:30:02 -0500 > > > > Subject: [PATCH 6/8] cirrus/ccache: add explicit cache keys.. > > > > > > > > Since otherwise,

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2023-01-04 Thread Tom Lane
Andres Freund writes: > Hm. Seems confusing for this to continue being called rcancelrequested() and > to be called via if(CANCEL_REQUESTED()), if we're not even documenting that > it's intended to be usable that way? Yeah. I'm not very happy with this line of development at all, because I

Re: meson oddities

2023-01-04 Thread Andres Freund
Hi, On 2023-01-04 23:17:30 +0100, Peter Eisentraut wrote: > I meant the latter, which I see is already in there, but it doesn't actually > fully work. It only looks at the subdirectory (like "lib"), not the whole > path (like "/usr/local/pgsql/lib"). With the attached patch I have it > working

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2023-01-04 Thread Andres Freund
Hi, On 2023-01-04 16:46:05 +1300, Thomas Munro wrote: > postgres=# select 'x' ~ 'hello world .*'; > -[ RECORD 1 ] > ?column? | f > > postgres=# select * from pg_backend_memory_contexts where name = > 'RegexpMemoryContext'; > -[ RECORD 1 ]-+- > name |

Re: Optimizing Node Files Support

2023-01-04 Thread Tom Lane
vignesh C writes: > The patch does not apply on top of HEAD as in [1], please post a rebased > patch: Yeah. The way that I'd been thinking of optimizing the copy functions was more or less as attached: continue to write all the COPY_SCALAR_FIELD macro calls, but just make them expand to no-ops

Re: Add a test to ldapbindpasswd

2023-01-04 Thread Andrew Dunstan
On 2023-01-04 We 16:26, Andrew Dunstan wrote: > On 2023-01-02 Mo 09:45, Andrew Dunstan wrote: >> On 2023-01-01 Su 18:31, Andrew Dunstan wrote: >>> On 2023-01-01 Su 14:02, Thomas Munro wrote: On Mon, Jan 2, 2023 at 3:04 AM Andrew Dunstan wrote: > On 2022-12-19 Mo 11:16, Andrew Dunstan

Re: fix and document CLUSTER privileges

2023-01-04 Thread Gilles Darold
Le 04/01/2023 à 19:18, Nathan Bossart a écrit : On Wed, Jan 04, 2023 at 02:25:13PM +0100, Gilles Darold wrote: This is the current behavior of the CLUSTER command and current patch adds a sentence about the silent behavior in the documentation. This is good but I just want to ask if we could

Re: An oversight in ExecInitAgg for grouping sets

2023-01-04 Thread David Rowley
On Tue, 3 Jan 2023 at 10:25, Tom Lane wrote: > The thing that I find really distressing here is that it's been > like this for years and none of our automated testing caught it. > You'd have expected valgrind testing to do so ... but it does not, > because we've never marked that word NOACCESS.

Re: meson oddities

2023-01-04 Thread Peter Eisentraut
On 04.01.23 20:35, Andres Freund wrote: Unless someone comes up with a proposal to address the above broader issues, also taking into account current packaging practices etc., then I think we should do a short-term solution to either port the subdir-appending to the meson scripts or remove it

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2023-01-04 Thread Andres Freund
Hi, On 2022-12-29 00:40:52 -0800, Noah Misch wrote: > Incidentally, the affected test contains comment "# DROP TABLE containing > block which standby has in a pinned buffer". The standby holds no pin at > that moment; the LOCK TABLE pins system catalog pages, but it drops every > pin it

Re: Logical replication - schema change not invalidating the relation cache

2023-01-04 Thread Tom Lane
vignesh C writes: > [ v3-0001-Fix-for-invalidating-logical-replication-relation.patch ] (btw, please don't send multiple patch versions with the same number, it's very confusing.) I looked briefly at this patch. I wonder why you wrote a whole new callback function instead of just using

Re: Rework of collation code, extensibility

2023-01-04 Thread Peter Eisentraut
On 22.12.22 06:40, Jeff Davis wrote: On Sat, 2022-12-17 at 19:14 -0800, Jeff Davis wrote: Attached is a new patch series. I think there are enough changes that this has become more of a "rework" of the collation code rather than just a refactoring. This is a continuation of some prior

Re: meson oddities

2023-01-04 Thread Andres Freund
Hi, On 2023-01-04 16:18:38 -0500, Tom Lane wrote: > Robert Haas writes: > > If we don't do as Peter suggests, then any difference between the > > results of one build system and the other could either be a bug or an > > intentional deviation. There will be no easy way to know which it is. > >

Re: Add a test to ldapbindpasswd

2023-01-04 Thread Andrew Dunstan
On 2023-01-02 Mo 09:45, Andrew Dunstan wrote: > On 2023-01-01 Su 18:31, Andrew Dunstan wrote: >> On 2023-01-01 Su 14:02, Thomas Munro wrote: >>> On Mon, Jan 2, 2023 at 3:04 AM Andrew Dunstan wrote: On 2022-12-19 Mo 11:16, Andrew Dunstan wrote: > There is currently no test for the use of

Re: verbose mode for pg_input_error_message?

2023-01-04 Thread Andrew Dunstan
On 2023-01-02 Mo 10:44, Tom Lane wrote: > Andrew Dunstan writes: >> I've been wondering if it might be a good idea to have a third parameter >> for pg_input_error_message() which would default to false, but which if >> true would cause it to emit the detail and hint fields, if any, as well >> as

Re: meson oddities

2023-01-04 Thread Tom Lane
Robert Haas writes: > If we don't do as Peter suggests, then any difference between the > results of one build system and the other could either be a bug or an > intentional deviation. There will be no easy way to know which it is. > And if or when people switch build systems, stuff will be

Re: Getting an error if we provide --enable-tap-tests switch on SLES 12

2023-01-04 Thread Andres Freund
Hi, On 2023-01-04 17:27:55 +0530, tushar wrote: > We found that if we provide *--enable-tap-tests * switch at the time of PG > sources configuration, getting this below error > " > checking for Perl modules required for TAP tests... Can't locate IPC/Run.pm > in @INC (you may need to install the

Re: meson oddities

2023-01-04 Thread Robert Haas
On Wed, Jan 4, 2023 at 2:35 PM Andres Freund wrote: > > I think we should get the two build systems to produce the same installation > > layout when given equivalent options. > > I'm not convinced that that's the right thing to do. Distributions have > helper infrastructure for buildsystems - why

Re: pgsql: Delay commit status checks until freezing executes.

2023-01-04 Thread Peter Geoghegan
On Wed, Jan 4, 2023 at 10:41 AM Andres Freund wrote: > > It's currently possible for VACUUM to set the all-frozen bit while > > unsetting the all-visible bit, due to a race condition [1]. This is > > your long standing bug. So apparently nobody is qualified to commit > > patches in this area. > >

Re: allowing for control over SET ROLE

2023-01-04 Thread Robert Haas
On Tue, Jan 3, 2023 at 5:03 PM Noah Misch wrote: > I'd start with locations where the patch already added documentation. In the > absence of documentation otherwise, a reasonable person could think WITH SET > controls just SET ROLE. The documentation of WITH SET is a good place to list > what

Re: moving extraUpdatedCols out of RangeTblEntry (into ModifyTable)

2023-01-04 Thread Tom Lane
I wrote: > After further thought: maybe we should get radical and postpone this > work all the way to executor startup. The downside of that is having > to do it over again on each execution of a prepared plan. But the > upside is that when the UPDATE targets a many-partitioned table, > we would

Re: grouping pushdown

2023-01-04 Thread David Rowley
On Wed, 4 Jan 2023 at 23:21, Spring Zhong wrote: > The plan is apparently inefficient, since the hash aggregate goes after the > Cartesian product. We could expect the query's performance get much improved > if the HashAggregate node can be pushed down to the SCAN node. > Is someone has

Re: meson oddities

2023-01-04 Thread Andres Freund
Hi, On 2023-01-04 12:35:35 +0100, Peter Eisentraut wrote: > On 16.11.22 18:07, Andres Freund wrote: > > > > If I just want to install postgres into a prefix without 'postgresql' > > > > added in > > > > a bunch of directories, e.g. because I already have pg-$version to be > > > > in the > > > >

Re: Add index scan progress to pg_stat_progress_vacuum

2023-01-04 Thread Imseih (AWS), Sami
Thanks for the review! Addressed the comments. > "Increment the indexes completed." (dot at the end) instead? Used the commenting format being used in other places in this file with an inclusion of a double-dash. i.,e. /* Wraparound emergency -- end current index scan */ > It seems to me that

Re: pgsql: Delay commit status checks until freezing executes.

2023-01-04 Thread Andres Freund
Hi, On 2023-01-04 09:59:37 -0800, Peter Geoghegan wrote: > On Wed, Jan 4, 2023 at 7:03 AM Robert Haas wrote: > > and which functions return fully reliable results, I do > > not think you should be committing your own patches in this area. > > My mistake here had nothing to do with my own goals.

Re: fix and document CLUSTER privileges

2023-01-04 Thread Nathan Bossart
On Wed, Jan 04, 2023 at 02:25:13PM +0100, Gilles Darold wrote: > This is the current behavior of the CLUSTER command and current patch adds a > sentence about the silent behavior in the documentation. This is good but I > just want to ask if we could want to fix this behavior too or just keep >

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-04 Thread Nathan Bossart
On Wed, Jan 04, 2023 at 10:57:43AM +0530, Amit Kapila wrote: > On Tue, Jan 3, 2023 at 11:40 PM Nathan Bossart > wrote: >> My approach was to add a variable to LogicalRepWorker that indicated >> whether a worker needed to be restarted immediately. While this is a >> little weird because the

pgbench - adding pl/pgsql versions of tests

2023-01-04 Thread Hannu Krosing
Hello Hackers, The attached patch adds pl/pgsql versions of "tpcb-like" and "simple-update" internal test scripts The tests perform functionally exactly the same, but are generally faster as they avoid most client-server latency. The reason I'd like to have them as part of pgbench are two 1.

Re: pgsql: Delay commit status checks until freezing executes.

2023-01-04 Thread Peter Geoghegan
On Wed, Jan 4, 2023 at 7:03 AM Robert Haas wrote: > But that having been said, I'm kind of astonished that you didn't know > about this already. The freezing behavior is in general extremely hard > to get right, and I guess I feel if you don't understand how the > underlying functions work,

Re: psql's FETCH_COUNT (cursor) is not being respected for CTEs

2023-01-04 Thread Robert Haas
On Wed, Jan 4, 2023 at 11:36 AM Tom Lane wrote: > As you well know, psql's FETCH_COUNT mechanism is far older than > single-row mode. I don't think anyone's tried to transpose it > onto that. I agree that it seems like a good idea to try. > There will be more per-row overhead, but the increase

Re: [PATCH] CF app: add "Returned: Needs more interest"

2023-01-04 Thread Jacob Champion
On Tue, Jan 3, 2023 at 8:56 PM vignesh C wrote: > I'm not sure if this should be included in commitfest as we generally > include the postgres repository patches in the commitfest. I felt we > could have the discussion in the thread and remove the entry from > commitfest. Is there a good way to

Re: wake up logical workers after ALTER SUBSCRIPTION

2023-01-04 Thread Nathan Bossart
On Wed, Jan 04, 2023 at 09:41:47AM +0530, Amit Kapila wrote: > I am not sure if I understand the problem you are trying to solve with > this part of the patch. Are you worried that after we mark some of the > relation's state as READY, all the table syncs are in the READY state > but we will not

Re: Infinite Interval

2023-01-04 Thread jian he
On Tue, Jan 3, 2023 at 6:14 AM Joseph Koshakow wrote: > I have another patch, this one adds validations to operations that > return intervals and updated error messages. I tried to give all of the > error messages meaningful text, but I'm starting to think that almost all > of them should just

Re: psql's FETCH_COUNT (cursor) is not being respected for CTEs

2023-01-04 Thread Tom Lane
Robert Haas writes: > On Wed, Jan 4, 2023 at 10:22 AM Daniel Verite wrote: >> A solution would be for psql to use PQsetSingleRowMode() to retrieve >> results row-by-row, as opposed to using a cursor, and then allocate >> memory for only FETCH_COUNT rows at a time. > Is there any reason that

Re: psql's FETCH_COUNT (cursor) is not being respected for CTEs

2023-01-04 Thread Robert Haas
On Wed, Jan 4, 2023 at 10:22 AM Daniel Verite wrote: > A solution would be for psql to use PQsetSingleRowMode() to retrieve > results row-by-row, as opposed to using a cursor, and then allocate > memory for only FETCH_COUNT rows at a time. Incidentally it solves > other problems like queries

Re: psql's FETCH_COUNT (cursor) is not being respected for CTEs

2023-01-04 Thread Daniel Verite
Jakub Wartak wrote: > It might be a not so well known fact (?) that CTEs are not executed > with cursor when asked to do so, but instead silently executed with > potential huge memory allocation going on. Patch is attached. My one > doubt is that not every statement starting with "WITH"

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-04 Thread Ankit Kumar Pandey
Attaching test cases for this (+ small change in doc). Tested this in one of WIP branch where I had modified select_active_windows and it failed as expected. Please let me know if something can be improved in this. Regards, Ankit Kumar Pandey From 7647759eb92e1a0560bcff73b4169be8694f83d8

Re: explain analyze rows=%.0f

2023-01-04 Thread Ibrar Ahmed
On Sun, Nov 6, 2022 at 10:12 AM Tom Lane wrote: > Robert Haas writes: > > On Fri, Jul 22, 2022 at 6:47 AM Amit Kapila > wr= > ote: > >> I feel the discussion has slightly deviated which makes it unclear > >> whether this patch is required or not? > > > My opinion is that showing some

Re: pgsql: Delay commit status checks until freezing executes.

2023-01-04 Thread Robert Haas
On Wed, Jan 4, 2023 at 1:53 AM Peter Geoghegan wrote: > I think that we should definitely have a comment directly over > TransactionIdDidAbort(). Though I wouldn't mind reorganizing these > other comments, or making the comment over TransactionIdDidAbort() > mostly just point to the other

Re: Add a new pg_walinspect function to extract FPIs from WAL records

2023-01-04 Thread Drouvot, Bertrand
Hi, On 12/27/22 12:48 PM, Bharath Rupireddy wrote: Hi, Here's a patch that implements the idea of extracting full page images from WAL records [1] [2] with a function in pg_walinspect. This new function accepts start and end lsn and returns full page image info such as WAL record lsn,

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-01-04 Thread Alexander Korotkov
Hi, Pavel! On Wed, Jan 4, 2023 at 3:43 PM Pavel Borisov wrote: > On Wed, 4 Jan 2023 at 12:52, Pavel Borisov wrote: > > On Wed, 4 Jan 2023 at 12:41, vignesh C wrote: > > > > > > On Fri, 1 Jul 2022 at 16:49, Alexander Korotkov > > > wrote: > > > > > > > > Hackers, > > > > > > > > When working

Re: Data loss on logical replication, 12.12 to 14.5, ALTER SUBSCRIPTION

2023-01-04 Thread Amit Kapila
On Tue, Jan 3, 2023 at 8:50 PM Michail Nikolaev wrote: > > > Does that by any chance mean you are using a non-community version of > > Postgres which has some other changes? > > It is a managed Postgres service in the general cloud. Usually, such > providers apply some custom minor patches. > The

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-04 Thread Dilip Kumar
On Wed, Jan 4, 2023 at 6:40 PM Amit Kapila wrote: > > On Wed, Jan 4, 2023 at 4:52 PM Dilip Kumar wrote: > > > > 2. > > + * Since the database structure (schema of subscription tables, > > constraints, > > + * etc.) of the publisher and subscriber could be different, applying > > + *

Re: fix and document CLUSTER privileges

2023-01-04 Thread Gilles Darold
Le 16/12/2022 à 05:57, Nathan Bossart a écrit : Here is a new version of the patch. I've moved the privilege checks to a new function, and I added a note in the docs about clustering partitioned tables in a transaction block (it's not allowed). Getting into review of this patch I wonder why

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-04 Thread Amit Kapila
On Wed, Jan 4, 2023 at 4:52 PM Dilip Kumar wrote: > > 2. > + * Since the database structure (schema of subscription tables, constraints, > + * etc.) of the publisher and subscriber could be different, applying > + * transactions in parallel mode on the subscriber side can cause some > + *

Re: GSOC2023

2023-01-04 Thread Jesper Pedersen
Hi, On 12/28/22 21:10, diaa wrote: *Hi Sir.* I’m Computer engineering student from Egypt Interested in Database Management Systems. Can I know details about the list of ideas for 2023 projects or how to prepare myself to be ready with the required knowledge? Thanks for your interest in GSoC

Re: POC: Lock updated tuples in tuple_update() and tuple_delete()

2023-01-04 Thread Pavel Borisov
On Wed, 4 Jan 2023 at 12:52, Pavel Borisov wrote: > > Hi, Vignesh! > > On Wed, 4 Jan 2023 at 12:41, vignesh C wrote: > > > > On Fri, 1 Jul 2022 at 16:49, Alexander Korotkov > > wrote: > > > > > > Hackers, > > > > > > When working in the read committed transaction isolation mode > > >

Re: [PoC] Implementation of distinct in Window Aggregates

2023-01-04 Thread Ankit Kumar Pandey
On 29/12/22 20:58, Ankit Kumar Pandey wrote: On 24/12/22 18:22, Ankit Pandey wrote: Hi, This is a PoC patch which implements distinct operation in window aggregates (without order by and for single column aggregation, final version may vary wrt these limitations). Purpose of this PoC is to

Re: Todo: Teach planner to evaluate multiple windows in the optimal order

2023-01-04 Thread Ankit Kumar Pandey
On 04/01/23 09:32, David Rowley wrote: It looks like that works by accident. I see no mention of this either in the comments or in [1]. This kind of troubles me because function name /select_active_windows///doesn't tell me if its only job is to reorder window clauses for optimizing sort.

Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

2023-01-04 Thread vignesh C
On Tue, 1 Nov 2022 at 15:54, Etsuro Fujita wrote: > > Hi David, > > On Sat, Oct 1, 2022 at 5:54 AM David Zhang wrote: > > After rebase the file `postgres_fdw.out` and applied to master branch, > > make and make check are all ok for postgres_fdw. > > Thanks for testing! Attached is a rebased

Re: making relfilenodes 56 bits

2023-01-04 Thread vignesh C
On Fri, 21 Oct 2022 at 11:31, Michael Paquier wrote: > > On Thu, Sep 29, 2022 at 09:23:38PM -0400, Tom Lane wrote: > > Hmmm ... I'd tend to do SELECT COUNT(*) FROM. But can't we provide > > any actual checks on the sanity of the output? I realize that the > > output's far from static, but still

Re: Using AF_UNIX sockets always for tests on Windows

2023-01-04 Thread vignesh C
On Fri, 2 Dec 2022 at 18:08, Andrew Dunstan wrote: > > > On 2022-12-01 Th 21:10, Andres Freund wrote: > > Hi, > > > > On 2022-12-01 20:56:18 -0500, Tom Lane wrote: > >> Andres Freund writes: > >>> On 2022-12-01 20:30:36 -0500, Tom Lane wrote: > If we remove that, won't we have a whole lot

psql's FETCH_COUNT (cursor) is not being respected for CTEs

2023-01-04 Thread Jakub Wartak
Hi -hackers, I've spent some time fighting against "out of memory" errors coming out of psql when trying to use the cursor via FETCH_COUNT. It might be a not so well known fact (?) that CTEs are not executed with cursor when asked to do so, but instead silently executed with potential huge memory

Re: Understanding, testing and improving our Windows filesystem code

2023-01-04 Thread vignesh C
On Tue, 25 Oct 2022 at 09:42, Thomas Munro wrote: > > I pushed the bug fixes from this series, without their accompanying > tests. Here's a rebase of the test suite, with all those tests now > squashed into the main test patch, and also the > tell-Windows-to-be-more-like-Unix patch. Registered

Re: Reduce timing overhead of EXPLAIN ANALYZE using rdtsc?

2023-01-04 Thread David Geier
Hi, CFBot shows some compilation errors as in [1], please post an updated version for the same: 09:08:12.525] /usr/bin/ld: src/bin/pg_test_timing/pg_test_timing.p/pg_test_timing.c.o: warning: relocation against `cycles_to_sec' in read-only section `.text' [09:08:12.525] /usr/bin/ld:

Getting an error if we provide --enable-tap-tests switch on SLES 12

2023-01-04 Thread tushar
Hi, We found that if we provide *--enable-tap-tests * switch at the time of PG sources configuration, getting this below error " checking for Perl modules required for TAP tests... Can't locate IPC/Run.pm in @INC (you may need to install the IPC::Run module) (@INC contains:

Re: meson oddities

2023-01-04 Thread Peter Eisentraut
On 16.11.22 18:07, Andres Freund wrote: If I just want to install postgres into a prefix without 'postgresql' added in a bunch of directories, e.g. because I already have pg-$version to be in the prefix, there's really no good way to do so - you can't even specify --sysconfdir or such, because

Re: Perform streaming logical transactions by background workers and parallel apply

2023-01-04 Thread Dilip Kumar
On Wed, Jan 4, 2023 at 4:25 PM houzj.f...@fujitsu.com wrote: > > Attach the new patch set. > Apart from addressing Sawada-San's comments, I also did some other minor > changes in the patch: I have done a high-level review of 0001, and later I will do a detailed review of this while reading

Re: [PATCH] Improve ability to display optimizer analysis using OPTIMIZER_DEBUG

2023-01-04 Thread Ankit Kumar Pandey
On 04/01/23 06:27, David Rowley wrote: I think it's better you leave this then. I think if someone comes along and demonstrates the feature's usefulness and can sell us having it so we can easily enable it by GUC then maybe that's the time to consider it. I don't think ticking off a TODO item

GSOC2023

2023-01-04 Thread diaa
Hi Sir.I’m Computer engineering student from Egypt Interested in Database Management Systems.Can I know details about the list of ideas for 2023 projects or how to prepare myself to be ready with the required knowledge?Please if you can help me don't ignore my email.Sincerely.Diaa Badr.

grouping pushdown

2023-01-04 Thread Spring Zhong
Hi hackers, I came across a problem on how to improve the performance of queries with GROUP BY clause when the grouping columns have much duplicate data. For example: create table t1(i1) as select 1 from generate_series(1,1); create table t2(i2) as select 2 from generate_series(1,1);

Re: on placeholder entries in view rule action query's range table

2023-01-04 Thread vignesh C
On Fri, 9 Dec 2022 at 12:20, Amit Langote wrote: > > On Fri, Dec 9, 2022 at 3:07 PM Amit Langote wrote: > > On Thu, Dec 8, 2022 at 6:12 PM Alvaro Herrera > > wrote: > > > On 2022-Dec-07, Amit Langote wrote: > > > > However, this > > > > approach of not storing the placeholder in the stored

Re: Reduce timing overhead of EXPLAIN ANALYZE using rdtsc?

2023-01-04 Thread vignesh C
On Tue, 3 Jan 2023 at 14:08, David Geier wrote: > > Hi Lukas, > > On 1/2/23 20:50, Lukas Fittl wrote: > > Thanks for continuing to work on this patch, and my apologies for > > silence on the patch. > > It would be great if you could review it. > Please also share your thoughts around exposing the

Re: WIP: Aggregation push-down - take2

2023-01-04 Thread vignesh C
On Thu, 17 Nov 2022 at 16:34, Antonin Houska wrote: > > Tomas Vondra wrote: > > > Hi, > > > > I did a quick initial review of the v20 patch series. I plan to do a > > more thorough review over the next couple days, if time permits. In > > general I think the patch is in pretty good shape. > >

Re: Prefetch the next tuple's memory during seqscans

2023-01-04 Thread vignesh C
On Wed, 23 Nov 2022 at 03:28, David Rowley wrote: > > On Thu, 3 Nov 2022 at 06:25, Andres Freund wrote: > > Attached is an experimental patch/hack for that. It ended up being more > > beneficial to make the access ordering more optimal than prefetching the > > tuple > > contents, but I'm not at

Re: pgsql: Delay commit status checks until freezing executes.

2023-01-04 Thread Peter Geoghegan
On Tue, Jan 3, 2023 at 4:54 PM Andres Freund wrote: > There's some changes from TransactionIdDidCommit() to !TransactionIdDidAbort() > that don't look right to me. If the server crashed while xid X was > in-progress, TransactionIdDidCommit(X) will return false, but so will >

Re: pgsql: Delay commit status checks until freezing executes.

2023-01-04 Thread Andres Freund
Hi, On 2023-01-03 19:23:41 +, Peter Geoghegan wrote: > Delay commit status checks until freezing executes. > > pg_xact lookups are relatively expensive. Move the xmin/xmax commit > status checks from the point that freeze plans are prepared to the point > that they're actually executed.

  1   2   >