ve (for example) ScanKey functionality, which can work with different
indexes (and such a functions has a uint16 argument for strategy number).
Or are you talking about a single huge enum for all index types? I don't
mind trying to do something like this, but I'm not sure how
"beautiful" it will be.
--
Best regards,
Daniil Davydov
Hi,
On Tue, Aug 19, 2025 at 6:31 PM Matheus Alcantara
wrote:
>
> On Tue Aug 19, 2025 at 12:57 AM -03, Daniil Davydov wrote:
> > You have started a very long transaction, which holds its xid and prevents
> > vacuum from freezing it. But what if the backend is stuck not inside
Hi,
On Tue, Aug 19, 2025 at 7:14 AM Matheus Alcantara
wrote:
>
> On Wed Aug 13, 2025 at 4:29 PM -03, Daniil Davydov wrote:
> >
> > What exactly do we mean by "active listener"? According to the source code,
> > the active listener (as far as I understand) is th
.
2) Add try/catch block to the parallel_vacuum_process_all_indexes, so we can
release workers even after an error. This required adding a static
variable to account
for the total number of reserved workers (av_nworkers_reserved).
3) Cap autovacuum_max_parallel_workers by max_worker_processes only ins
c1;
session2=# listen c1;
session3=# notify c1; // sets minimal xid to N inside StoreEntryXid
session3=# notify c1;
session1=# listen c1; // sets minimal xid to (N + 1) inside ReleaseEntryXid
After these steps, session2 still needs to process a message with xid = N, but
autovacuum legally can freeze it and truncate clog.
Maybe we should check whether there are no listeners with smaller queue
position compared to the current backend?
4)
After ReleaseEntryXid, minXid field contains not actually minimal xid
in the queue,
but maximum processed xid. Maybe this parameter can be renamed?
--
Best regards,
Daniil Davydov
I guess that "ArchiveCallbacks->check_configured_cb != NULL" should be
placed at the beginning of the while loop (by analogy with
pgarch_ArchiverCopyLoop).
6)
Please, run pgindent on your patch.
--
Best regards,
Daniil Davydov
first I'd like to
settle the core logic of the patch.
I attach a v7 patch to this letter. No changes yet, just rebased on the newest
commit in master branch.
[1]
https://www.postgresql.org/message-id/flat/CAJDiXghoi-FM4d5XVZzUyiuhv8DDm9JdGOU8KC47emasqi1GUw%40mail.gmail.com
[
cuum_max_threshold
> autovacuum_vacuum_scale_factor
> autovacuum_vacuum_threshold
>
Good catch, I'll fix it.
Thank you for the review! Please, see v9 patches :
1) Run pgindent + rebase patches on newest commit in master.
2) Introduce changes for documentation.
3) Rename local varia
g), but I didn't find any.
BTW, I think that "phdr->pd_special == phdr->pd_upper" is the best way
to decide whether we need to set the XLOG_HEAP_INIT_PAGE
flag to WAL record (at least this would eliminate the occurrence of the
described problem).
--
Best regards,
Daniil Davydov
Hi,
On Fri, Jul 18, 2025 at 2:43 AM Masahiko Sawada wrote:
>
> On Mon, Jul 14, 2025 at 3:49 AM Daniil Davydov <3daniss...@gmail.com> wrote:
> >
> > This log level is used only "for messages about parallel workers launched".
> > I think that such logs re
< nworkers)
> + {
> + /* Provide as many workers as we can. */
> + can_launch = AutoVacuumShmem->av_available_parallel_workers;
> + AutoVacuumShmem->av_available_parallel_workers = 0;
> + }
> + else
> + {
> + /* OK, we can provide al
Hi,
On Tue, Jul 8, 2025 at 10:20 PM Matheus Alcantara
wrote:
>
> On Sun Jul 6, 2025 at 5:00 AM -03, Daniil Davydov wrote:
> > I will keep the 'max_worker_processes' limit, so autovacuum will not
> > waste time initializing a parallel context if there is no chan
ltpw))
> +
>
Yes, this is the relic of a past implementation. I'll delete this macro.
>
> I've made some tests and I can confirm that is working correctly for
> what I can see. I think that would be to start include the documentation
> changes, what do you think?
>
Hi,
On Thu, Jul 3, 2025 at 9:18 PM Yugo Nagata wrote:
>
> On Tue, 1 Jul 2025 18:56:11 +0700
> Daniil Davydov <3daniss...@gmail.com> wrote:
>
> > CatalogIndexInsert is kinda "popular" function. It can be called in
> > different situations, not in all of wh
Hi,
On Fri, Jul 4, 2025 at 7:10 AM Fujii Masao wrote:
>
> Patch pushed. Thanks, everyone!
>
Glad to hear this. Thanks!
--
Best regards,
Daniil Davydov
Postgres Professional
Hi,
On Wed, Jul 2, 2025 at 6:24 PM Peter Eisentraut wrote:
>
> On 30.06.25 05:21, Daniil Davydov wrote:
> > Hi,
> > I noticed that some asserts and cycles use magic numbers 1 and 0
> > instead of BTLessStrategyNumber and InvalidStrategy.
> > At the same time, th
Hi,
On Tue, Jul 1, 2025 at 5:47 PM Yugo Nagata wrote:
>
> On Mon, 30 Jun 2025 18:32:47 +0700
> Daniil Davydov <3daniss...@gmail.com> wrote:
>
> > On Mon, Jun 30, 2025 at 3:47 PM Yugo Nagata wrote:
> > >
> > > I agree with Tom Lane that the
Hi,
On Mon, Jun 30, 2025 at 3:47 PM Yugo Nagata wrote:
>
> On Fri, 27 Jun 2025 18:53:02 +0700
> Daniil Davydov <3daniss...@gmail.com> wrote:
> > This patch fixes postgres behavior if I first create a function and
> > then try to CREATE OR REPLACE it in concurrent t
).
--
Best regards,
Daniil Davydov
From 67d2125240021f3f0f25d56630dad1d70940b40d Mon Sep 17 00:00:00 2001
From: Daniil Davidov
Date: Mon, 30 Jun 2025 10:10:24 +0700
Subject: [PATCH v1] Get rid of magic numbers
---
src/backend/access/brin/brin_minmax.c | 4 ++--
src/backend/access/brin
/www.postgresql.org/message-id/flat/CAJDiXghv2JF5zbLyyybokWKM%2B-GYsTG%2Bhw7xseLNgJOJwf0%2B8w%40mail.gmail.com
--
Best regards,
Daniil Davydov
nking about
> how your application behaves. Why do you need to have multiple
> transactions creating the same function at the same time?
>
Yep, I agree that this use case is bad, but some clients use such
logic and think that mentioned errors are a bug.
So I decided to clarify this moment.
--
Best regards,
Daniil Davydov
but nobody has enough time to fix it.
My question is, can we consider this behavior to be a mistake or a
normal occurrence?
[1] https://www.postgresql.org/message-id/flat/4B9EA2FD.8090902%40agliodbs.com
--
Best regards,
Daniil Davydov
Hi,
On Wed, Jun 18, 2025 at 5:37 AM Masahiko Sawada wrote:
>
> On Sun, May 25, 2025 at 10:22 AM Daniil Davydov <3daniss...@gmail.com> wrote:
> >
> > Thanks everybody for feedback! I attach a v4 patch to this letter.
> > Main features :
> > 1) 'paralle
Hi,
On Thu, Jun 12, 2025 at 2:40 AM Tom Lane wrote:
>
> Daniil Davydov <3daniss...@gmail.com> writes:
> > Ability to do a DROP temporary table of other session leads to error :
>
> [ shrug... ] Don't do that. A superuser is capable of doing lots
> of things t
ables. I described them and suggested corrections in
this thread [1].
[1]
https://www.postgresql.org/message-id/flat/CAJDiXgj72Axj0d4ojKdRWG_rnkfs4uWY414NL%3D15sCvh7-9rwg%40mail.gmail.com
--
Best regards,
Daniil Davydov
From 7775fc7afbefceebf88d8be1cd94da939807035c Mon Sep 17 00:00:00 2001
From: Daniil
mail.gmail.com
Sorry, this is an outdated thread. I planned to continue to discuss
mentioned topic here :
https://www.postgresql.org/message-id/CAJDiXghdFcZ8=nh4g69te7irr3q0ufyxxb3zdg09_gtnzxw...@mail.gmail.com
--
Best regards,
Daniil Davydov
'data.csv' with (freeze);
insert into test select generate_series(200, 300);
delete from test where id % 50 = 0;
commit;
--
Best regards,
Daniil Davydov
To measure the time, I hardcoded calls of GetCurrentTimestamp and
TimestampDifference.
I got ~7% improvement with the patch. Note, that table had only 2
forks - main and fsm (I haven't figured out how to force postgres to
create a visibility map for temp table within the transaction block).
--
Best regards,
Daniil Davydov
add a "break" right after calling InvalidateLocalBuffer()?
> Since the buffer has already been invalidated, continuing the loop
> may not be necessary.
Thanks for the review! I'll fix both remarks. Please see the v2 patch.
--
Best regards,
Daniil Davydov
From 203be6d23be59db2
Hi,
On Sat, May 31, 2025 at 7:49 AM Michael Paquier wrote:
>
> On Fri, May 30, 2025 at 06:01:16PM +0700, Daniil Davydov wrote:
> > For now we fully scan local buffers for each fork of the temporary
> > relation that we want to truncate (in order to drop its buffers). It
le and indexes. What if we collect all this information in
advance and iterate over the local/shared buffers only once?
I understand that it will look kinda ugly, but it will increase
performance for sure.
--
Best regards,
Daniil Davydov
From 7b3c3e816b502a94d1ccc6889d86e07d2e1555fd Mon Sep 17 00:00:
Hi,
On Fri, May 23, 2025 at 6:12 AM Masahiko Sawada wrote:
>
> On Thu, May 22, 2025 at 12:44 AM Daniil Davydov <3daniss...@gmail.com> wrote:
> >
> > On Wed, May 21, 2025 at 5:30 AM Masahiko Sawada
> > wrote:
> > >
> > > I find that the name
that N workers
(from bgworkers pool) are *always* at our disposal. And when we use
such workers we are not dependent on other operations in the cluster
and we don't interfere with other operations by taking resources away
from them.
If we give the user too much freedom in parallel index a
Hi,
On Wed, May 21, 2025 at 9:59 AM Fujii Masao wrote:
> I've pushed the patch. Thanks!
>
Glad to hear it, thank you!
--
Best regards,
Daniil Davydov
.pl is not verifying anything.
I'll do as you asked (add all meson and Make stuff), but please don't
focus on it. The creation of the real test is still in progress. (I'll
try to complete it as soon as possible).
In this letter I will divide the patch into 2 parts : implementation
and sand
.nabortstats, parsed.abortstats);
***
I suppose to add initialization of `nstats` and `nabortstats` to
ParsePrepareRecord (see attached patch).
--
Best regards,
Daniil Davydov
From 6338b32aeb170dfd23ae6d313ab2fb77058e0645 Mon Sep 17 00:00:00 2001
From: Daniil Davidov
Date: Thu, 15 May 2025 12:19:0
are targeting on master branch
(bc35adee8d7ad38e7bef40052f196be55decddec)
--
Best regards,
Daniil Davydov
From cfb7e675d9a1b05aef0cdaeeca5f6edd4bcd3b70 Mon Sep 17 00:00:00 2001
From: Daniil Davidov
Date: Sat, 10 May 2025 01:07:42 +0700
Subject: [PATCH v1] Parallel index autovacuum with bgworkers
ing bgworkers
instead of additional a/v workers. BTW, what do you think about this
idea?
--
Best regards,
Daniil Davydov
me technical difficulties to
> implement it without new infrastructure, we can revisit this approach.
OK, in the near future I'll implement it and send a new patch to this
thread. I'll be glad if you will take a look on it)
[1]
https://www.postgresql.org/message-id/CAA5RZ0vfBg%3Dc_0Sa1Tpxv8tueeBk8C5qTf9TrxKBbXUqPc99Ag%40mail.gmail.com
--
Best regards,
Daniil Davydov
arameter (perhaps with
the addition of a new one, as I wrote in the previous letter).
--
Best regards,
Daniil Davydov
can jump dramatically, which is unexpected for
the user. Autovacuum will also be dependent on other resources
(bgworkers pool). The current design does not imply this.
I wanted to create a patch that would fit into the existing mechanism
without drastic innovations. But if you think that the above is not so
important, then we can reuse VACUUM PARALLEL code and it would
simplify the final implementation)
--
Best regards,
Daniil Davydov
.
>
> Increasing a/v workers will not help speed up a specific table, what I
> am suggesting is that instead of speeding up one table, let's just allow
> other tables to not be starved of a/v cycles due to lack of a/v workers.
OK, I got it. But what if vacuuming of a single table will take (for
example) 60% of all time? This is still a possible situation, and the
fast vacuum of all other tables will not help us.
--
Best regards,
Daniil Davydov
n that I
mentioned above.
Also, we don't consume additional resources during autovacuum in this
patch - total number of a/v workers always <= autovacuum_max_workers.
BTW, see v2 patch, attached to this letter (bug fixes) :-)
--
Best regards,
Daniil Davydov
From 1c93a729b844a1dfe109e8d9e54d5cc0
ove - we don't want to consume additional resources. Also
we don't want to complicate communication between processes (the idea
is that a/v workers can only send signals to the a/v launcher).
As a result, we created our own implementation of parallel index
processing control - see changes in vacuumparallel.c and autovacuum.c.
--
Best regards,
Daniil Davydov
check any Relation's flags here. We are
checking `RelationIsValid((bmr).rel) ?` to decide whether
BufferManagerRelation was created via BMR_REL or BMR_SMGR.
If the `rel` field is not NULL, we can definitely say that BMR_REL was
used, so we should call RelationGetSmgr in order
we see this prefix.`
I will be glad to hear your opinion.
--
Best regards,
Daniil Davydov
[1]
https://www.postgresql.org/message-id/CAJDiXgj72Axj0d4ojKdRWG_rnkfs4uWY414NL%3D15sCvh7-9rwg%40mail.gmail.com
[2]
https://www.postgresql.org/message-id/CAJDiXgj%2B5UKLWSUT5605rJhuw438NmEK
; where you
can do some experiments with append-optimized tables.
I hope that patch (targeted on 'master' branch,
2c7bd2ba507e273f2d7fe1b2f6d30775ed4f3c09) will be useful for this
thread.
--
Best regards,
Daniil Davydov
From 224378c11d270aabe28bdd32efacd37ed1984bd1 Mon Sep 17 00:00:00 2001
From
at is being
discussed here. I may start a new discussion in the near future, but
as part of this discussion I would like to make sure that people agree
with the statements from my previous letter.
Thanks for your help.
--
Best regards,
Daniil Davydov
same if block, once with schemaname null (in
> makeRangeVar) and once with it populated (end of
> makeRangeVarFromQualifiedName) is a bit annoying.
>
> makeRangeVar has the same problem, assuming permanent when the schemaname
> argument is null.
Thank you for noticing it. I s
Hi,
On Mon, Mar 17, 2025 at 10:09 PM David G. Johnston
wrote:
>
> On Monday, March 17, 2025, Daniil Davydov <3daniss...@gmail.com> wrote:
>>
>>
>> >
>> > 2."you have not any temporary relations" --> "you have no any temporary
>&
ler from making mistakes, but we are giving
the wrong error message.
In my realization the caller will get the correct error message, and I
still think that we should keep v3 patch logic.
--
Best regards,
Daniil Davydov
where you can find if
(!OidIsValid(myTempNamespace)) check.
--
Best regards,
Daniil Davydov
instead of strncmp for exact matching.
Great catch! I'll fix it. Please, see v3 patch.
--
Best regards,
Daniil Davydov
From 74dc11678c55eb80ad48ad44fc77d770620ea0a7 Mon Sep 17 00:00:00 2001
From: Daniil Davidov
Date: Mon, 17 Mar 2025 15:43:15 +0700
Subject: [PATCH v3] Fix accessing other session
ng in previous letter (idk how to fix it in gmail client)
--
Best regards,
Daniil Davydov
From e5f568ece877a33d307ac6480cef9dd088412aae Mon Sep 17 00:00:00 2001
From: Daniil Davidov
Date: Mon, 17 Mar 2025 15:43:15 +0700
Subject: [PATCH v2] Fix accessing other sessions temp tables
---
src/ba
ere is a bug in the code that mistakenly
recognizes the table in such a case as permanent (not temporary), so
we cannot do what I just described. So, we have to get rid of this bug
and decide whether we reserve the right for the superuser to DROP such
tables.
I hope this remark will be useful.
--
Best regards,
Daniil Davydov
ithin
session) can access the temp tables of session. We want CRUD
operations to be performed only by the owner.
--
Best regards,
Daniil Davydov
[1]
https://www.postgresql.org/message-id/CAJDiXgj6TBzn=6ezx7+9bna9hpbitbu+muv-n3mhen_zs3n...@mail.gmail.com
ning the
resolution of this issue.
> I suggest adding a GUC that will allow superuser to do this
Waiting for your feedback on this issue :)
--
Best regards,
Daniil Davydov
this patch. Should we add it there?
--
Best regards,
Daniil Davydov
Hi,
I am posting the new v2 patch, which is now rebased on the `master` branch.
Waiting for your feedback :)
--
Best regards,
Daniil Davydov
From 2e43a1411ebcb37b2c0c1d6bac758e48799d2c4e Mon Sep 17 00:00:00 2001
From: Daniil Davidov
Date: Tue, 11 Mar 2025 17:11:16 +0700
Subject: [PATCH v2] Add
ues by step)
takes 40% less time to complete.
P.S.
v2 patch looks a bit scary, because it contains a lot of raw I/O
operations. At the moment, it is the best I came up with, but the
logic of the code (I hope) is correct.
--
Best regards,
Daniil Davydov
From bcca199a1787399c58c959a4357c2daefb8335
about 3-4%).
I attach a patch that adds Tuplestore to v25. What do you think about this idea?
--
Best regards,
Daniil Davydov
From a59cfcbb05bb07c94a4c0ad6531baa5e531629ae Mon Sep 17 00:00:00 2001
From: Daniil Davidov
Date: Sun, 9 Mar 2025 16:37:44 +0700
Subject: [PATCH] Replace holding tuples in
l tests for pg_resetwal (regression is called
to make sure that all postgres components are working correctly, but maybe
it can be replaced with something more "compact").
What do you think about the idea of adding such functionality?
--
Best regards,
Daniil Davydov
From 2993b376674cc0b565abd42
-N3mHeN_Zs3NBDw%40mail.gmail.com
What do you think about this idea?
--
Best regards,
Daniil Davydov
on
> change
> to keep the buildsystems in sync.
Thank you for your comment. I'm applying a patch that also includes
configure.ac, meson_options.txt and meson.build changes.
It would also be interesting to know what you think about the idea of
allowing such a parameter to be configured.
.
--
Best regards,
Daniil Davydov
From d962b1a9b5e2852706e256abbb6d9ca5155842a7 Mon Sep 17 00:00:00 2001
From: Daniil Davidov
Date: Thu, 6 Feb 2025 14:34:36 +0700
Subject: [PATCH] SLRU_PAGES_PER_SEGMENT as configure parameter
---
configure | 52
src
uot; field
(check attached patch for REL_17_STABLE).
What do you think about this?
--
Best regards,
Daniil Davydov
From 828e798d2a4d42aa901f4c02e9a0c76ebb057c41 Mon Sep 17 00:00:00 2001
From: Daniil Davidov
Date: Mon, 27 Jan 2025 18:36:10 +0700
Subject: [PATCH] Add marcos for safety access to smgr field
want to monitor the correctness of these variables, it might be
better to add a few checks, as in the attached patch.
I'm not sure if the comment in the error message is written correctly,
but first I would like to hear your opinion.
--
Best regards,
Daniil Davydov
From 04e41969ef9ea2c96af218a5
ected
behavior for users.
I also agree that taking it into account will reduce performance, but
maybe we can clarify this aspect in documentation (?)
--
Best regards,
Daniil Davydov
uch cases to additionally scan pg_class.
I would like to know your opinion.
--
Best regards,
Daniil Davydov
pgc and .c files (like in attached patch)
--
Best regards,
Daniil Davydov
From 36b01661fd6b9083b4cd1620020ebf4f737740b9 Mon Sep 17 00:00:00 2001
From: Daniil Davidov
Date: Tue, 10 Dec 2024 12:34:59 +0700
Subject: [PATCH] Fix refer to a null pointer
---
src/interfaces/ecpg/test/expected/sql-by
est way out of the situation would be to skip
locking temporary tables for
extension
--
Best regards,
Daniil Davydov
From da84d916e568be0d2414303f4a1e1d01b0bc6abd Mon Sep 17 00:00:00 2001
From: Daniil Davidov
Date: Fri, 6 Dec 2024 16:41:53 +0700
Subject: [PATCH] Fix bug with locking temp relation for e
(BufferDescriptorGetContentLock(bufHdr),
LW_EXCLUSIVE));
***
They fails because LockBuffer function just skip acquiring lock for
temporary table :
***
if (BufferIsLocal(buffer))
return;
***
I suppose that fix can be done as in the attached to this email patch
(for REL_17_STABLE)
--
Best regards,
Daniil
o another table (with
the exception of POST_ALTER object access hook, but it looks more like
a hack).
--
Best regards,
Daniil Davydov
uot;could not access temporary relations of other sessions".
> --
> Regards,
> Rafia Sabih
> CYBERTEC PostgreSQL International GmbH
>
Thanks for your comment. I attach a patch with a fixed error message.
Also you can find it in commit fest
(https://commitfest.postgresql.org/51/537
gly, but I think it's inevitable, given that we
don't know exactly what the name of the temporary schema of other
session will be.
--
Best regards,
Daniil Davydov
From 995002793b0af57b660565e6027615c083d8ce5e Mon Sep 17 00:00:00 2001
From: Daniil Davidov
Date: Mon, 28 Oct 2024 17:19:07 +
removal of
orphan tables. But the autovacuum is responsible for this and it uses a
different functionality. I'm wondering if there are any other cases. If
not, can we just handle it for example in ExecDropStmt and produce an error
like "cannot drop temporary tables of other sessions"?
emporary
namespace. But there is a patch (6faca9ae2878c8f642a2e5748d2dbb2b91341bec)
that protects us from race condition in this case.
--
Best regards,
Daniil Davydov
On Tue, Oct 22, 2024 at 5:55 PM Tom Lane wrote:
> Maxim Orlov writes:
> > But for the second one: do we really need any lock for temp rel
bb2b91341bec)
that protects us from race condition in this case. -- Best regards,
Daniil Davydov.
78 matches
Mail list logo
