On 2022-11-30 2:30 p.m., Greg Stark wrote:
On Tue, 29 Nov 2022 at 21:16, Tom Lane <t...@sss.pgh.pa.us> wrote:
I actually think that that problem should be soluble with a
slightly different approach. The thing that feels insoluble
is that you can't do this without acquiring sufficient locks
to prevent addition of new partitions while the insertion is
in progress. That will be expensive in itself, and it will
turn ATTACH PARTITION into a performance disaster.
I think there`s a lot of room to manoeuvre here. This is a new feature
that doesn't need to be 100% complete or satisfy any existing
standard. There are lots of options for compromises that leave room
for future improvements.
1) We could just say sure ATTACH is slow if you're attaching an
non-empty partition
2) We could invent a concept like convalidated and let people attach a
partition without validating the uniqueness and then validate it later
concurrently
3) We could say ATTACH doesn't work now and come up with a better
strategy in the future
Also, don't I vaguely recall something in exclusion constraints about
having some kind of in-memory "intent" list where you declared that
you're about to insert a value, you validate it doesn't violate the
constraint and then you're free to insert it because anyone else will
see your intent in memory? There might be a need for some kind of
global object that only holds inserted keys long enough that other
sessions are guaranteed to see the key in the correct index. And that
could maybe even be in memory rather than on disk.
This isn't a simple project but I don't think it's impossible as long
as we keep an open mind about the requirements.
In the current global unique index implementation, ATTACH can be slow if
there are concurrent inserts happening. ATTACH tries to acquire
shareLock on all existing partitions and partition-to-be before it scans
and sorts them for uniqueness check. It will release them only after all
partitions have been checked. If there are concurrent inserts, ATTACH
has to wait for all inserts complete. Likewise, if ATTACH is in
progress, inserts have to wait as well. This is an issue now.
If we were to make ATTACH acquire a lower level lock (AccessShareLock),
scans a partition, and then release it. there is nothing stopping any
concurrent inserts from inserting a conflict right after it finishes
checking. This is another issue. There is no transaction level lock
being triggered here like in multiple concurent inserts case
Another email thread called "create index concurrently on partitioned
index" discuss some approaches that may be used to solve the attach
issue here, basically to allow ATTACH PARTITION CONCURRENTLY...
regards
Cary Huang
---------------------------------
HighGo Software Canada
