Re: Deadlock risk while inserting directly into partition?

2021-06-28 Thread Laurenz Albe
On Mon, 2021-06-28 at 23:46 +1200, David Rowley wrote: > On Sat, 26 Jun 2021 at 16:41, Justin Pryzby wrote: > > On Thu, Jun 24, 2021 at 10:27:06AM +1200, David Rowley wrote: > > > I think the reasons for doing operations directly on partitions are > > > being reduced with each release. What

Re: Deadlock risk while inserting directly into partition?

2021-06-28 Thread David Rowley
On Mon, 28 Jun 2021 at 15:59, Amit Langote wrote: > > On Fri, Jun 25, 2021 at 10:26 AM David Rowley wrote: > > I wonder, since we can't delay taking locks until after run-time > > pruning due to being unable to invalidate cached plans, maybe instead > > we could tag on any PartitionPruneInfo

Re: Deadlock risk while inserting directly into partition?

2021-06-28 Thread David Rowley
On Sat, 26 Jun 2021 at 16:41, Justin Pryzby wrote: > > On Thu, Jun 24, 2021 at 10:27:06AM +1200, David Rowley wrote: > > I think the reasons for doing operations directly on partitions are > > being reduced with each release. What operations do people really > > need to do on partitions now?

Re: Deadlock risk while inserting directly into partition?

2021-06-28 Thread Amit Kapila
On Mon, Jun 28, 2021 at 9:50 AM Amit Langote wrote: > > On Thu, Jun 24, 2021 at 7:27 AM David Rowley wrote: > > On Wed, 23 Jun 2021 at 21:07, Amit Kapila wrote: > > > I noticed that while inserting directly into a partition table we > > > compute the PartitionCheckExpr by traversing all the

Re: Deadlock risk while inserting directly into partition?

2021-06-28 Thread Amit Langote
On Mon, Jun 28, 2021 at 12:58 PM Amit Langote wrote: > On Fri, Jun 25, 2021 at 10:26 AM David Rowley wrote: > > On Thu, 24 Jun 2021 at 12:32, David Rowley wrote: > > > The overhead of taking these locks is pretty significant for > > > partitioned tables with lots of partitions where only 1 of

Re: Deadlock risk while inserting directly into partition?

2021-06-27 Thread Amit Langote
On Thu, Jun 24, 2021 at 7:27 AM David Rowley wrote: > On Wed, 23 Jun 2021 at 21:07, Amit Kapila wrote: > > I noticed that while inserting directly into a partition table we > > compute the PartitionCheckExpr by traversing all the parent partitions > > via > >

Re: Deadlock risk while inserting directly into partition?

2021-06-27 Thread Amit Langote
On Fri, Jun 25, 2021 at 10:26 AM David Rowley wrote: > On Thu, 24 Jun 2021 at 12:32, David Rowley wrote: > > The overhead of taking these locks is pretty significant for > > partitioned tables with lots of partitions where only 1 of them > > survives run-time partition pruning. That's really

Re: Deadlock risk while inserting directly into partition?

2021-06-25 Thread Justin Pryzby
On Thu, Jun 24, 2021 at 10:27:06AM +1200, David Rowley wrote: > I think the reasons for doing operations directly on partitions are > being reduced with each release. What operations do people really > need to do on partitions now? TRUNCATE is probably one, maybe there's > still a need to CREATE

Re: Deadlock risk while inserting directly into partition?

2021-06-24 Thread David Rowley
On Thu, 24 Jun 2021 at 12:32, David Rowley wrote: > The overhead of taking these locks is pretty significant for > partitioned tables with lots of partitions where only 1 of them > survives run-time partition pruning. That's really terrible for > people that want to PREPARE queries and just look

Re: Deadlock risk while inserting directly into partition?

2021-06-24 Thread Simon Riggs
On Thu, Jun 24, 2021 at 1:45 AM Tom Lane wrote: > > David Rowley writes: > > ... What I mean is that as we add > > more and more fixes to improve performance of partitioning, that there > > comes a point where the ability to directly reference partitions is a > > hindrance rather than something

Re: Deadlock risk while inserting directly into partition?

2021-06-23 Thread David Rowley
On Thu, 24 Jun 2021 at 12:45, Tom Lane wrote: > I don't think that the > ability to access partitions directly is a material problem here; > I doubt that we need to lock every partition in the plan when run-time > routing is working (surely we only need to lock the partition mapping); > and most

Re: Deadlock risk while inserting directly into partition?

2021-06-23 Thread Tom Lane
David Rowley writes: > ... What I mean is that as we add > more and more fixes to improve performance of partitioning, that there > comes a point where the ability to directly reference partitions is a > hindrance rather than something that's useful. Right now that > hindrance is the fact that we

Re: Deadlock risk while inserting directly into partition?

2021-06-23 Thread Tom Lane
David Rowley writes: > On Thu, 24 Jun 2021 at 10:38, Tom Lane wrote: >> I strongly disagree. That's essentially betting the whole farm on >> our always being able to optimize parent-level operations fully, >> which I do not think we are anywhere close to. > Did you have anything in particular

Re: Deadlock risk while inserting directly into partition?

2021-06-23 Thread David Rowley
On Thu, 24 Jun 2021 at 10:38, Tom Lane wrote: > > David Rowley writes: > > I've recently been thinking it would be good if you were unable to > > access partitions directly by name at all. > > I strongly disagree. That's essentially betting the whole farm on > our always being able to optimize

Re: Deadlock risk while inserting directly into partition?

2021-06-23 Thread Tom Lane
David Rowley writes: > I've recently been thinking it would be good if you were unable to > access partitions directly by name at all. I strongly disagree. That's essentially betting the whole farm on our always being able to optimize parent-level operations fully, which I do not think we are

Re: Deadlock risk while inserting directly into partition?

2021-06-23 Thread David Rowley
On Wed, 23 Jun 2021 at 21:07, Amit Kapila wrote: > I noticed that while inserting directly into a partition table we > compute the PartitionCheckExpr by traversing all the parent partitions > via > ExecPartitionCheck()->RelationGetPartitionQual()->generate_partition_qual(). > We take

RE: Deadlock risk while inserting directly into partition?

2021-06-23 Thread houzj.f...@fujitsu.com
On Wednesday, June 23, 2021 5:07 PM Amit Kapila wrote: > I noticed that while inserting directly into a partition table we compute the > PartitionCheckExpr by traversing all the parent partitions via > ExecPartitionCheck()->RelationGetPartitionQual()->generate_partition_qual(). > We take

Deadlock risk while inserting directly into partition?

2021-06-23 Thread Amit Kapila
I noticed that while inserting directly into a partition table we compute the PartitionCheckExpr by traversing all the parent partitions via ExecPartitionCheck()->RelationGetPartitionQual()->generate_partition_qual(). We take AccessShareLock on parent tables while generating qual. Now, on the