On Tue, Jan 11, 2022 at 2:11 PM Konstantin Knizhnik wrote:
>
> We have faced with the similar problem in Zenith (open source Aurora)
> and have to implement back pressure mechanism to prevent overflow of WAL
> at stateless compute nodes
> and too long delays of [age reconstruction. Our
On 11.01.2022 03:06, Bossart, Nathan wrote:
I noticed this thread and thought I'd share my experiences building
something similar for Multi-AZ DB clusters [0]. It's not a strict RPO
mechanism, but it does throttle backends in an effort to keep the
replay lag below a configured maximum. I
I noticed this thread and thought I'd share my experiences building
something similar for Multi-AZ DB clusters [0]. It's not a strict RPO
mechanism, but it does throttle backends in an effort to keep the
replay lag below a configured maximum. I can share the code if there
is interest.
I wrote
On Wed, Jan 5, 2022 at 10:05 PM Dilip Kumar wrote:
> On Thu, Jan 6, 2022 at 11:27 AM SATYANARAYANA NARLAPURAM
> wrote:
>
> > On Wed, Jan 5, 2022 at 9:46 AM Andres Freund wrote:
> >>
> >> Hi,
> >>
> >> On 2021-12-29 11:31:51 -0800, Andres Freund wrote:
> >> > That's pretty much the same -
On Thu, Jan 6, 2022 at 11:27 AM SATYANARAYANA NARLAPURAM
wrote:
> On Wed, Jan 5, 2022 at 9:46 AM Andres Freund wrote:
>>
>> Hi,
>>
>> On 2021-12-29 11:31:51 -0800, Andres Freund wrote:
>> > That's pretty much the same - XLogInsert() can trigger an
>> > XLogWrite()/Flush().
>> >
>> > I think
On Wed, Jan 5, 2022 at 9:46 AM Andres Freund wrote:
> Hi,
>
> On 2021-12-29 11:31:51 -0800, Andres Freund wrote:
> > That's pretty much the same - XLogInsert() can trigger an
> > XLogWrite()/Flush().
> >
> > I think it's a complete no-go to add throttling to these places. It's
> quite
> >
On Wed, Jan 5, 2022 at 11:16 PM Andres Freund wrote:
>
> Hi,
>
> On 2021-12-29 11:31:51 -0800, Andres Freund wrote:
> > That's pretty much the same - XLogInsert() can trigger an
> > XLogWrite()/Flush().
> >
> > I think it's a complete no-go to add throttling to these places. It's quite
> >
Hi,
On 2021-12-29 11:31:51 -0800, Andres Freund wrote:
> That's pretty much the same - XLogInsert() can trigger an
> XLogWrite()/Flush().
>
> I think it's a complete no-go to add throttling to these places. It's quite
> possible that it'd cause new deadlocks, and it's almost guaranteed to have
>
On Wed, Dec 22, 2021 at 4:23 PM SATYANARAYANA NARLAPURAM <
satyanarlapu...@gmail.com> wrote:
> Hi Hackers,
>
> I am considering implementing RPO (recovery point objective) enforcement
> feature for Postgres where the WAL writes on the primary are stalled when
> the WAL distance between the
Hi,
On 2021-12-29 23:06:31 -0800, SATYANARAYANA NARLAPURAM wrote:
> I am afraid there are problems with making the RPO check post releasing the
> locks. By this time the transaction is committed and visible to the other
> backends (ProcArrayEndTransaction is already called) though the intention
>
On Thu, Dec 30, 2021 at 12:20 AM Dilip Kumar wrote:
> On Thu, Dec 30, 2021 at 1:41 PM Bharath Rupireddy <
> bharath.rupireddyforpostg...@gmail.com> wrote:
>
>>
>> >
>> > Yeah, that's true, but even if we are blocking the transactions from
>> committing then also it is possible that a new
On Thu, Dec 30, 2021 at 1:41 PM Bharath Rupireddy <
bharath.rupireddyforpostg...@gmail.com> wrote:
>
> >
> > Yeah, that's true, but even if we are blocking the transactions from
> committing then also it is possible that a new connection can come and
> generate more WAL, yeah but I agree with
On Thu, Dec 30, 2021 at 1:21 PM Dilip Kumar wrote:
>
> On Thu, Dec 30, 2021 at 12:36 PM SATYANARAYANA NARLAPURAM
> wrote:
>>>
>>>
>>> Yeah, I think that would make sense, even though we will be allowing a new
>>> backend to get connected insert WAL, and get committed but after that, it
>>>
On Thu, Dec 30, 2021 at 12:36 PM SATYANARAYANA NARLAPURAM <
satyanarlapu...@gmail.com> wrote:
>
>> Yeah, I think that would make sense, even though we will be allowing a
>> new backend to get connected insert WAL, and get committed but after that,
>> it will be throttled. However, if the number
On Wed, Dec 29, 2021 at 10:38 PM Dilip Kumar wrote:
> On Thu, Dec 30, 2021 at 1:09 AM Andres Freund wrote:
>
>> Hi,
>>
>> On 2021-12-29 11:34:53 -0800, SATYANARAYANA NARLAPURAM wrote:
>> > On Wed, Dec 29, 2021 at 11:31 AM Andres Freund
>> wrote:
>> > Andres, thanks for the comments. Agreed on
On Thu, Dec 30, 2021 at 1:09 AM Andres Freund wrote:
> Hi,
>
> On 2021-12-29 11:34:53 -0800, SATYANARAYANA NARLAPURAM wrote:
> > On Wed, Dec 29, 2021 at 11:31 AM Andres Freund
> wrote:
> > Andres, thanks for the comments. Agreed on this based on the previous
> > discussions on this thread.
Hi,
On 2021-12-29 11:34:53 -0800, SATYANARAYANA NARLAPURAM wrote:
> On Wed, Dec 29, 2021 at 11:31 AM Andres Freund wrote:
> Andres, thanks for the comments. Agreed on this based on the previous
> discussions on this thread. Could you please share your thoughts on adding
> it after
On Wed, Dec 29, 2021 at 11:31 AM Andres Freund wrote:
> Hi,
>
> On 2021-12-27 16:40:28 -0800, SATYANARAYANA NARLAPURAM wrote:
> > > Yet another problem is that if we are in XlogInsert() that means we are
> > > holding the buffer locks on all the pages we have modified, so if we
> add a
> > >
Hi,
On 2021-12-27 16:40:28 -0800, SATYANARAYANA NARLAPURAM wrote:
> > Yet another problem is that if we are in XlogInsert() that means we are
> > holding the buffer locks on all the pages we have modified, so if we add a
> > hook at that level which can make it wait then we would also block any
On Wed, Dec 29, 2021 at 11:16 AM Stephen Frost wrote:
> Greetings,
>
> On Wed, Dec 29, 2021 at 14:04 SATYANARAYANA NARLAPURAM <
> satyanarlapu...@gmail.com> wrote:
>
>> Stephen, thank you!
>>
>> On Wed, Dec 29, 2021 at 5:46 AM Stephen Frost wrote:
>>
>>> Greetings,
>>>
>>> * SATYANARAYANA
Greetings,
On Wed, Dec 29, 2021 at 14:04 SATYANARAYANA NARLAPURAM <
satyanarlapu...@gmail.com> wrote:
> Stephen, thank you!
>
> On Wed, Dec 29, 2021 at 5:46 AM Stephen Frost wrote:
>
>> Greetings,
>>
>> * SATYANARAYANA NARLAPURAM (satyanarlapu...@gmail.com) wrote:
>> > On Sat, Dec 25, 2021 at
Stephen, thank you!
On Wed, Dec 29, 2021 at 5:46 AM Stephen Frost wrote:
> Greetings,
>
> * SATYANARAYANA NARLAPURAM (satyanarlapu...@gmail.com) wrote:
> > On Sat, Dec 25, 2021 at 9:25 PM Dilip Kumar
> wrote:
> > > On Sun, Dec 26, 2021 at 10:36 AM SATYANARAYANA NARLAPURAM <
> > >
Greetings,
* SATYANARAYANA NARLAPURAM (satyanarlapu...@gmail.com) wrote:
> On Sat, Dec 25, 2021 at 9:25 PM Dilip Kumar wrote:
> > On Sun, Dec 26, 2021 at 10:36 AM SATYANARAYANA NARLAPURAM <
> > satyanarlapu...@gmail.com> wrote:
> >>> Actually all the WAL insertions are done under a critical
On Sat, Dec 25, 2021 at 9:25 PM Dilip Kumar wrote:
> On Sun, Dec 26, 2021 at 10:36 AM SATYANARAYANA NARLAPURAM <
> satyanarlapu...@gmail.com> wrote:
>
>>
>>> Actually all the WAL insertions are done under a critical section
>>> (except few exceptions), that means if you see all the references of
On Sun, Dec 26, 2021 at 10:36 AM SATYANARAYANA NARLAPURAM <
satyanarlapu...@gmail.com> wrote:
>
>> Actually all the WAL insertions are done under a critical section (except
>> few exceptions), that means if you see all the references of XLogInsert(),
>> it is always called under the critical
On Sun, Dec 26, 2021 at 1:06 PM SATYANARAYANA NARLAPURAM
wrote:
>
> Got it, understood the concern. But can we document the limitations of the
> hook and let the hook take care of it? I don't expect an error to be thrown
> here since we are not planning to allocate memory or make file system
On Sat, Dec 25, 2021 at 6:01 PM Dilip Kumar wrote:
> On Sun, Dec 26, 2021 at 3:52 AM SATYANARAYANA NARLAPURAM <
> satyanarlapu...@gmail.com> wrote:
>
>>
>>
>> On Fri, Dec 24, 2021 at 3:13 AM Dilip Kumar
>> wrote:
>>
>>> On Fri, Dec 24, 2021 at 3:27 AM SATYANARAYANA NARLAPURAM <
>>>
On Sun, Dec 26, 2021 at 3:52 AM SATYANARAYANA NARLAPURAM <
satyanarlapu...@gmail.com> wrote:
>
>
> On Fri, Dec 24, 2021 at 3:13 AM Dilip Kumar wrote:
>
>> On Fri, Dec 24, 2021 at 3:27 AM SATYANARAYANA NARLAPURAM <
>> satyanarlapu...@gmail.com> wrote:
>>
>>>
>>> XLogInsert in my opinion is
On Fri, Dec 24, 2021 at 3:13 AM Dilip Kumar wrote:
> On Fri, Dec 24, 2021 at 3:27 AM SATYANARAYANA NARLAPURAM <
> satyanarlapu...@gmail.com> wrote:
>
>>
>>>
>> XLogInsert in my opinion is the best place to call it and the hook can be
>> something like this "void xlog_insert_hook(NULL)" as all
On Fri, Dec 24, 2021 at 4:43 PM Dilip Kumar wrote:
>
> On Fri, Dec 24, 2021 at 3:27 AM SATYANARAYANA NARLAPURAM
> wrote:
>>
>> XLogInsert in my opinion is the best place to call it and the hook can be
>> something like this "void xlog_insert_hook(NULL)" as all the throttling
>> logic required
On Fri, Dec 24, 2021 at 3:27 AM SATYANARAYANA NARLAPURAM <
satyanarlapu...@gmail.com> wrote:
>
>>
> XLogInsert in my opinion is the best place to call it and the hook can be
> something like this "void xlog_insert_hook(NULL)" as all the throttling
> logic required is the current flush position
Please find the attached draft patch.
On Thu, Dec 23, 2021 at 2:47 AM Bharath Rupireddy <
bharath.rupireddyforpostg...@gmail.com> wrote:
> On Thu, Dec 23, 2021 at 5:53 AM SATYANARAYANA NARLAPURAM
> wrote:
> >
> > Hi Hackers,
> >
> > I am considering implementing RPO (recovery point objective)
On Thu, Dec 23, 2021 at 5:18 AM Ashutosh Bapat
wrote:
> On Thu, Dec 23, 2021 at 5:53 AM SATYANARAYANA NARLAPURAM
> wrote:
> >
> > Hi Hackers,
> >
> > I am considering implementing RPO (recovery point objective) enforcement
> feature for Postgres where the WAL writes on the primary are stalled
On Thu, Dec 23, 2021 at 5:53 AM SATYANARAYANA NARLAPURAM
wrote:
>
> Hi Hackers,
>
> I am considering implementing RPO (recovery point objective) enforcement
> feature for Postgres where the WAL writes on the primary are stalled when the
> WAL distance between the primary and standby exceeds the
On Thu, Dec 23, 2021 at 5:53 AM SATYANARAYANA NARLAPURAM
wrote:
>
> Hi Hackers,
>
> I am considering implementing RPO (recovery point objective) enforcement
> feature for Postgres where the WAL writes on the primary are stalled when the
> WAL distance between the primary and standby exceeds the
Hi Hackers,
I am considering implementing RPO (recovery point objective) enforcement
feature for Postgres where the WAL writes on the primary are stalled when
the WAL distance between the primary and standby exceeds the configured
(replica_lag_in_bytes) threshold. This feature is useful
36 matches
Mail list logo
