Re: [HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-10-30 Thread Andres Freund
On Tuesday, October 30, 2012 04:24:21 PM Alvaro Herrera wrote: > Tom Lane escribió: > > Andres Freund writes: > > > On Tuesday, October 30, 2012 03:20:03 PM Alvaro Herrera wrote: > > >> Am I the only one who finds this rather bizarre? Maybe this was okay > > >> when xlog data would only come from

Re: [HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-10-30 Thread Tom Lane
Alvaro Herrera writes: > And yeah, I was thinking in one sum for the header and another one for > the data. I don't think it's worth the space. > If we're using CRC to detect end of WAL, what sense does it > make to have to read the whole record if we can detect the end by just > looking at the

Re: [HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-10-30 Thread Alvaro Herrera
Tom Lane escribió: > Andres Freund writes: > > On Tuesday, October 30, 2012 03:20:03 PM Alvaro Herrera wrote: > >> Am I the only one who finds this rather bizarre? Maybe this was okay > >> when xlog data would only come from WAL files stored in the data > >> directory at recovery, but if we're no

Re: [HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-10-30 Thread Tom Lane
Andres Freund writes: > On Tuesday, October 30, 2012 03:20:03 PM Alvaro Herrera wrote: >> Am I the only one who finds this rather bizarre? Maybe this was okay >> when xlog data would only come from WAL files stored in the data >> directory at recovery, but if we're now receiving these from a remo

Re: [HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-10-30 Thread Andres Freund
On Tuesday, October 30, 2012 03:20:03 PM Alvaro Herrera wrote: > Andres Freund escribió: > > On Monday, October 29, 2012 08:58:53 PM Alvaro Herrera wrote: > > > Heikki Linnakangas escribió: > > > > > > Andres commented elsewhere about reading xlog records, processing them > > > as they came in, an

Re: [HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-10-30 Thread Andres Freund
On Monday, October 29, 2012 08:58:53 PM Alvaro Herrera wrote: > Heikki Linnakangas escribió: > > Hmm. I was thinking that making this work in a non-backend context > > would be too hard, so I didn't give that much thought, but I guess > > there isn't many dependencies to backend functions after all

Re: [HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-09-17 Thread Andres Freund
On Monday, September 17, 2012 04:18:28 PM Heikki Linnakangas wrote: > On 17.09.2012 17:08, Tom Lane wrote: > > Heikki Linnakangas writes: > >> On 17.09.2012 13:01, Andres Freund wrote: > >>> It seems we would need one additional callback for both approaches > >>> like: ->error(severity, format, ..

Re: [HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-09-17 Thread Andres Freund
On Monday, September 17, 2012 04:08:01 PM Tom Lane wrote: > Heikki Linnakangas writes: > > On 17.09.2012 13:01, Andres Freund wrote: > >> It seems we would need one additional callback for both approaches like: > >> ->error(severity, format, ...) > >> For both to avoid having to draw in elog.c. >

Re: [HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-09-17 Thread Heikki Linnakangas
On 17.09.2012 17:08, Tom Lane wrote: Heikki Linnakangas writes: On 17.09.2012 13:01, Andres Freund wrote: It seems we would need one additional callback for both approaches like: ->error(severity, format, ...) For both to avoid having to draw in elog.c. Yeah. Another approach would be to re

Re: [HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-09-17 Thread Tom Lane
Heikki Linnakangas writes: > On 17.09.2012 13:01, Andres Freund wrote: >> It seems we would need one additional callback for both approaches like: >> ->error(severity, format, ...) >> For both to avoid having to draw in elog.c. > Yeah. Another approach would be to return the error string from >

Re: [HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-09-17 Thread Andres Freund
On Monday, September 17, 2012 12:52:32 PM Heikki Linnakangas wrote: > On 17.09.2012 12:07, Andres Freund wrote: > > On Monday, September 17, 2012 10:30:35 AM Heikki Linnakangas wrote: > >> The user of the facility doesn't need to be aware of record boundaries, > >> that's the responsibility of the

Re: [HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-09-17 Thread Andres Freund
On Monday, September 17, 2012 01:50:33 PM Heikki Linnakangas wrote: > On 17.09.2012 14:42, Andres Freund wrote: > > On Monday, September 17, 2012 12:55:47 PM Heikki Linnakangas wrote: > >> On 17.09.2012 13:01, Andres Freund wrote: > >>> On Monday, September 17, 2012 11:07:28 AM Andres Freund wrote:

Re: [HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-09-17 Thread Andres Freund
On Monday, September 17, 2012 12:55:47 PM Heikki Linnakangas wrote: > On 17.09.2012 13:01, Andres Freund wrote: > > On Monday, September 17, 2012 11:07:28 AM Andres Freund wrote: > >> On Monday, September 17, 2012 10:30:35 AM Heikki Linnakangas wrote: > >>> On 17.09.2012 11:12, Andres Freund wrote:

Re: [HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-09-17 Thread Heikki Linnakangas
On 17.09.2012 13:01, Andres Freund wrote: On Monday, September 17, 2012 11:07:28 AM Andres Freund wrote: On Monday, September 17, 2012 10:30:35 AM Heikki Linnakangas wrote: On 17.09.2012 11:12, Andres Freund wrote: On Monday, September 17, 2012 09:40:17 AM Heikki Linnakangas wrote: If you don'

Re: [HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-09-17 Thread Andres Freund
On Monday, September 17, 2012 11:07:28 AM Andres Freund wrote: > On Monday, September 17, 2012 10:30:35 AM Heikki Linnakangas wrote: > > On 17.09.2012 11:12, Andres Freund wrote: > > > On Monday, September 17, 2012 09:40:17 AM Heikki Linnakangas wrote: > > > If you don't want the capability to forw

Re: [HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-09-17 Thread Andres Freund
On Monday, September 17, 2012 10:30:35 AM Heikki Linnakangas wrote: > On 17.09.2012 11:12, Andres Freund wrote: > > On Monday, September 17, 2012 09:40:17 AM Heikki Linnakangas wrote: > >> On 15.09.2012 03:39, Andres Freund wrote: > >> 2. We should focus on reading WAL, I don't see the point of mix

Re: [HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-09-17 Thread Andres Freund
Hi Heikki, On Monday, September 17, 2012 09:40:17 AM Heikki Linnakangas wrote: > On 15.09.2012 03:39, Andres Freund wrote: > > Features: > > - streaming reading/writing > > - filtering > > - reassembly of records > > > > Reusing the ReadRecord infrastructure in situations where the code that > >

Re: [HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-09-17 Thread Heikki Linnakangas
On 15.09.2012 03:39, Andres Freund wrote: Features: - streaming reading/writing - filtering - reassembly of records Reusing the ReadRecord infrastructure in situations where the code that wants to do so is not tightly integrated into xlog.c is rather hard and would require changes to rather int

[HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-09-14 Thread Andres Freund
Features: - streaming reading/writing - filtering - reassembly of records Reusing the ReadRecord infrastructure in situations where the code that wants to do so is not tightly integrated into xlog.c is rather hard and would require changes to rather integral parts of the recovery code which doesn