Re: [HACKERS] walprotocol.h vs frontends

2011-08-16 Thread Magnus Hagander
On Tue, Aug 16, 2011 at 00:05, Simon Riggs si...@2ndquadrant.com wrote: On Mon, Aug 15, 2011 at 10:32 PM, Magnus Hagander mag...@hagander.net wrote: At present the WALSender only sends from one file at a time, so sending a message when we open a new file would be straightforward. Are you

Re: [HACKERS] walprotocol.h vs frontends

2011-08-16 Thread Peter Eisentraut
On mån, 2011-08-15 at 18:39 +0100, Peter Geoghegan wrote: If you want to upgrade a system running 8.3 (that uses float based timestamps) in using pg_upgrade you must compile 9.0 (or 8.4 or 9.1) with --disable-integer-datetimes. If at some point in the future you then want to upgrade to

Re: [HACKERS] walprotocol.h vs frontends

2011-08-16 Thread Simon Riggs
On Tue, Aug 16, 2011 at 9:35 AM, Magnus Hagander mag...@hagander.net wrote: So right now what we do is allow a single packet to span multiple files, but since MAX_SEND_SIZE is 128KB it will always be smaller than a single file, so we can only ever span two files at most. Unless someone has

[HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Magnus Hagander
I'm trying to make my streaming log receiver work properly with 9.1, and have come across a couple of things. The first one that's causing trouble is that the definition of the protocol is currently in walprotocol.h, which is not include:able in a frontend application. AFAICT, this is because it

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Peter Geoghegan
On 15 August 2011 12:22, Magnus Hagander mag...@hagander.net wrote: The basic reason for this is that we're putting TimestampTz fields in the protocol. This also means that the protocol actually changes definition depending on if the server is compiled with integer or float timestamps.

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: I'm trying to make my streaming log receiver work properly with 9.1, and have come across a couple of things. The first one that's causing trouble is that the definition of the protocol is currently in walprotocol.h, which is not include:able in a

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Magnus Hagander
On Mon, Aug 15, 2011 at 16:20, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: I'm trying to make my streaming log receiver work properly with 9.1, and have come across a couple of things. The first one that's causing trouble is that the definition of the

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: On Mon, Aug 15, 2011 at 16:20, Tom Lane t...@sss.pgh.pa.us wrote: However, for a narrow fix, I could see moving the data type definition to someplace with fewer dependencies.  Perhaps split it into a separate file timestamp_type.h, or something like

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Magnus Hagander
On Mon, Aug 15, 2011 at 16:53, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: On Mon, Aug 15, 2011 at 16:20, Tom Lane t...@sss.pgh.pa.us wrote: However, for a narrow fix, I could see moving the data type definition to someplace with fewer dependencies.  Perhaps

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Steve Singer
On 11-08-15 10:00 AM, Peter Geoghegan wrote: Without commenting on what should be done in your specific case, I wonder whether it's time to fully retire the deprecated double representation of timestamps. Is anyone actually expected to rely on their availability when 9.2 is released? This also

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Heikki Linnakangas
On 15.08.2011 18:54, Magnus Hagander wrote: On Mon, Aug 15, 2011 at 16:53, Tom Lanet...@sss.pgh.pa.us wrote: Magnus Hagandermag...@hagander.net writes: I also ran into a similar problem with some WAL macro definitions that are in xlog_internal.h. I've moved them to xlogdefs.h in the attached

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Perhaps we should change the protocol so that it explicitly says which file the streamed piece of WAL belongs to. That way a client could write it to the correct file without knowing about all those macros. Yeah, maybe.

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Magnus Hagander
On Mon, Aug 15, 2011 at 18:12, Tom Lane t...@sss.pgh.pa.us wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Perhaps we should change the protocol so that it explicitly says which file the streamed piece of WAL belongs to. That way a client could write it to the correct

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Peter Geoghegan
On 15 August 2011 16:56, Steve Singer ssin...@ca.afilias.info wrote: This would mean that anyone using the floating point timestamps today won't be able to use pg_upgrade to upgrade to whichever version we remove them from.  8.3 had float based timestamps as the default and I suspect many

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: On Mon, Aug 15, 2011 at 16:53, Tom Lane t...@sss.pgh.pa.us wrote: Don't think you should expose fsec_t, nor most of those macros.  The foo_per_bar values are just namespace clutter. Hmm, ok. I just went for what seemed like a reasonable subset. I do

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Magnus Hagander
On Mon, Aug 15, 2011 at 18:40, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: On Mon, Aug 15, 2011 at 16:53, Tom Lane t...@sss.pgh.pa.us wrote: Don't think you should expose fsec_t, nor most of those macros.  The foo_per_bar values are just namespace clutter.

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Steve Singer
On 11-08-15 12:33 PM, Peter Geoghegan wrote: On 15 August 2011 16:56, Steve Singerssin...@ca.afilias.info wrote: This would mean that anyone using the floating point timestamps today won't be able to use pg_upgrade to upgrade to whichever version we remove them from. 8.3 had float based

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Tom Lane
Steve Singer ssin...@ca.afilias.info writes: On 11-08-15 12:33 PM, Peter Geoghegan wrote: On 15 August 2011 16:56, Steve Singerssin...@ca.afilias.info wrote: This would mean that anyone using the floating point timestamps today won't be able to use pg_upgrade to upgrade to whichever version

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Peter Geoghegan
On 15 August 2011 18:09, Steve Singer ssin...@ca.afilias.info wrote: Really? I find that slightly surprising, considering that a quick look at master's timestamp.c suggests that the choice to use the in64 representation over the double representation is entirely a case of compile time

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Simon Riggs
On Mon, Aug 15, 2011 at 5:15 PM, Magnus Hagander mag...@hagander.net wrote: On Mon, Aug 15, 2011 at 18:12, Tom Lane t...@sss.pgh.pa.us wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Perhaps we should change the protocol so that it explicitly says which file the streamed

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Magnus Hagander
On Mon, Aug 15, 2011 at 21:12, Simon Riggs si...@2ndquadrant.com wrote: On Mon, Aug 15, 2011 at 5:15 PM, Magnus Hagander mag...@hagander.net wrote: On Mon, Aug 15, 2011 at 18:12, Tom Lane t...@sss.pgh.pa.us wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Perhaps we should

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Dimitri Fontaine
Magnus Hagander mag...@hagander.net writes: Why not have a specific protocol message to indicate a change of filename? I don't mean a WAL message, I mean a streaming protocol message. That we could have, and it would work as long as it's always sent as the first packet in any stream. If we

Re: [HACKERS] walprotocol.h vs frontends

2011-08-15 Thread Simon Riggs
On Mon, Aug 15, 2011 at 10:32 PM, Magnus Hagander mag...@hagander.net wrote: At present the WALSender only sends from one file at a time, so sending a message when we open a new file would be straightforward. Are you sure? We can receive a single message spanning multiple files... You're