Re: [HACKERS] Patch: incorrect array offset in backend replication tar header

2012-09-27 Thread Brian Weaver
the pg_basebackup worked. The archive will include all the wal files if I make wal_keep_segments high enough. -- Brian On Thu, Sep 27, 2012 at 6:01 PM, Magnus Hagander mag...@hagander.net wrote: On Tue, Sep 25, 2012 at 5:08 PM, Brian Weaver cmdrcluel...@gmail.com wrote: Unless I misread the code, the tar

Re: [HACKERS] Patch: incorrect array offset in backend replication tar header

2012-09-27 Thread Brian Weaver
OK, here is my attempt at patching and correcting the issue in this thread. I have done my best to test to ensure that hot standby, pg_basebackup, and pg_restore of older files work without issues. I think this might be a larger patch that expected, I took some liberties of trying to clean up a

Re: [HACKERS] Patch: incorrect array offset in backend replication tar header

2012-09-27 Thread Brian Weaver
On Thu, Sep 27, 2012 at 6:43 PM, Tom Lane t...@sss.pgh.pa.us wrote: Brian Weaver cmdrcluel...@gmail.com writes: OK, here is my attempt at patching and correcting the issue in this thread. I have done my best to test to ensure that hot standby, pg_basebackup, and pg_restore of older files work

[HACKERS] EVENT Keyword and CREATE TABLE

2012-09-26 Thread Brian Weaver
I think I just got bitten hard by a commit in mid July... git sha1 3855968. In some of our old tables going back several years we a column named 'event' as in: CREATE TABLE tblaudittrail ( id bigint NOT NULL, siteid integer NOT NULL, entrytype character varying(25), form

Re: [HACKERS] Patch: incorrect array offset in backend replication tar header

2012-09-25 Thread Brian Weaver
: Brian Weaver cmdrcluel...@gmail.com writes: Here are lines 321 through 329 of 'archive_read_support_format_tar.c' from libarchive 321 /* Recognize POSIX formats. */ 322 if ((memcmp(header-magic, ustar\0, 6) == 0) 323 (memcmp(header-version, 00, 2) == 0)) 324

Re: [HACKERS] Patch: incorrect array offset in backend replication tar header

2012-09-25 Thread Brian Weaver
wrote: On 9/25/12 3:38 PM, Brian Weaver wrote: I want to modify pg_basebackup to include the WAL files in the tar output. Doesn't pg_basebackup -x do exactly that? Regards, Marko Tiikkaja -- /* insert witty comment here */ -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] Patch: incorrect array offset in backend replication tar header

2012-09-25 Thread Brian Weaver
on a single task. I'm far too interrupt driven at work. -- Brian On Tue, Sep 25, 2012 at 10:30 AM, Tom Lane t...@sss.pgh.pa.us wrote: Brian Weaver cmdrcluel...@gmail.com writes: If you're willing to wait a bit on me to code and test my extensions to pg_basebackup I will try to address some

[HACKERS] Patch: incorrect array offset in backend replication tar header

2012-09-24 Thread Brian Weaver
While researching the way streaming replication works I was examining the construction of the tar file header. By comparing documentation on the tar header format from various sources I certain the following patch should be applied to so the group identifier is put into thee header properly.

Re: [HACKERS] Patch: incorrect array offset in backend replication tar header

2012-09-24 Thread Brian Weaver
Actually I found one other issue while continuing my investigation. The insertion of the 'ustar' and version '00' has the '00' version at the wrong offset. The patch is attached. -- Brian On Mon, Sep 24, 2012 at 7:51 PM, Brian Weaver cmdrcluel...@gmail.com wrote: While researching the way

Re: [HACKERS] Patch: incorrect array offset in backend replication tar header

2012-09-24 Thread Brian Weaver
Um I apologize for the third e-mail on the topic. It seems that my C coding is a bit rusty from years of neglect. No sooner had I hit the send button then I realized that trying to embed a null character in a string might not work, especially when it's followed by two consecutive zeros. Here

Re: [HACKERS] Patch: incorrect array offset in backend replication tar header

2012-09-24 Thread Brian Weaver
calling it a night. I need to send a note to the libarchive folks too because I *think* I found a potential buffer overrun in one of their octal conversion routines. -- Brian On Mon, Sep 24, 2012 at 10:07 PM, Tom Lane t...@sss.pgh.pa.us wrote: Brian Weaver cmdrcluel...@gmail.com writes: While

[HACKERS] Problem with multi-job pg_restore

2012-05-01 Thread Brian Weaver
I think I've discovered an issue with multi-job pg_restore on a 700 GB data file created with pg_dump. Before anyone points out that the preferred procedure is to use the newest pg_dump to backup a database before doing pg_restore let just say, Yes I'm aware of that advice and unfortunately it

Re: [HACKERS] Problem with multi-job pg_restore

2012-05-01 Thread Brian Weaver
...@sss.pgh.pa.us wrote: Brian Weaver cmdrcluel...@gmail.com writes: I think I've discovered an issue with multi-job pg_restore on a 700 GB data file created with pg_dump. Just to clarify, you mean parallel restore, right?  Are you using any options beyond -j, that is any sort of selective

Re: [HACKERS] Problem with multi-job pg_restore

2012-05-01 Thread Brian Weaver
be appreciated. On Tue, May 1, 2012 at 12:59 PM, Tom Lane t...@sss.pgh.pa.us wrote: Brian Weaver cmdrcluel...@gmail.com writes: I'm confused.  A copy-to-stdout ought to be something that pg_dump would do, not pg_restore.  Are you sure this is related at all?                        regards, tom lane

Re: [HACKERS] Problem with multi-job pg_restore

2012-05-01 Thread Brian Weaver
...@sss.pgh.pa.us wrote: Brian Weaver cmdrcluel...@gmail.com writes: Doh! I missed a script that was run by cron that does a nightly backup. That's the likely offender for the 'copy-to-stdout' I've removed it from the nightly run. I'll see if have any better luck with this run. Still not sure