[GENERAL] File size consideration of file_fdw in PostgreSQL

2013-06-22 Thread Xiaobo Gu
Hi, We have very large files in size will be created as external tables in PostgreSQL via file_fdw, we have the following considerations: 1. Can file_fdw handle files with size large than the size of RAM in the server. 2. Will file_fdw scan the full file every time when the external table is

Re: [GENERAL] File System backup

2013-06-22 Thread Kevin Grittner
Alban Hertroys haram...@gmail.com wrote: itishree sukla itishree.su...@gmail.com wrote: Can any one give me more suggestion, about this problem. Every time my os got restart, postmaster.pid is missing. The pid file should only be present when postgres is running.  A clean OS shutdown should

[GENERAL] WAL archiving not starting at the beginning

2013-06-22 Thread Michael Angeletti
Hi pg devs, fellow pg users I'm Michael, and this is my first post here. I asked this question last night: http://dba.stackexchange.com/questions/45077 which details the problem (or not?) I'm having. The gist of it is that Postgres is not archiving the first WAL segments for my cluster.

[GENERAL] Standby stopped working after PANIC: WAL contains references to invalid pages

2013-06-22 Thread Dan Kogan
Hello, Today our standby instance stopped working with this error in the log: 2013-06-22 16:27:32 UTC [8367]: [247-1] [] WARNING: page 158130 of relation pg_tblspc/16447/PG_9.2_201204301/16448/39154429 is uninitialized 2013-06-22 16:27:32 UTC [8367]: [248-1] [] CONTEXT: xlog redo vacuum: rel

Re: [GENERAL] Standby stopped working after PANIC: WAL contains references to invalid pages

2013-06-22 Thread Lonni J Friedman
Looks like some kind of data corruption. Question is whether it came from the master, or was created by the standby. If you re-seed the standby with a full (base) backup, does the problem go away? On Sat, Jun 22, 2013 at 12:43 PM, Dan Kogan d...@iqtell.com wrote: Hello, Today our standby

Re: [GENERAL] Standby stopped working after PANIC: WAL contains references to invalid pages

2013-06-22 Thread Dan Kogan
I am in the process of doing that now. I'll reply again with results once that is done. -Original Message- From: Lonni J Friedman [mailto:netll...@gmail.com] Sent: Saturday, June 22, 2013 4:11 PM To: Dan Kogan Cc: pgsql-general@postgresql.org Subject: Re: [GENERAL] Standby stopped

Re: [GENERAL] Standby stopped working after PANIC: WAL contains references to invalid pages

2013-06-22 Thread Dan Kogan
Re-seeding the standby with a full base backup does seem to make the error go away. The standby started, caught up and has been working for about 2 hours. The file in the error message was an index. We rebuilt it just in case. Is there any way to debug the issue at this point? -Original

Re: [GENERAL] Standby stopped working after PANIC: WAL contains references to invalid pages

2013-06-22 Thread Lonni J Friedman
Assuming that you still have $PGDATA from the broken instance (such that you can reproduce the crash again), there might be a way to debug it further. I'd guess that something like bad RAM or storage could cause an index to get corrupted in this fashion, but the fact that you're using AWS makes

Re: [GENERAL] File System backup

2013-06-22 Thread itishree sukla
yes, when i tried to start, postgresql service using init.d its gave me the error removed stale pid, postgresql failed to start. Regards, Itishree On Sat, Jun 22, 2013 at 8:05 PM, Kevin Grittner kgri...@ymail.com wrote: Alban Hertroys haram...@gmail.com wrote: itishree sukla