[ADMIN] ERROR: could not open file "base/125542/12631" Corruption?

2013-10-04 Thread Mike Broers
Hello, our postgresql 9.2.4 qa database (thankfully its just qa) seems to be hosed. Starting at around 3:39am last night I started seeing errors about missing files and now I cannot run a pgdump or a vacuum without it complaining about files that it cannot find with errors like this: ERROR: could

Re: [ADMIN] ERROR: could not open file "base/125542/12631" Corruption?

2013-10-04 Thread Alejandro Brust
El 04/10/2013 11:40, Mike Broers escribió: > Hello, our postgresql 9.2.4 qa database (thankfully its just qa) seems > to be hosed. > > Starting at around 3:39am last night I started seeing errors about > missing files and now I cannot run a pgdump or a vacuum without it > complaining about files

Re: [ADMIN] ERROR: could not open file "base/125542/12631" Corruption?

2013-10-04 Thread Mike Broers
Strange, this is happening in a totally different environment now too. The only thing these two environments share is a SAN, but I wouldnt think something going on at the SAN level would make files disappear. Any suggestions are greatly appreciated. On Fri, Oct 4, 2013 at 9:40 AM, Mike Broers

Re: [ADMIN] ERROR: could not open file "base/125542/12631" Corruption?

2013-10-04 Thread Alejandro Brust
El 04/10/2013 14:10, Mike Broers escribió: > Strange, this is happening in a totally different environment now too. > The only thing these two environments share is a SAN, but I wouldnt > think something going on at the SAN level would make files disappear. > Any suggestions are greatly apprecia

Re: [ADMIN] ERROR: could not open file "base/125542/12631" Corruption?

2013-10-04 Thread Alvaro Herrera
Alejandro Brust escribió: > U could try some like "SET/ zero_damaged_pages = on"and perform a > vacuum-db and may be pg_dump I don't think this is a good idea. It might cause data loss. In any case it's unlikely to fix the reported problem. -- Álvaro Herrerahttp://www.2nd

Re: [ADMIN] ERROR: could not open file "base/125542/12631" Corruption?

2013-10-04 Thread Mike Broers
Update - someone unleashed a 'cleanup script' yesterday via puppet to multiple hosts and greedily deleted files that had not been modified in 15 days. This is the most likely culprit so mystery basically solved. Thankfully this is in QA, whew! It would be interesting to still know if there are w