> On Fri, Jan 29, 2010 at 12:03 AM, Mason Hale <ma...@onespot.com> wrote: > > Of course the best solution is to avoid this issue entirely. Something as > > easy to miss as file permissions should not cause data corruption, > > especially in the process meant to fail over from a crashing primary > > database. > > I believe that such a file permission problem does nothing but > shut down the standby by a FATAL error, and wouldn't cause data > corruption. So if you remove the trigger file with a wrong > permission after the shutdown, you can restart a recovery well > by just starting the standby postgres. > > Perhaps my wording of "data corruption" was too harsh?
While I did not remove the trigger file, I did rename recovery.conf to recovery.conf.old. That file contained the recovery_command configuration that identified the trigger file. So that rename should have eliminated the problem. But it didn't. Even after making this change and taking the trigger file out of the equation my database failed to come online. Maybe that wasn't data corruption. Maybe the issue was repairable. I just know that with my 3+ years of experience working with Postgres and the help of the #postgresql IRC channel, I was not able to revive the database at a time when I desperately needed it to work. The failover process failed for me at the worst possible time. I will surely be careful about trigger file permissions in the future. I just shared my experience so that future DBA's who might make the same mistake in a similar situation don't have to deal with the same unexpected results. - Mason