Re: [firebird-support] Firebird begin backup and forced writes off, timestamp update delayed
24.01.2020 09:08, Kjell Rilbe kjell.ri...@marknadsinformation.se [firebird-support] wrote: > I assume this is caused by forced writes off and the fact that Windows > may delay some writes for an arbitrary amount of time. In other words, I > assume that the Firebird engine does in fact not write to the main > databse file after alter database begin backup, but writes that were > made before this may be delayed by Windows for quite a while, meaning > it's not in fact safe to start copying the main database file until > Windows has flushed all writes. > > Now, my question is how to properly handle this to achieve safe and > reliable backups? Your assumption is right, but you don't need to worry. Even if Windows doesn't update file metadata and doesn't flush system buffers to the disk, copy routines will read the data from the system buffers. That's how cache is working. -- WBR, SD. ++ Visit http://www.firebirdsql.org and click the Documentation item on the main (top) menu. Try FAQ and other links from the left-side menu there. Also search the knowledgebases at http://www.ibphoenix.com/resources/documents/ ++ Yahoo Groups Links <*> To visit your group on the web, go to: http://groups.yahoo.com/group/firebird-support/ <*> Your email settings: Individual Email | Traditional <*> To change settings online go to: http://groups.yahoo.com/group/firebird-support/join (Yahoo! ID required) <*> To change settings via email: firebird-support-dig...@yahoogroups.com firebird-support-fullfeatu...@yahoogroups.com <*> To unsubscribe from this group, send an email to: firebird-support-unsubscr...@yahoogroups.com <*> Your use of Yahoo Groups is subject to: https://info.yahoo.com/legal/us/yahoo/utos/terms/
Re: [firebird-support] Firebird begin backup and forced writes off, timestamp update delayed
Den 2020-01-24 kl. 09:08, skrev Kjell Rilbe kjell.ri...@marknadsinformation.se [firebird-support]: > > Hi, > > As posted here: > https://dba.stackexchange.com/questions/258124/firebird-begin-backup-and-forced-writes-off-timestamp-update-delayed > > > Feel free to reply here, at StackExchange, or both. I will make sure the > StackExchange question gets updated to show any helpful info that > appears here. > > We have a Firebird database that's almost 200 Gbyte in sice, and for > performance we have forced writes off. Please don't debate the risks of > forced writes, we are aware of them. This is Firebird 3.0.4 on Windows > Server 2016. > > For backup, we use alter database begin backup and alter database end > backup, and copy the main database file using FastCopy. We can see that > the delta file is created right away when executing alter database begin > backup. But the main database file usually gets an updated timestamp > quite some time later, often within a few minutes, but sometimes it > takes longer. > > I assume this is caused by forced writes off and the fact that Windows > may delay some writes for an arbitrary amount of time. In other words, I > assume that the Firebird engine does in fact not write to the main > databse file after alter database begin backup, but writes that were > made before this may be delayed by Windows for quite a while, meaning > it's not in fact safe to start copying the main database file until > Windows has flushed all writes. > > Now, my question is how to properly handle this to achieve safe and > reliable backups? > > Up to now I've scheduled file copy to 3 hours after alter database begin > backup, and I also included a dummy transaction right after the alter > database begin backup. > > But are there better approaches? > > I came up with the idea to use gfix to switch forced writes on before > executing alter database begin backup (and switch it back off later). I > assume this will cause the locked state to be flushed to disk right > away, but I also assume that writes that were made before switching > forced writes on will still suffer the arbitrary delay from Windows' > write cache. Correct? Or is gfix or the Firebird engine actually able to > force flush all previous writes that are already in the Windows write > cache? > > Another idea is to use Sysinternals Sync util to flush Windows' write > cache for the entire disk. For overall system performace, this would not > be a problem, considering backup is scheduled to a low-traffic time of > day. > > We could use nbackup instead of FastCopy. Would this help? In other > words: would nbackup's reads of the main database file see the new > still-cached versions of the database pages being copied, or would it > see the outdated on-disk versions? > > In fact, I'm not sure if FastCopy actually sees the new still-cached > versions or not, but it fails when it notices that the source file's > timestamp has been changed since it started copying, so it fails anyway. > There's no apparent way to avoid this. > Come to think of it, if it's possible, shouldn't alter database begin backup (as well as nbackup -L) always flush any writes still in Windows' cache? I mean, the purpose of the operation is to make sure the main database file is frozen, but if preceding cached writes are not flushed, this frozen state is actually not guaranteed. Question: Is it possible for alter database begin backup and nbackup -L to force flush any preceding writes that are still in Windows' cache? Regards, Kjell [Non-text portions of this message have been removed]
[firebird-support] Firebird begin backup and forced writes off, timestamp update delayed
Hi, As posted here: https://dba.stackexchange.com/questions/258124/firebird-begin-backup-and-forced-writes-off-timestamp-update-delayed Feel free to reply here, at StackExchange, or both. I will make sure the StackExchange question gets updated to show any helpful info that appears here. We have a Firebird database that's almost 200 Gbyte in sice, and for performance we have forced writes off. Please don't debate the risks of forced writes, we are aware of them. This is Firebird 3.0.4 on Windows Server 2016. For backup, we use alter database begin backup and alter database end backup, and copy the main database file using FastCopy. We can see that the delta file is created right away when executing alter database begin backup. But the main database file usually gets an updated timestamp quite some time later, often within a few minutes, but sometimes it takes longer. I assume this is caused by forced writes off and the fact that Windows may delay some writes for an arbitrary amount of time. In other words, I assume that the Firebird engine does in fact not write to the main databse file after alter database begin backup, but writes that were made before this may be delayed by Windows for quite a while, meaning it's not in fact safe to start copying the main database file until Windows has flushed all writes. Now, my question is how to properly handle this to achieve safe and reliable backups? Up to now I've scheduled file copy to 3 hours after alter database begin backup, and I also included a dummy transaction right after the alter database begin backup. But are there better approaches? I came up with the idea to use gfix to switch forced writes on before executing alter database begin backup (and switch it back off later). I assume this will cause the locked state to be flushed to disk right away, but I also assume that writes that were made before switching forced writes on will still suffer the arbitrary delay from Windows' write cache. Correct? Or is gfix or the Firebird engine actually able to force flush all previous writes that are already in the Windows write cache? Another idea is to use Sysinternals Sync util to flush Windows' write cache for the entire disk. For overall system performace, this would not be a problem, considering backup is scheduled to a low-traffic time of day. We could use nbackup instead of FastCopy. Would this help? In other words: would nbackup's reads of the main database file see the new still-cached versions of the database pages being copied, or would it see the outdated on-disk versions? In fact, I'm not sure if FastCopy actually sees the new still-cached versions or not, but it fails when it notices that the source file's timestamp has been changed since it started copying, so it fails anyway. There's no apparent way to avoid this. Mvh, Kjell [Non-text portions of this message have been removed]
