On Mon, Feb 1, 2016 at 11:20 AM, Rowan Worth <rowanw at dugeo.com> wrote:
> > > I've seen situations > > where files have been deleted in Linux (Not allowed in Windows) and data > > still gets dumped into the now deleted file. > > > ? > This is not an error, this is standard POSIX file semantics - if you have a > file descriptor referring to a file which gets deleted from disk, the file > descriptor is guaranteed to remain valid until it is closed. NFS even goes > to some lengths to make this work. > > I didn't claim that this was an error, but a possibility to introduce confusion and data corruption. For instance, if the first DB file gets deleted, but the controlling program still owns it, the file gets deleted, the OS still allows the first instance to do what it needs to do with that file. The only 'things' that know that file exists is the OS and the single instance of that application. Now say a second instance of the application starts up, recreates the original file (New INODE) and starts creating journal/wal files, all of a sudden you have two programs that have very real and very valid temp files that could potentially be accessed by both processes and potentially get your data into a really nasty state. Is the application going to pass or fail ACID because of external pressures? Yanking the power cable is considered an external pressure. This has nothing to do with the thread, but just an illustration that your software is at the mercy of the OS. And if the OS isn't telling your program the truth then it is the OS's fault, not the application or SQLite. > Is it the programs fault that > > the file is physically deleted but the program isn't aware? Is it > supposed > > to check every single time it writes a byte to see if the file exists, if > > the partition is healthy, and the root file system passes a FSCK at every > > commit? > > > > :) > > > > > SQLite doesn't know or care what sector the bytes > > live at on the drive physically. It doesn't know about fragmentation, it > > doesn't know what size the clusters are, or anything that is important > for > > file storage. > > > But SQLite *does* know how to interface with the file system. It's all very > well to say the OS/hardware sometimes lies, but in this specific instance > SQLite hasn't even asked the question. > > That is all it has. An interface. Good word. The information exchanged is cut down to the basics. It doesn't know what the OS REALLY did with that fsync, or if those magnetic fields are aligned properly on the platter, or stored in those transistor cells properly. I'm not sure that I understand where you're going with "hasn't even asked the question"? Are you referring to pre-hard power off or post? What I've understood this thread as being "My application received a SQLITE_OK for a commit, but, after a hard power loss by yanking the power cable, the reboot comes back and my commit isn't there." Its because the OS didn't write the data when the software asked it to. My point is how far can the software go to make sure the OS is doing its job? Which is why I wrote that paragraph above where you smilied. ;) > As I indicated in the last paragraph of my mail, I'm not in favour of > fsync-directory-on-commit in the general case. But that's because I worry > about the performance impact, not because SQLite is doing its best and at > the mercy of the OS. I can't remember who in this thread suggested it but > fsync-directory-on-close seemed like a great compromise. > > -Rowan > *ONLY* if configurable, and *ONLY* if not a default. Most of my apps are setup at highest risk because the data they contain doesn't matter and I want performance vs safety. There are applications I have written that are data critical and I've got full sync on, my desk is illuminated by the HDD light while I wait for that INSERT finishes.
