Re: [sqlite] Accessing Database on Network
something like sql server is likely better for the task. sqlite can lag quite a bit when it needs to get file locks over the network to stay atomic. On 7/28/05, djm <[EMAIL PROTECTED]> wrote: > Hello, > > The documentation suggests that its unsafe to use SQLite when the > database file is on a windows network server and various other > machines may want to simultaneously access it. If however none of > these machines change the data in the databse (all accesses are just > queries) is it then completely safe? > > Presumably only querying the database (and not inserting/altering etc) > translates internally in sqlite to opening the database in read_only > mode, which is presumably safe to do in several clients concurrently? > If so from what version of windows/sqlite can I be confident? > > > -- > Best regards, > djm mailto:[EMAIL PROTECTED] > > > -- Cory Nelson http://www.int64.org
Re: [sqlite] Multi-threading.
Just FYI... All you people posting to comp.sys.15yearoldarguments - you know you're also cross-posting to the sqlite mailing list, right? Tim Mrs. Brisby wrote: On Sat, 2005-07-30 at 20:29 +0200, Jan-Eric Duden wrote: Mrs. Brisby wrote: On Sat, 2005-07-30 at 14:30 +0200, Jan-Eric Duden wrote: Win9X doesn't support the API async file operations. WinNT/2K/XP does support it. It supports everything it needs to: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/waitformultipleobjects.asp it doesn't support async file NOTIFICATIONS, (afaik), but those aren't necessary... How should waitformultipleobjects help with aync file io? It doesn't. Read below. For async file io on Win32 you would open a file with CreateFile and pass the FILE_FLAG_OVERLAPPED flag. After that, functions like WriteFile and ReadFile work asynchronously. Unfortunately, on Win9X/ME it is not supported for files on disk. For example, see the docs of WriteFile/ReadFile/WriteFileEx/ReadFileEx: *"Windows 95/98/Me: *For asynchronous write operations, /hFile/ can be a communications resource opened with the FILE_FLAG_OVERLAPPED flag by *CreateFile*, or a socket handle returned by *socket* or *accept*. You cannot perform asynchronous write operations on mailslots, named pipes, or *disk files*. " No you don't. It's not just that Windows 3.11/95/98/ME don't support FILE_FLAG_OVERLAPPED - its that their filesystem isn't reentrant. It cannot do a context switch during that either. You use a large read-size (bigger than the disk-block) and it can context-switch inbetween disk operations. Threads look like a win here! You can also use the disk-block reads. The operating system you're talking about is uniprocessor only. Process control returns to your program just as fast as a thread-switch can occur, and you've saved a context-switch. Use waitformultipleobjects to check your event pump after each readFile or writeFile operation. [[ as a side note: no, it's not really that easy. If the disk is asleep, spun down, or if you run two programs you need to do extra work: * wake up the disk * lock the underlying physical disk Because your code is simpler, these things aren't such a big deal. ]] I will personally advocate splitting processes into a worker-master situation- I just don't recommend pool-threading. That's a lie. I recommend against pool-threading.
Re[2]: [sqlite] Accessing Database on Network
Hello, > Understanding what underlying operations aren't safe, is helpful. Im not sure what this should mean. Can you please elaborate. > If there's never any writers, why bother keeping it on the network? I do have reasons, most of which are based on particular customer requirenments. > Have your program "check the network server" for a newer version of the > file and copy it down if necessary. > This will reduce network traffic because you won't be checking for locks > every query... All good suggestions. Notwithstanding, Id still like very much to know the answer to the question if there's anybody reading who knows it? Thanks ion Advance. -- Best regards, djmmailto:[EMAIL PROTECTED]
Re: [sqlite] Multi-threading.
On Sat, 2005-07-30 at 20:29 +0200, Jan-Eric Duden wrote: > Mrs. Brisby wrote: > > >On Sat, 2005-07-30 at 14:30 +0200, Jan-Eric Duden wrote: > > > > > >>Win9X doesn't support the API async file operations. > >>WinNT/2K/XP does support it. > >> > >> > > > >It supports everything it needs to: > > > >http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/waitformultipleobjects.asp > > > > > >it doesn't support async file NOTIFICATIONS, (afaik), but those aren't > >necessary... > > > > > > > How should waitformultipleobjects help with aync file io? It doesn't. Read below. > For async file io on Win32 you would open a file with CreateFile and > pass the FILE_FLAG_OVERLAPPED flag. > After that, functions like WriteFile and ReadFile work asynchronously. > Unfortunately, on Win9X/ME it is not supported for files on disk. > For example, see the docs of WriteFile/ReadFile/WriteFileEx/ReadFileEx: > *"Windows 95/98/Me: *For asynchronous write operations, /hFile/ can be a > communications resource opened with the FILE_FLAG_OVERLAPPED flag by > *CreateFile*, or a socket handle returned by *socket* or *accept*. You > cannot perform asynchronous write operations on mailslots, named pipes, > or *disk files*. " No you don't. It's not just that Windows 3.11/95/98/ME don't support FILE_FLAG_OVERLAPPED - its that their filesystem isn't reentrant. It cannot do a context switch during that either. You use a large read-size (bigger than the disk-block) and it can context-switch inbetween disk operations. Threads look like a win here! You can also use the disk-block reads. The operating system you're talking about is uniprocessor only. Process control returns to your program just as fast as a thread-switch can occur, and you've saved a context-switch. Use waitformultipleobjects to check your event pump after each readFile or writeFile operation. [[ as a side note: no, it's not really that easy. If the disk is asleep, spun down, or if you run two programs you need to do extra work: * wake up the disk * lock the underlying physical disk Because your code is simpler, these things aren't such a big deal. ]] I will personally advocate splitting processes into a worker-master situation- I just don't recommend pool-threading. That's a lie. I recommend against pool-threading.
Re: [sqlite] Multi-threading.
Mrs. Brisby wrote: On Sat, 2005-07-30 at 14:30 +0200, Jan-Eric Duden wrote: Win9X doesn't support the API async file operations. WinNT/2K/XP does support it. It supports everything it needs to: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/waitformultipleobjects.asp it doesn't support async file NOTIFICATIONS, (afaik), but those aren't necessary... How should waitformultipleobjects help with aync file io? For async file io on Win32 you would open a file with CreateFile and pass the FILE_FLAG_OVERLAPPED flag. After that, functions like WriteFile and ReadFile work asynchronously. Unfortunately, on Win9X/ME it is not supported for files on disk. For example, see the docs of WriteFile/ReadFile/WriteFileEx/ReadFileEx: *"Windows 95/98/Me: *For asynchronous write operations, /hFile/ can be a communications resource opened with the FILE_FLAG_OVERLAPPED flag by *CreateFile*, or a socket handle returned by *socket* or *accept*. You cannot perform asynchronous write operations on mailslots, named pipes, or *disk files*. " Regarding threads: Threads complicate things. People think they "understand threads" long before they actually do. Goto complicates things. People think they "understand goto" long before they actually do. That's a completely normal property of human nature. You should get used to it. This behavior is not limited to the issue of using thread or goto. It applies to everything that needs some skill. Most people just don't know what they can do or can't do until they fail. Even if they fail they think they succeeded. To make things worse, once you taught someone to know better, then someone else will take his/her place. Agreed.
Re: [sqlite] Accessing Database on Network
On Thu, 2005-07-28 at 11:28 +0200, djm wrote: > Hello, > > The documentation suggests that its unsafe to use SQLite when the > database file is on a windows network server and various other > machines may want to simultaneously access it. If however none of > these machines change the data in the databse (all accesses are just > queries) is it then completely safe? > > Presumably only querying the database (and not inserting/altering etc) > translates internally in sqlite to opening the database in read_only > mode, which is presumably safe to do in several clients concurrently? > If so from what version of windows/sqlite can I be confident? Understanding what underlying operations aren't safe, is helpful. If there's never any writers, why bother keeping it on the network? Have your program "check the network server" for a newer version of the file and copy it down if necessary. This will reduce network traffic because you won't be checking for locks every query...
Re: [sqlite] Multi-threading.
On Sat, 2005-07-30 at 14:30 +0200, Jan-Eric Duden wrote: > Win9X doesn't support the API async file operations. > WinNT/2K/XP does support it. It supports everything it needs to: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/waitformultipleobjects.asp it doesn't support async file NOTIFICATIONS, (afaik), but those aren't necessary... > Regarding threads: > > >Threads complicate things. People think they "understand threads" long > >before they actually do. Goto complicates things. People think they > >"understand goto" long before they actually do. > > That's a completely normal property of human nature. You should get used to > it. > This behavior is not limited to the issue of using thread or goto. > It applies to everything that needs some skill. > Most people just don't know what they can do or can't do until they fail. > Even if they fail they think they succeeded. > To make things worse, once you taught someone to know better, then someone > else will take his/her place. Agreed.
Re: [sqlite] Accessing Database on Network
On Jul 30, 2005, at 8:22 AM, djm wrote: Nobody? Hello, The documentation suggests that its unsafe to use SQLite when the database file is on a windows network server and various other machines may want to simultaneously access it. If however none of these machines change the data in the databse (all accesses are just queries) is it then completely safe? Presumably only querying the database (and not inserting/altering etc) translates internally in sqlite to opening the database in read_only mode, which is presumably safe to do in several clients concurrently? If so from what version of windows/sqlite can I be confident? others may weigh in with a more knowledgeable response, however, as far as I can see, if you are only reading from a db, it should be safe no matter what conditions you use it in. -- Puneet Kishor
Re: [sqlite] Accessing Database on Network
Nobody? > Hello, > The documentation suggests that its unsafe to use SQLite when the > database file is on a windows network server and various other > machines may want to simultaneously access it. If however none of > these machines change the data in the databse (all accesses are just > queries) is it then completely safe? > Presumably only querying the database (and not inserting/altering etc) > translates internally in sqlite to opening the database in read_only > mode, which is presumably safe to do in several clients concurrently? > If so from what version of windows/sqlite can I be confident?
