Re: [sqlite] 3.5.x and pthreads design
D. Richard Hipp wrote: For threads within the same process, fcntl is broken by design in POSIX. (You can clearly tell which parts of Unix were invented by Thompson and Richie and which parts were added later by clueless committees. Posix advisory locks belong in the latter category.) SQLite contains a work-around to this problem based on pthreads locks. It should be able to open multiple connections to the same database within the same process and use them independently and locking should work correctly. You should not have to change anything. It should just work. Thanks Richard, exactly what I wanted to know. So for threads only pth locks are used - for me, great news :) So I can safely remove my own lock and just let SQLite do it internally. Do I need to initialize the sqlite internal pthread locks at startup ? Or is it automatically initialized when the first sqlite3_* function is called? Last question, should I use for each thread its own connection now or use one global and share it along all threads? About what I've found on the list so far, it sound that one global connection will serialize all threads. Is the behavior the same if each thread has its own connection structure? Thank you for your quick and kindly respond! Xat - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] 3.5.x and pthreads design
On Nov 25, 2007, at 5:34 PM, Teufel wrote:
Hi everyone,
I am using sqlite 3.3 awhile now for some statistic updates in
multithreaded enviroment. Now I would like to move on to 3.5.2 use
it more for other tasks too.
Since the sqlite db is placed here on a fs, which has a broken
fcntl, I disabled it by putting "#define fcntl (A,B,C) 0". As I
only use it within the same process with couple of threads, I am
doing currently the locking with pthread mutexes like:
each thread {
/* worker stuff */
...
if new stats did arrive:
compete on mutex
-> when lock is acquired, do
open database
read something (eg sqlite_exec "select ")
if does not exist, do write (sqlite_exec
"insert into table...")
close database->
release lock
}
Of course, this is very simple and serializing everything including
open/close of the db image. My question is now, what could I
improve with 3.5.x now as it has buildin thread safety for the same
database (as mentioned in 34to35) but I did not find how this
thread safety is made (using mutexes, rwlock? still fcntl?)
I cannot rely on fcntl, so for thread-safety, it's all up to
pthread mutexes/rwlock
For threads within the same process, fcntl is broken by design in POSIX.
(You can clearly tell which parts of Unix were invented by Thompson
and Richie
and which parts were added later by clueless committees. Posix advisory
locks belong in the latter category.) SQLite contains a work-around to
this problem based on pthreads locks. It should be able to open
multiple connections to the same database within the same process
and use them independently and locking should work correctly. You
should not have to change anything. It should just work.
D. Richard Hipp
[EMAIL PROTECTED]
-
To unsubscribe, send email to [EMAIL PROTECTED]
-
[sqlite] 3.5.x and pthreads design
Hi everyone,
I am using sqlite 3.3 awhile now for some statistic updates in
multithreaded enviroment. Now I would like to move on to 3.5.2 use it
more for other tasks too.
Since the sqlite db is placed here on a fs, which has a broken fcntl, I
disabled it by putting "#define fcntl (A,B,C) 0". As I only use it
within the same process with couple of threads, I am doing currently the
locking with pthread mutexes like:
each thread {
/* worker stuff */
...
if new stats did arrive:
compete on mutex
-> when lock is acquired, do
open database
read something (eg sqlite_exec "select ")
if does not exist, do write (sqlite_exec
"insert into table...")
close database
-> release lock
}
Of course, this is very simple and serializing everything including
open/close of the db image. My question is now, what could I improve
with 3.5.x now as it has buildin thread safety for the same database (as
mentioned in 34to35) but I did not find how this thread safety is made
(using mutexes, rwlock? still fcntl?)
I cannot rely on fcntl, so for thread-safety, it's all up to pthread
mutexes/rwlock
Following idea's crossed my mind:
a)
only open the database once in each thread and keep the connection open,
but lock write operations with mutex (rwlock). This would give all
threads concurrency for reading and only serialize writing.
For this scenario, I am unsure about the fact if one thread did a write,
how all other threads will notice the change in the database. Is this
now handled internally by sqlite 3.5? Can I keep open multiply
connection to the same database and operate on it. Maybe the VFS will
take in effect now. So thats why I am asking to be sure.
b)
Instead each thread has it's own connection handle create one global and
share it along with all threads (and protect it by mutex?). This would
be an improvement if a) is not possible. At leased the database would
not be opened/closed all the time but serialized read/write commands.
Most of the time, only "select" is happening on the same table. So
concurrent access to the table would be perfect.
To use the new vfs or thread-safety over the same database, do I need to
initialize anything?
Thanks in advance,
Xat
-
To unsubscribe, send email to [EMAIL PROTECTED]
-
