Ken,

Thanks for you comments.  I have coded and tested a module just like
test_server.c and by disabling the safety checks i have also been able
to code and test an example which uses a single connection, single
transaction, single table with up to 50 threads doing
insert/update/delete with no errors.

Granted it was synchronised, so all execution of sqlite3_ api routines
was done serially, but in my experience thats quite a normal
requirement for multithreading.

Its hard to make something which doesnt require synchronisation,
especially when internal state and files are concerned.  But thats not
a problem, because if the synchronisation is granular enough it will
only ever block as frequently as it would normallly crash.  So in
other words, blocking becomes a race condition, and performance is not
hindered very much at all.

If you would like i can code up a test case which demonstrates this
and send it too you, but i think i should stop posting to this topic.
Some people are getting a bit too defensive and annoyed...

I am grateful for all the time people have given me, and for the
software itself.  At the end of the day i only brought this up because
i wanted to help improve it.

Emerson

On 1/5/07, Ken <[EMAIL PROTECTED]> wrote:
Emerson,

 I agree with you somewhat. Not 100% convinced but, I like you am a little dissapointed how sqlite handles  
"threadsafe" and multiple connections. Even in the "test_server.c" module is not 
"concurrent" As it serializes all processing to a single thread, this is not concurrent processing.

 Now if i could take that server thread and create a pool of workers and hand 
off the working set request to one of those, then this would be concurrent 
since each thread would have the capability to run independently of the other 
and at the same time on an SMP hardware system such as Solaris,Hpux etc... But 
this is specifically disallowed since the connection when using the shared 
cache must be used by the creating thread.

 So far I've not been able to successfully find any threadsafe examples of 
sqlite that actually work in a concurrent manner.  Even the threadtest2.c 
crased (bug fix coming in 3.3.10)  but even it does not start up a transaction 
and run multiple inserts/step commands.

 I would like to see a working example of how sqlite can be concurrently 
accessed via multiple threads performing    Write (insert/update/delete) and by 
other threads performing selects against a single table.

 IMHO, the sqlite3_step function if it can only be executed serially (ie must be wrapped 
in a mutex) should do this mutexing internally and not be exposed and left in the hands 
of the user. If one compiles the code with --enable-threadsafe, shouldn't the API be able 
to handle areas such as these internally and not generate "misuse" errors.



Emerson Clarke <[EMAIL PROTECTED]> wrote: Roger,

Of course you can test threading behaviour, yes its not exactly
repeatable but under most circumstances and with enough test cases you
can catch the problems.

I don't think sqlite is such a large and complicated piece of software
that it would be impossible to reproduce such errors.

Everyone keeps saying its well documented and referring to the fact
that the reasoning has already been done, but i havnt heard anything
other than the usual "thats the way its always been" responses, or the
"its designed that way beacuse its easier" argument.

That does not count as documentation or reasoning.  If anything the
source code is the documentation, but thats not quite the same as an
essay on the thread safety of the sqlite vm is it ?

Anyway, i can see im flogging a dead horse here. This is as futile as
arguing religion with a priest. :P

Theres not much point forking any maintaining the code if on one else
sees any validity in my arguments.

Thanks to everyone anyway,

Emerson

On 1/5/07, Roger Binns  wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Emerson Clarke wrote:
> > I have to admit i am a little dissapointed.  As the primary author of
> > the software i would have thought that you would have a good
> > understanding of what the thread safety characteristics of your own
> > api were.
>
> He does!  It is well documented and tested.  *You* are the one who wants
> to do various things differently.
>
> > Suggesting that suppressing the safety checks will result in random
> > and non reproducable failures seems a little extreme, if not
> > superstitious.
>
> Err, you are using threading.  Explain exactly how if you got a failure
> due to threading you could reproduce it exactly.  You do know that you
> can't run x86 processors in lockstep because at the cycle level they
> have effectively random behaviour due to cache eviction and replacement
> policies.  That one reason for the Itanium if you need two processors to
> have exactly the same behaviour on the same code at the cycle level.
>
> > As i was discussing with Nicholas, api's are not by default thread
> > unsafe.  In fact it is the other way around, most api's should be
> > perfectly thread safe if access to them is synchronised.
>
> This all comes down to semantics.  For trivial libraries, sure you can
> just wrap a mutex around it all.  For non-trivial libraries, best
> practise is for the author to design and implement for whatever thread
> usage they believe is best suited for the library and the platforms it
> operates one.
>
> > Im sorry for being so harsh, and i know im not winning any friends
> > here,
>
> So far noone has agreed with you :-)
>
> > I have tested the changes i made with some 50 threads performing
> > probably thousands of statements a second, and had no errors.
>
> What does that prove?  You cannot prove threading by running stuff and
> saying "see it didn't crash".  The only way to prove threading is by
> reasoning about the design and verifying the implementation matches the
> design. drh already did that years ago.
>
> > If you are willing to work through the problem, i am willing to assist
> > you in any way that i can.  Wether it be writing test cases or making
> > modifications to the source.
>
> You can always maintain your own forked version of SQLite and link to it
> from the wiki.
>
> Roger
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
>
> iD8DBQFFnnDQmOOfHg372QQRAtz1AJwIzJbrlnH0xImPSJYzvutAtMomVACfSltc
> KaSDdgBuwW4ITN1UWms5DMI=
> =EqF9
> -----END PGP SIGNATURE-----
>
> -----------------------------------------------------------------------------
> To unsubscribe, send email to [EMAIL PROTECTED]
> -----------------------------------------------------------------------------
>
>

-----------------------------------------------------------------------------
To unsubscribe, send email to [EMAIL PROTECTED]
-----------------------------------------------------------------------------





-----------------------------------------------------------------------------
To unsubscribe, send email to [EMAIL PROTECTED]
-----------------------------------------------------------------------------

Reply via email to