subject:"Re\: \[sqlite\] Unlock Notify problems"

Re: [sqlite] Unlock Notify problems

2013-07-10 Thread Owen Haynes

Thanks for the help,
I have found the problem, it lies very deep in some query code which gives
up reading new rows and never returns the read lock, was not easy to spot.

Owen


On 10 July 2013 12:42, Simon Slavin  wrote:

>
> On 10 Jul 2013, at 12:28pm, Owen Haynes  wrote:
>
> > I only call sqlite3_finalize() when the thread has finished with the
> > connection, which is when the thread is deleted. sqlite3_reset is used
> on B
> > after it it been notified, and when a query is a success. It looks like B
> > returns a code 100 in the step function sometimes, should I do
> > something different with this code?
>
> http://www.sqlite.org/c3ref/c_abort.html
>
> "#define SQLITE_ROW 100  /* sqlite3_step() has another row ready
> */"
>
> In other words, your query has another row of data ready for you to ask it
> about.
>
> > When it is returned with this code it seems to not go into
> > sqlite3ConnectionUnlocked.
>
> Well naturally.  Your query has not finished returning results.  The file
> has to remain locked until /all/ results have been returned.  For all you
> know there are another hundred rows of data to be read before the query has
> finished executing.
>
> The expected results from sqlite_step() are as follows:
>
> SQLITE_ROW — a row of data has been assembled for you to ask about
>
> SQLITE_DONE — no errors have occurred and there are no more results from
> that query.  Now is a good time to do a _reset() or a _finalize()
>
> anything else — some sort of error has occurred and must be handled
> according to the result code
>
> Simon.
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Unlock Notify problems

2013-07-10 Thread Simon Slavin

On 10 Jul 2013, at 12:28pm, Owen Haynes  wrote:

> I only call sqlite3_finalize() when the thread has finished with the
> connection, which is when the thread is deleted. sqlite3_reset is used on B
> after it it been notified, and when a query is a success. It looks like B
> returns a code 100 in the step function sometimes, should I do
> something different with this code?

http://www.sqlite.org/c3ref/c_abort.html

"#define SQLITE_ROW 100  /* sqlite3_step() has another row ready */"

In other words, your query has another row of data ready for you to ask it 
about.

> When it is returned with this code it seems to not go into
> sqlite3ConnectionUnlocked.

Well naturally.  Your query has not finished returning results.  The file has 
to remain locked until /all/ results have been returned.  For all you know 
there are another hundred rows of data to be read before the query has finished 
executing.

The expected results from sqlite_step() are as follows:

SQLITE_ROW — a row of data has been assembled for you to ask about

SQLITE_DONE — no errors have occurred and there are no more results from that 
query.  Now is a good time to do a _reset() or a _finalize()

anything else — some sort of error has occurred and must be handled according 
to the result code

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Unlock Notify problems

2013-07-10 Thread Owen Haynes

I only call sqlite3_finalize() when the thread has finished with the
connection, which is when the thread is deleted. sqlite3_reset is used on B
after it it been notified, and when a query is a success. It looks like B
returns a code 100 in the step function sometimes, should I do
something different with this code?
When it is returned with this code it seems to not go into
sqlite3ConnectionUnlocked.

Owen



On 9 July 2013 17:28, Dan Kennedy  wrote:

> On 07/09/2013 11:10 PM, Owen Haynes wrote:
>
>> Here is some information I have gathered.
>>
>> We have a writer thread connection called A and a reader called B, another
>> reader called C.
>>
>>
>> A is writing
>> B is waiting on A with pBlockingConnection = A, and pUnlockConnection A
>> A is Now done,
>> A is waiting on B with pBlockingConnection = B and pUnlockConnection B
>> B is now done but never goes to check to see if any connections are
>> waiting
>> to be unblocked, sqlite3ConnectionUnlocked never gets called. I have a
>> print out every time it gets called with the connection which calls it.
>> C then wants to read and has a pBlockingConnection = B and
>> pUnlockConnection =
>> B.
>>
>> And all the connections are then locked up.
>>
>> This information was gathered from pUnlockConnection and
>> pBlockingConnection
>> and by examining the blocked list. All writes are done using "BEGIN
>> IMMEDIATE TRANSACTION"
>>
>
> Never called...
>
> What is the value of B->autoCommit?
>
> Also, how do we know that B is actually "done"?
>
> If it has no explicitly opened transaction (i.e. if B->autoCommit==1),
> then the sqlite3ConnectionUnlocked() function should be called every
> time B halts an SQL statement (which happens either in the
> sqlite3_reset() or sqlite3_finalize() call, or sometimes from within
> sqlite3_step() if it returns SQLITE_DONE or an error).
>
> Is B calling sqlite3_reset() or sqlite3_finalize() on its statement
> handles when it is finished with them?
>
> Dan.
>
>
>
>
>
>
>> Owen
>>
>>
>> On 9 July 2013 12:51, Dan Kennedy  wrote:
>>
>>  On 07/09/2013 06:41 PM, Owen Haynes wrote:
>>>
>>>  I have been using a debugger to look at pBlockedConnection  and
 pUnlockConnection
 also added some extra print statements to the code, sqlite is also
 compiled
 with debug enabled, and no asserts seem to get triggered.


>>> How do you know A has finished? Does "PRAGMA lock_status"
>>> indicate that it is holding no locks at all?
>>>
>>> So in "deadlock 1", when you have:
>>>
>>>
>>> Connection A is has finished
>>> Connection B is waiting on Connection A
>>> Connection C is waiting on Connection B
>>>
>>> Then the following are all true?
>>>
>>>B->pUnlockConnection = A;
>>>B->pBlockingConnection = A;
>>>C->pUnlockConnection = B;
>>>C->pBlockingConnection = B;
>>>
>>> Are both B and C in the linked list that starts at global
>>> variable sqlite3BlockedList (and connected by pNextBlocked)?
>>>
>>> Dan.
>>>
>>>
>>>
>>>
>>>
>>>
>>>  Owen




 On 9 July 2013 11:20, Dan Kennedy  wrote:

   On 07/09/2013 04:08 PM, Dan Kennedy wrote:

>   On 06/20/2013 03:20 PM, Owen Haynes wrote:
>
>>   Hello,
>>
>>> I am currently having some problems with the unlock notify and
>>> getting
>>> in a
>>> state of deadlock.
>>>
>>> I am using code based on http://www.sqlite.org/unlock_***
>>> ***notify.html <
>>> http://www.**sqlite.org/unlock_**notify.**html
>>> >
>>> 
>>> 
>>> >
>>>
 **,

>>> with
>>> the latest sqlite.
>>>
>>> The setup is as follows:
>>> - WAL is on
>>> - Multi Threaded is on
>>> - Temp store is memory
>>> - Database is stored in /dev/shm
>>> - Synchronous is set as normal
>>> - Multi-threading is on
>>> - connection is set as SQLITE_OPEN_READWRITE |
>>> SQLITE_OPEN_CREATE |
>>> SQLITE_OPEN_NOMUTEX | SQLITE_OPEN_SHAREDCACHE
>>>
>>> Unlock notify does work I can see my log messages printed when the
>>> callback
>>> is called but sometimes I can end up in the following dead lock
>>> situation.
>>>
>>> Each connection is in its own thread
>>>
>>> *Deadlock 1*
>>> Connection A is using database
>>> Connection B is waiting on Connection A
>>> Connection C is waiting on Connection B
>>>
>>> Connection A is has finished
>>> Connection B is waiting on Connection A
>>> Connection C is waiting on Connection B
>>>
>>>   How do you know this is what is happening? Are you looking
>>>
>> at the sqlite3.pBlockedConnection and sqlite3.pUnlockConnection
> pointers in a debugger? Or some other method?

Re: [sqlite] Unlock Notify problems

2013-07-09 Thread Dan Kennedy


On 07/09/2013 11:10 PM, Owen Haynes wrote:

Here is some information I have gathered.

We have a writer thread connection called A and a reader called B, another
reader called C.


A is writing
B is waiting on A with pBlockingConnection = A, and pUnlockConnection A
A is Now done,
A is waiting on B with pBlockingConnection = B and pUnlockConnection B
B is now done but never goes to check to see if any connections are waiting
to be unblocked, sqlite3ConnectionUnlocked never gets called. I have a
print out every time it gets called with the connection which calls it.
C then wants to read and has a pBlockingConnection = B and pUnlockConnection =
B.

And all the connections are then locked up.

This information was gathered from pUnlockConnection and pBlockingConnection
and by examining the blocked list. All writes are done using "BEGIN
IMMEDIATE TRANSACTION"


Never called...

What is the value of B->autoCommit?

Also, how do we know that B is actually "done"?

If it has no explicitly opened transaction (i.e. if B->autoCommit==1),
then the sqlite3ConnectionUnlocked() function should be called every
time B halts an SQL statement (which happens either in the
sqlite3_reset() or sqlite3_finalize() call, or sometimes from within
sqlite3_step() if it returns SQLITE_DONE or an error).

Is B calling sqlite3_reset() or sqlite3_finalize() on its statement
handles when it is finished with them?

Dan.







Owen


On 9 July 2013 12:51, Dan Kennedy  wrote:


On 07/09/2013 06:41 PM, Owen Haynes wrote:


I have been using a debugger to look at pBlockedConnection  and
pUnlockConnection
also added some extra print statements to the code, sqlite is also
compiled
with debug enabled, and no asserts seem to get triggered.



How do you know A has finished? Does "PRAGMA lock_status"
indicate that it is holding no locks at all?

So in "deadlock 1", when you have:


Connection A is has finished
Connection B is waiting on Connection A
Connection C is waiting on Connection B

Then the following are all true?

   B->pUnlockConnection = A;
   B->pBlockingConnection = A;
   C->pUnlockConnection = B;
   C->pBlockingConnection = B;

Are both B and C in the linked list that starts at global
variable sqlite3BlockedList (and connected by pNextBlocked)?

Dan.







Owen




On 9 July 2013 11:20, Dan Kennedy  wrote:

  On 07/09/2013 04:08 PM, Dan Kennedy wrote:

  On 06/20/2013 03:20 PM, Owen Haynes wrote:

  Hello,

I am currently having some problems with the unlock notify and getting
in a
state of deadlock.

I am using code based on 
http://www.sqlite.org/unlock_notify.html


**,

with
the latest sqlite.

The setup is as follows:
- WAL is on
- Multi Threaded is on
- Temp store is memory
- Database is stored in /dev/shm
- Synchronous is set as normal
- Multi-threading is on
- connection is set as SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE |
SQLITE_OPEN_NOMUTEX | SQLITE_OPEN_SHAREDCACHE

Unlock notify does work I can see my log messages printed when the
callback
is called but sometimes I can end up in the following dead lock
situation.

Each connection is in its own thread

*Deadlock 1*
Connection A is using database
Connection B is waiting on Connection A
Connection C is waiting on Connection B

Connection A is has finished
Connection B is waiting on Connection A
Connection C is waiting on Connection B

  How do you know this is what is happening? Are you looking

at the sqlite3.pBlockedConnection and sqlite3.pUnlockConnection
pointers in a debugger? Or some other method?

Also, if this is easy to reproduce, try running with SQLITE_DEBUG
defined. This will enable some complex asserts that check for
various problems in the unlock-notify code.

Dan.






  *Deadlock 2*

Connection A is using database
Connection B is waiting on Connection A
Connection C is waiting on Connection A

Connection A is has finished
Connection B is waiting on Connection A
Connection C is waiting on Connection A

For some reason sometimes the notify callback does not get issued and
end
up with 2 threads waiting for the callback back to be issued.

Connection B is a writer thread the others only read.

Any ideas?

  Are you handling the case where the xNotify callback is invoked by

sqlite3_unlock_notify() before it returns?

Dan.


  ___

sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

  __**_

sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-**bin/mailman/listinfo/sqlite-**users

Re: [sqlite] Unlock Notify problems

2013-07-09 Thread Owen Haynes

Here is some information I have gathered.

We have a writer thread connection called A and a reader called B, another
reader called C.


A is writing
B is waiting on A with pBlockingConnection = A, and pUnlockConnection A
A is Now done,
A is waiting on B with pBlockingConnection = B and pUnlockConnection B
B is now done but never goes to check to see if any connections are waiting
to be unblocked, sqlite3ConnectionUnlocked never gets called. I have a
print out every time it gets called with the connection which calls it.
C then wants to read and has a pBlockingConnection = B and pUnlockConnection =
B.

And all the connections are then locked up.

This information was gathered from pUnlockConnection and pBlockingConnection
and by examining the blocked list. All writes are done using "BEGIN
IMMEDIATE TRANSACTION"

Owen


On 9 July 2013 12:51, Dan Kennedy  wrote:

> On 07/09/2013 06:41 PM, Owen Haynes wrote:
>
>> I have been using a debugger to look at pBlockedConnection  and
>> pUnlockConnection
>> also added some extra print statements to the code, sqlite is also
>> compiled
>> with debug enabled, and no asserts seem to get triggered.
>>
>
>
> How do you know A has finished? Does "PRAGMA lock_status"
> indicate that it is holding no locks at all?
>
> So in "deadlock 1", when you have:
>
>
> Connection A is has finished
> Connection B is waiting on Connection A
> Connection C is waiting on Connection B
>
> Then the following are all true?
>
>   B->pUnlockConnection = A;
>   B->pBlockingConnection = A;
>   C->pUnlockConnection = B;
>   C->pBlockingConnection = B;
>
> Are both B and C in the linked list that starts at global
> variable sqlite3BlockedList (and connected by pNextBlocked)?
>
> Dan.
>
>
>
>
>
>
>> Owen
>>
>>
>>
>>
>> On 9 July 2013 11:20, Dan Kennedy  wrote:
>>
>>  On 07/09/2013 04:08 PM, Dan Kennedy wrote:
>>>
>>>  On 06/20/2013 03:20 PM, Owen Haynes wrote:

  Hello,
>
> I am currently having some problems with the unlock notify and getting
> in a
> state of deadlock.
>
> I am using code based on 
> http://www.sqlite.org/unlock_notify.html
> 
> >**,
>
> with
> the latest sqlite.
>
> The setup is as follows:
>- WAL is on
>- Multi Threaded is on
>- Temp store is memory
>- Database is stored in /dev/shm
>- Synchronous is set as normal
>- Multi-threading is on
>- connection is set as SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE |
> SQLITE_OPEN_NOMUTEX | SQLITE_OPEN_SHAREDCACHE
>
> Unlock notify does work I can see my log messages printed when the
> callback
> is called but sometimes I can end up in the following dead lock
> situation.
>
> Each connection is in its own thread
>
> *Deadlock 1*
> Connection A is using database
> Connection B is waiting on Connection A
> Connection C is waiting on Connection B
>
> Connection A is has finished
> Connection B is waiting on Connection A
> Connection C is waiting on Connection B
>
>  How do you know this is what is happening? Are you looking
>>> at the sqlite3.pBlockedConnection and sqlite3.pUnlockConnection
>>> pointers in a debugger? Or some other method?
>>>
>>> Also, if this is easy to reproduce, try running with SQLITE_DEBUG
>>> defined. This will enable some complex asserts that check for
>>> various problems in the unlock-notify code.
>>>
>>> Dan.
>>>
>>>
>>>
>>>
>>>
>>>
>>>  *Deadlock 2*
> Connection A is using database
> Connection B is waiting on Connection A
> Connection C is waiting on Connection A
>
> Connection A is has finished
> Connection B is waiting on Connection A
> Connection C is waiting on Connection A
>
> For some reason sometimes the notify callback does not get issued and
> end
> up with 2 threads waiting for the callback back to be issued.
>
> Connection B is a writer thread the others only read.
>
> Any ideas?
>
>  Are you handling the case where the xNotify callback is invoked by
 sqlite3_unlock_notify() before it returns?

 Dan.


  ___
>>> sqlite-users mailing list
>>> sqlite-users@sqlite.org
>>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>> 
>>> >
>>>
>>>  __**_
>> sqlite-users mailing list
>> sqlite-users@sqlite.org
>> http://sqlite.org:8080/cgi-**bin/mailman/listinfo/sqlite-**users
>>
>
> __**_
> sqlite-

Re: [sqlite] Unlock Notify problems

2013-07-09 Thread Dan Kennedy


On 07/09/2013 06:41 PM, Owen Haynes wrote:

I have been using a debugger to look at pBlockedConnection  and
pUnlockConnection
also added some extra print statements to the code, sqlite is also compiled
with debug enabled, and no asserts seem to get triggered.



How do you know A has finished? Does "PRAGMA lock_status"
indicate that it is holding no locks at all?

So in "deadlock 1", when you have:

Connection A is has finished
Connection B is waiting on Connection A
Connection C is waiting on Connection B

Then the following are all true?

  B->pUnlockConnection = A;
  B->pBlockingConnection = A;
  C->pUnlockConnection = B;
  C->pBlockingConnection = B;

Are both B and C in the linked list that starts at global
variable sqlite3BlockedList (and connected by pNextBlocked)?

Dan.







Owen




On 9 July 2013 11:20, Dan Kennedy  wrote:


On 07/09/2013 04:08 PM, Dan Kennedy wrote:


On 06/20/2013 03:20 PM, Owen Haynes wrote:


Hello,

I am currently having some problems with the unlock notify and getting
in a
state of deadlock.

I am using code based on 
http://www.sqlite.org/unlock_**notify.html,
with
the latest sqlite.

The setup is as follows:
   - WAL is on
   - Multi Threaded is on
   - Temp store is memory
   - Database is stored in /dev/shm
   - Synchronous is set as normal
   - Multi-threading is on
   - connection is set as SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE |
SQLITE_OPEN_NOMUTEX | SQLITE_OPEN_SHAREDCACHE

Unlock notify does work I can see my log messages printed when the
callback
is called but sometimes I can end up in the following dead lock
situation.

Each connection is in its own thread

*Deadlock 1*
Connection A is using database
Connection B is waiting on Connection A
Connection C is waiting on Connection B

Connection A is has finished
Connection B is waiting on Connection A
Connection C is waiting on Connection B


How do you know this is what is happening? Are you looking
at the sqlite3.pBlockedConnection and sqlite3.pUnlockConnection
pointers in a debugger? Or some other method?

Also, if this is easy to reproduce, try running with SQLITE_DEBUG
defined. This will enable some complex asserts that check for
various problems in the unlock-notify code.

Dan.







*Deadlock 2*
Connection A is using database
Connection B is waiting on Connection A
Connection C is waiting on Connection A

Connection A is has finished
Connection B is waiting on Connection A
Connection C is waiting on Connection A

For some reason sometimes the notify callback does not get issued and end
up with 2 threads waiting for the callback back to be issued.

Connection B is a writer thread the others only read.

Any ideas?


Are you handling the case where the xNotify callback is invoked by
sqlite3_unlock_notify() before it returns?

Dan.



__**_
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-**bin/mailman/listinfo/sqlite-**users


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Unlock Notify problems

2013-07-09 Thread Owen Haynes

I have been using a debugger to look at pBlockedConnection  and
pUnlockConnection
also added some extra print statements to the code, sqlite is also compiled
with debug enabled, and no asserts seem to get triggered.

Owen




On 9 July 2013 11:20, Dan Kennedy  wrote:

> On 07/09/2013 04:08 PM, Dan Kennedy wrote:
>
>> On 06/20/2013 03:20 PM, Owen Haynes wrote:
>>
>>> Hello,
>>>
>>> I am currently having some problems with the unlock notify and getting
>>> in a
>>> state of deadlock.
>>>
>>> I am using code based on 
>>> http://www.sqlite.org/unlock_**notify.html,
>>> with
>>> the latest sqlite.
>>>
>>> The setup is as follows:
>>>   - WAL is on
>>>   - Multi Threaded is on
>>>   - Temp store is memory
>>>   - Database is stored in /dev/shm
>>>   - Synchronous is set as normal
>>>   - Multi-threading is on
>>>   - connection is set as SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE |
>>> SQLITE_OPEN_NOMUTEX | SQLITE_OPEN_SHAREDCACHE
>>>
>>> Unlock notify does work I can see my log messages printed when the
>>> callback
>>> is called but sometimes I can end up in the following dead lock
>>> situation.
>>>
>>> Each connection is in its own thread
>>>
>>> *Deadlock 1*
>>> Connection A is using database
>>> Connection B is waiting on Connection A
>>> Connection C is waiting on Connection B
>>>
>>> Connection A is has finished
>>> Connection B is waiting on Connection A
>>> Connection C is waiting on Connection B
>>>
>>
> How do you know this is what is happening? Are you looking
> at the sqlite3.pBlockedConnection and sqlite3.pUnlockConnection
> pointers in a debugger? Or some other method?
>
> Also, if this is easy to reproduce, try running with SQLITE_DEBUG
> defined. This will enable some complex asserts that check for
> various problems in the unlock-notify code.
>
> Dan.
>
>
>
>
>
>
>>> *Deadlock 2*
>>> Connection A is using database
>>> Connection B is waiting on Connection A
>>> Connection C is waiting on Connection A
>>>
>>> Connection A is has finished
>>> Connection B is waiting on Connection A
>>> Connection C is waiting on Connection A
>>>
>>> For some reason sometimes the notify callback does not get issued and end
>>> up with 2 threads waiting for the callback back to be issued.
>>>
>>> Connection B is a writer thread the others only read.
>>>
>>> Any ideas?
>>>
>>
>> Are you handling the case where the xNotify callback is invoked by
>> sqlite3_unlock_notify() before it returns?
>>
>> Dan.
>>
>>
> __**_
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-**bin/mailman/listinfo/sqlite-**users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Unlock Notify problems

2013-07-09 Thread Dan Kennedy


On 07/09/2013 04:08 PM, Dan Kennedy wrote:

On 06/20/2013 03:20 PM, Owen Haynes wrote:

Hello,

I am currently having some problems with the unlock notify and 
getting in a

state of deadlock.

I am using code based on http://www.sqlite.org/unlock_notify.html, with
the latest sqlite.

The setup is as follows:
  - WAL is on
  - Multi Threaded is on
  - Temp store is memory
  - Database is stored in /dev/shm
  - Synchronous is set as normal
  - Multi-threading is on
  - connection is set as SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE |
SQLITE_OPEN_NOMUTEX | SQLITE_OPEN_SHAREDCACHE

Unlock notify does work I can see my log messages printed when the 
callback
is called but sometimes I can end up in the following dead lock 
situation.


Each connection is in its own thread

*Deadlock 1*
Connection A is using database
Connection B is waiting on Connection A
Connection C is waiting on Connection B

Connection A is has finished
Connection B is waiting on Connection A
Connection C is waiting on Connection B


How do you know this is what is happening? Are you looking
at the sqlite3.pBlockedConnection and sqlite3.pUnlockConnection
pointers in a debugger? Or some other method?

Also, if this is easy to reproduce, try running with SQLITE_DEBUG
defined. This will enable some complex asserts that check for
various problems in the unlock-notify code.

Dan.






*Deadlock 2*
Connection A is using database
Connection B is waiting on Connection A
Connection C is waiting on Connection A

Connection A is has finished
Connection B is waiting on Connection A
Connection C is waiting on Connection A

For some reason sometimes the notify callback does not get issued and 
end

up with 2 threads waiting for the callback back to be issued.

Connection B is a writer thread the others only read.

Any ideas?


Are you handling the case where the xNotify callback is invoked by
sqlite3_unlock_notify() before it returns?

Dan.



___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Unlock Notify problems

2013-07-09 Thread Owen Haynes

Yes I am covering the case where xNotify is called before sqlite3_unlock_notify
is returned.

My struct which I pass into sqlite3_unlock_notify has a fired flag, this
would then get set to true if xNotify is called in sqlite3_unlock_notify.
This flag is then checked after the sqlite3_unlock_notify call to see if it
has changed from false.

I do near enough a carbon copy of the example code on
http://www.sqlite.org/unlock_notify.html.


Owen



On 9 July 2013 10:08, Dan Kennedy  wrote:

> On 06/20/2013 03:20 PM, Owen Haynes wrote:
>
>> Hello,
>>
>> I am currently having some problems with the unlock notify and getting in
>> a
>> state of deadlock.
>>
>> I am using code based on 
>> http://www.sqlite.org/unlock_**notify.html,
>> with
>> the latest sqlite.
>>
>> The setup is as follows:
>>   - WAL is on
>>   - Multi Threaded is on
>>   - Temp store is memory
>>   - Database is stored in /dev/shm
>>   - Synchronous is set as normal
>>   - Multi-threading is on
>>   - connection is set as SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE |
>> SQLITE_OPEN_NOMUTEX | SQLITE_OPEN_SHAREDCACHE
>>
>> Unlock notify does work I can see my log messages printed when the
>> callback
>> is called but sometimes I can end up in the following dead lock situation.
>>
>> Each connection is in its own thread
>>
>> *Deadlock 1*
>>
>> Connection A is using database
>> Connection B is waiting on Connection A
>> Connection C is waiting on Connection B
>>
>> Connection A is has finished
>> Connection B is waiting on Connection A
>> Connection C is waiting on Connection B
>>
>> *Deadlock 2*
>>
>> Connection A is using database
>> Connection B is waiting on Connection A
>> Connection C is waiting on Connection A
>>
>> Connection A is has finished
>> Connection B is waiting on Connection A
>> Connection C is waiting on Connection A
>>
>> For some reason sometimes the notify callback does not get issued and end
>> up with 2 threads waiting for the callback back to be issued.
>>
>> Connection B is a writer thread the others only read.
>>
>> Any ideas?
>>
>
> Are you handling the case where the xNotify callback is invoked by
> sqlite3_unlock_notify() before it returns?
>
> Dan.
>
> __**_
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-**bin/mailman/listinfo/sqlite-**users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Unlock Notify problems

2013-07-09 Thread Dan Kennedy


On 06/20/2013 03:20 PM, Owen Haynes wrote:

Hello,

I am currently having some problems with the unlock notify and getting in a
state of deadlock.

I am using code based on http://www.sqlite.org/unlock_notify.html, with
the latest sqlite.

The setup is as follows:
  - WAL is on
  - Multi Threaded is on
  - Temp store is memory
  - Database is stored in /dev/shm
  - Synchronous is set as normal
  - Multi-threading is on
  - connection is set as SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE |
SQLITE_OPEN_NOMUTEX | SQLITE_OPEN_SHAREDCACHE

Unlock notify does work I can see my log messages printed when the callback
is called but sometimes I can end up in the following dead lock situation.

Each connection is in its own thread

*Deadlock 1*
Connection A is using database
Connection B is waiting on Connection A
Connection C is waiting on Connection B

Connection A is has finished
Connection B is waiting on Connection A
Connection C is waiting on Connection B

*Deadlock 2*
Connection A is using database
Connection B is waiting on Connection A
Connection C is waiting on Connection A

Connection A is has finished
Connection B is waiting on Connection A
Connection C is waiting on Connection A

For some reason sometimes the notify callback does not get issued and end
up with 2 threads waiting for the callback back to be issued.

Connection B is a writer thread the others only read.

Any ideas?


Are you handling the case where the xNotify callback is invoked by
sqlite3_unlock_notify() before it returns?

Dan.

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Unlock Notify problems

2013-07-09 Thread Owen Haynes

I am still having this problem, and it happens very frequently.

Some more information

Normally have 7 connections on different threads, 2 of these threads are
writers

Owen
On 20 Jun 2013 09:20, "Owen Haynes"  wrote:

> Hello,
>
> I am currently having some problems with the unlock notify and getting in
> a state of deadlock.
>
> I am using code based on http://www.sqlite.org/unlock_notify.html, with
> the latest sqlite.
>
> The setup is as follows:
>  - WAL is on
>  - Multi Threaded is on
>  - Temp store is memory
>  - Database is stored in /dev/shm
>  - Synchronous is set as normal
>  - Multi-threading is on
>  - connection is set as SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE |
> SQLITE_OPEN_NOMUTEX | SQLITE_OPEN_SHAREDCACHE
>
> Unlock notify does work I can see my log messages printed when the
> callback is called but sometimes I can end up in the following dead lock
> situation.
>
> Each connection is in its own thread
>
> *Deadlock 1*
> Connection A is using database
> Connection B is waiting on Connection A
> Connection C is waiting on Connection B
>
> Connection A is has finished
> Connection B is waiting on Connection A
> Connection C is waiting on Connection B
>
> *Deadlock 2*
> Connection A is using database
> Connection B is waiting on Connection A
> Connection C is waiting on Connection A
>
> Connection A is has finished
> Connection B is waiting on Connection A
> Connection C is waiting on Connection A
>
> For some reason sometimes the notify callback does not get issued and end
> up with 2 threads waiting for the callback back to be issued.
>
> Connection B is a writer thread the others only read.
>
> Any ideas?
>
> Owen
>
>
>
>
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Unlock Notify problems

Re: [sqlite] Unlock Notify problems

Re: [sqlite] Unlock Notify problems

Re: [sqlite] Unlock Notify problems

Re: [sqlite] Unlock Notify problems

Re: [sqlite] Unlock Notify problems

Re: [sqlite] Unlock Notify problems

Re: [sqlite] Unlock Notify problems

Re: [sqlite] Unlock Notify problems

Re: [sqlite] Unlock Notify problems

Re: [sqlite] Unlock Notify problems

11 matches

Site Navigation

Mail list logo

Footer information