Re: [sqlite] Shared cache mode issue
On Tue, 2007-01-09 at 08:01 -0800, Peter James wrote: > On 1/9/07, Dan Kennedy <[EMAIL PROTECTED]> wrote: > But it looks to me like commit #3341 (August 2006) covers this > up. #3341 > changes things so that the shared-schema is reset whenever any > connection handle is closed, so it's not possible for the > pointer in > question to go stale. > > > Hey Dan... > > Thanks for confirming this, and I'll check out that patch. Would you > suggest that I file a bug on the issue I reported, or is resetting the > shared schema like that on connection close here to stay? I don't see why it's not here to stay. You could file a bug against 3.3.6 for reference purposes if you like. Dan. > - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Shared cache mode issue
On Mon, 2007-01-08 at 16:03 -0800, Peter James wrote:
> Hey folks...
>
> The context of this message is sqlite library version 3.3.6, using the
> shared-cache mode, effectively following the test_server.c example.
> Immediately upon switching to shared-cache mode we started seeing errors
> like so when preparing statements:
>
> [ERROR] (lib/sqlite/src/build.c:1220) no such collation sequence: garbage>
>
> Drilling down, this is what I'm understanding to be the case... Collators
> are attached to the individual sqlite* handles, remaining valid only while
> the connection to which the handle refers is valid. On the other hand, it
> appears that indexes are stored inside of the schema, and use a lookup
> string ("BINARY", "NOCASE") to find the contained column collators. This
> lookup string is actually in memory allocated as part of the collator, and
> is freed when the connection is closed, leaving a dangling pointer in the
> index.
This only happens with the default collation sequence. In build.c, a
pointer may be copied from sqlite3.pDfltColl->zName into the schema.
This is a bug, for the reasons identified above. (in cvs sources:
line 2479 of build.c).
But it looks to me like commit #3341 (August 2006) covers this up. #3341
changes things so that the shared-schema is reset whenever any
connection handle is closed, so it's not possible for the pointer in
question to go stale.
So if you upgrade to 3.3.7 or newer you should be Ok. Or if you can't
upgrade for your own reasons, maybe add something similar to #3341
(only one line).
Dan.
-
To unsubscribe, send email to [EMAIL PROTECTED]
-
Re: [sqlite] Shared cache mode issue
On 1/8/07, Peter James <[EMAIL PROTECTED]> wrote: Thanks for your response, Ken. I'm not sure I've explained myself properly. It's not that I'm calling sqlite3_enable_shared_cache() multiple times. It's that if I don't maintain a persistent connection while the server is running I end up with a dangling pointer and an error. 1. start server thread a. calls sqlite3_enable_shared_cache() b. waits for incoming commands 2. open connection #1 3. open connection #2 4. prepare and step a query with connection #1 (through the server) 5. close connection #1 6. prepare a query with connection #2 (through the server) I believe multiple connections are specifically warned against in the XUL/javascript documentation for using the firefox version of sqlite. They redesigned it in such a way that it works well only for firefox and can't be used with anything else easily. I use it from a firefox addon. -- The PixAddixImage Collector suite: http://groups-beta.google.com/group/pixaddix SqliteImporter and SqliteReplicator: Command line utilities for Sqlite http://www.reddawn.net/~jsprenkl/Sqlite Cthulhu Bucks! http://www.cthulhubucks.com - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Shared cache mode issue
On 1/8/07, Ken <[EMAIL PROTECTED]> wrote: You could always implement a sqlite3_open call and store it in the g variable, and close it when the server quits. Thanks for your response, Ken. I'm not sure I've explained myself properly. It's not that I'm calling sqlite3_enable_shared_cache() multiple times. It's that if I don't maintain a persistent connection while the server is running I end up with a dangling pointer and an error. 1. start server thread a. calls sqlite3_enable_shared_cache() b. waits for incoming commands 2. open connection #1 3. open connection #2 4. prepare and step a query with connection #1 (through the server) 5. close connection #1 6. prepare a query with connection #2 (through the server) The issue is that at step #6, I get the "no such collation sequence" error because some memory (the collation sequence names) that the shared schema structures depended on went away in step 5. I'm wondering if I'm misunderstanding something, and if there's any RTFM'ing I should have done, cuz I'm not seeing this as a requirement in any of the docs I read on the subject. Thanks, Pete.
Re: [sqlite] Shared cache mode issue
Here is a code snipet from my version if the server thread code
I found that it was doing an enable/disable on the shared cache with the
original logic.
You could always implement a sqlite3_open call and store it in the g variable,
and close it when the server quits.
void *sqlite3_server(void *NotUsed){
if( pthread_mutex_trylock() ){
sqlite3_enable_shared_cache(0);
return 0; /* Another server is already running */
}
// Only enable the shared cache 1 time
sqlite3_enable_shared_cache(1);
Peter James <[EMAIL PROTECTED]> wrote: Hey folks...
The context of this message is sqlite library version 3.3.6, using the
shared-cache mode, effectively following the test_server.c example.
Immediately upon switching to shared-cache mode we started seeing errors
like so when preparing statements:
[ERROR] (lib/sqlite/src/build.c:1220) no such collation sequence:
garbage>
Drilling down, this is what I'm understanding to be the case... Collators
are attached to the individual sqlite* handles, remaining valid only while
the connection to which the handle refers is valid. On the other hand, it
appears that indexes are stored inside of the schema, and use a lookup
string ("BINARY", "NOCASE") to find the contained column collators. This
lookup string is actually in memory allocated as part of the collator, and
is freed when the connection is closed, leaving a dangling pointer in the
index.
>From reading mozilla's docs on how they used the shared cache mode, I have
to guess this dangling pointer thing isn't normally a problem since the
"standard" thing to do is open a (dummy) connection at the beginning of the
server and maintain it until the server ends. In which case, the dummy
connection is the one containing default collator defs and that lookup
string's memory is always valid.
The error above surfaced in our initial implementation of the test_server.c
architecture, where we hadn't yet started using a dummy connection. We were
testing functionality and just opening and closing connections as needed
through the server thread, and noticed that if the first connection happened
to close after the second connection opened, we would end up with the above
error. Before realizing this was perhaps an artefact of not having a dummy
connection, we worked around it by doing a strcpy of the collator name into
malloc'd memory for the index, rather than just pointing at the collator..
Is this a correct interpretation of this situation? If there's a dependency
in shared cache mode where the client must maintain at least one persistent
connection, did I miss documentation on this fact?
I hope I explained myself properly. If not, let me know and I'll try
again. :-)
Thanks,
Pete.
Peter James <[EMAIL PROTECTED]> wrote: Hey folks...
The context of this message is sqlite library version 3.3.6, using the
shared-cache mode, effectively following the test_server.c example.
Immediately upon switching to shared-cache mode we started seeing errors
like so when preparing statements:
[ERROR] (lib/sqlite/src/build.c:1220) no such collation sequence:
garbage>
Drilling down, this is what I'm understanding to be the case... Collators
are attached to the individual sqlite* handles, remaining valid only while
the connection to which the handle refers is valid. On the other hand, it
appears that indexes are stored inside of the schema, and use a lookup
string ("BINARY", "NOCASE") to find the contained column collators. This
lookup string is actually in memory allocated as part of the collator, and
is freed when the connection is closed, leaving a dangling pointer in the
index.
>From reading mozilla's docs on how they used the shared cache mode, I have
to guess this dangling pointer thing isn't normally a problem since the
"standard" thing to do is open a (dummy) connection at the beginning of the
server and maintain it until the server ends. In which case, the dummy
connection is the one containing default collator defs and that lookup
string's memory is always valid.
The error above surfaced in our initial implementation of the test_server.c
architecture, where we hadn't yet started using a dummy connection. We were
testing functionality and just opening and closing connections as needed
through the server thread, and noticed that if the first connection happened
to close after the second connection opened, we would end up with the above
error. Before realizing this was perhaps an artefact of not having a dummy
connection, we worked around it by doing a strcpy of the collator name into
malloc'd memory for the index, rather than just pointing at the collator..
Is this a correct interpretation of this situation? If there's a dependency
in shared cache mode where the client must maintain at least one persistent
connection, did I miss documentation on this fact?
I hope I explained myself properly. If not, let me know and I'll try
again. :-)
Thanks,
Pete.
