[sqlite] Wrong Index Select with Large Table Joined to Small Table
On Fri, Jan 22, 2016 at 1:29 PM, Richard Hipp http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users>> wrote: > That index will not work on queries like this (obviously): > >SELECT * FROM t1 WHERE x IS NULL; > >The index is also useless for sorting: > > SELECT * FROM t1 ORDER BY x; In addition to Dominique's question on Jan 22 about why partial indexes are not useful for sorting, I have a question on the first statement. It is clear why the condition "x IS NULL" would cause the partial index to be worthless, but in my testing, the condition "x IS NOT NULL" also causes the planner to never select this index to use. It seems to me this partial index would be exactly the best index to use because of the clause which makes it a partial index. For me, this is a not an academic discussion - I have found that I have a lot of queries that either use partial indexes or could benefit from them. -Denis On Fri, Jan 22, 2016 at 7:29 AM, Richard Hipp wrote: > On 1/21/16, Denis Burke wrote: > > When > > would you NOT want to add the "where [indexedColumn] IS NOT NULL"? Seems > > like it would always be helpful. > > > > CREATE TABLE t1(x); > CREATE INDEX t1x ON t1(x) WHERE x IS NOT NULL; > > That index will not work on queries like this (obviously): > > SELECT * FROM t1 WHERE x IS NULL; > > The index is also useless for sorting: > > SELECT * FROM t1 ORDER BY x; > > Rule of thumb: Only use "WHERE x IS NOT NULL" if the column being > indexed contains many, many NULL values. > > -- > D. Richard Hipp > drh at sqlite.org >
[sqlite] SQLite crashing
Hi, ALL, I have a strange problem. I am trying to use sqlite in my program. It has a main application and couplef DLLs. I am getting the connection in one of the DLL, then the pointer is passed up to the main application. Upon exiting from the application I'm trying to close the connection and delete all the memory. Unfortunately upon exiting the application it crashes inside sqlite3_mutex_enter(). The comment above the function says: [quote] /* ** Obtain the mutex p. If some other thread already has the mutex, block ** until it can be obtained. */ [/quote] The DLL does not start any threads, in fact the application will be 1 thread only. So is there some compile-time switch I should use to mitigate the issue? Moreover I don't understand why am I getting the assertion - there is no MT involved. Can someone shed some lights? Thank you.
[sqlite] SQLite crashing
If it matters, I'm trying Win 8.1 with MSVC 2010. Thank you. On Mon, Jan 25, 2016 at 12:18 AM, Igor Korot wrote: > Hi, ALL, > I have a strange problem. > > I am trying to use sqlite in my program. It has a main application and > couplef DLLs. > > I am getting the connection in one of the DLL, then the pointer is passed up > to the main application. > > Upon exiting from the application I'm trying to close the connection and > delete all the memory. > > Unfortunately upon exiting the application it crashes inside > sqlite3_mutex_enter(). > The comment above the function says: > > [quote] > /* > ** Obtain the mutex p. If some other thread already has the mutex, block > ** until it can be obtained. > */ > [/quote] > > The DLL does not start any threads, in fact the application will be 1 > thread only. > So is there some compile-time switch I should use to mitigate the issue? > > Moreover I don't understand why am I getting the assertion - there is no MT > involved. > > Can someone shed some lights? > > Thank you.
[sqlite] Suggesting an Index to Use Can Improve Performance Even When Planner was Already Going to Use It
I was analyzing hundreds of queries in our system and a few stood out (average query time is subsecond, but these few that stood out were 10+ seconds). Here is the basic query for these problem few: SELECT T1.A from T1 INNER JOIN T2 on T1.B=T2.B and T1.C=T2.C where T1.D='2015-12-31' and T2.E=2 GROUP BY A When run as-is: it takes 20 seconds. and explain query plan is 3 lines: 1: SEARCH TABLE T2 USING AUTOMATIC PARTIAL COVERING INDEX (E=?) 2: SEARCH TABLE T1 USING INDEX Indx_T1 (D=?) 3: USE TEMP B-TREE FOR GROUP BY With my knowledge of the DB, these seem like the right indexes to use. In trying different options I noticed that when I added an INDEXED BY phrase to T1 it helped significantly - even though it was already using the same index I am suggesting! SELECT T1.A from T1 INDEXED BY Indx_T1 INNER JOIN T2 on T1.B=T2.B and T1.C=T2.C where T1.D='2015-12-31' and T2.E=2 GROUP BY A This query takes 0.6 second and this this explain query plan: 1: SEARCH TABLE T1 USING INDEX Indx_T1 (D=?) 2: SEARCH TABLE T2 USING AUTOMATIC PARTIAL COVERING INDEX (E=? AND B=? AND C=?) 3: USE TEMP B-TREE FOR GROUP BY FWIW. T1 has 1.4M rows and T2 has 2000. So my questions are simply: why would these two very similar queries have such difference performance results since it looks to me like SQLite is doing nearly the same thing in both cases, and since they are so different, why would the planner need that INDEXED BY hint in order to get the best performance?
[sqlite] Suggesting an Index to Use Can Improve Performance Even When Planner was Already Going to Use It
Adding the INDEXED BY changed the nesting order of the loops. Also, note the word AUTOMATIC in the query plan: This means that the QP has decided it is worthwhile to build an Index just for this one query. The first plan in English: Build an Index on T2.E = 2 that also contains the fields B and C For each record of T2 in the index Partial scan table T1 for records matching the constraint for D Compare fields B and C from T1 record and T2 index entry The second plan in English Build an Index on T2.E = 2 that also contains the fields B and C Partial scan table T1 for records matching the constraint for D Index lookup on E,B,C with B and C taken from T1 Creating an Index on T2 (E,B,C) should speed things up. Also, running ANALYZE should help the query planner in estimating correct costs. -Urspr?ngliche Nachricht- Von: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite-users-bounces at mailinglists.sqlite.org] Im Auftrag von Denis Burke Gesendet: Montag, 25. J?nner 2016 08:17 An: SQLite mailing list Betreff: [sqlite] Suggesting an Index to Use Can Improve Performance Even When Planner was Already Going to Use It I was analyzing hundreds of queries in our system and a few stood out (average query time is subsecond, but these few that stood out were 10+ seconds). Here is the basic query for these problem few: SELECT T1.A from T1 INNER JOIN T2 on T1.B=T2.B and T1.C=T2.C where T1.D='2015-12-31' and T2.E=2 GROUP BY A When run as-is: it takes 20 seconds. and explain query plan is 3 lines: 1: SEARCH TABLE T2 USING AUTOMATIC PARTIAL COVERING INDEX (E=?) 2: SEARCH TABLE T1 USING INDEX Indx_T1 (D=?) 3: USE TEMP B-TREE FOR GROUP BY With my knowledge of the DB, these seem like the right indexes to use. In trying different options I noticed that when I added an INDEXED BY phrase to T1 it helped significantly - even though it was already using the same index I am suggesting! SELECT T1.A from T1 INDEXED BY Indx_T1 INNER JOIN T2 on T1.B=T2.B and T1.C=T2.C where T1.D='2015-12-31' and T2.E=2 GROUP BY A This query takes 0.6 second and this this explain query plan: 1: SEARCH TABLE T1 USING INDEX Indx_T1 (D=?) 2: SEARCH TABLE T2 USING AUTOMATIC PARTIAL COVERING INDEX (E=? AND B=? AND C=?) 3: USE TEMP B-TREE FOR GROUP BY FWIW. T1 has 1.4M rows and T2 has 2000. So my questions are simply: why would these two very similar queries have such difference performance results since it looks to me like SQLite is doing nearly the same thing in both cases, and since they are so different, why would the planner need that INDEXED BY hint in order to get the best performance? ___ sqlite-users mailing list sqlite-users at mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users ___ Gunter Hick Software Engineer Scientific Games International GmbH FN 157284 a, HG Wien Klitschgasse 2-4, A-1130 Vienna, Austria Tel: +43 1 80100 0 E-Mail: hick at scigames.at This communication (including any attachments) is intended for the use of the intended recipient(s) only and may contain information that is confidential, privileged or legally protected. Any unauthorized use or dissemination of this communication is strictly prohibited. If you have received this communication in error, please immediately notify the sender by return e-mail message and delete all copies of the original communication. Thank you for your cooperation.
[sqlite] SQLite crashing
Igor Korot wrote: > Upon exiting from the application When exactly? Is DllMain() involved in any way? Regards, Clemens
[sqlite] Bug: Successfully committed transaction rolled back after power failure
Hi Rowan!
>> Shutting down power right after a successfully committed
>> transaction rolls back that transaction on next startup.
>
> nitpick: This is sqlite behaving as advertised. See
> https://www.sqlite.org/lockingv3.html section 5.0 step 6, and
> https://www.sqlite.org/atomiccommit.html section 3.11 which explain that
> the presence of a [well-formed] journal file is the mechanism via which
> sqlite discriminates between a committed and in-progress transaction.
>
> ie. according to sqlite a transaction is *not* successfully committed if
> the journal file is still present, so its well within its rights to
> rollback in this scenario.
Yes, a hot journal file means an incomplete transaction and should
absolutly roll back. What actually bugs me is that this happens after
COMMIT has returned without an error.
> That said, syncing the directory doesn't sound like a terrible idea. But
> it's not clear to me that the cost of another sync every transaction is
> worth a marginal reduction in the power-failure-leads-to-rollback window.
The actual costs are not clear to me either. I hope that someone on the
list with more experience in that field knows more.
> That's if it even reduces the window; it wouldn't surprise me to find that
> the journal is removed from disk after the same delay both with and without
> dirsync, the difference being that the dirsync prevents sqlite from
> returning control to your code until its certain the commit has persisted.
The filesystem used (QNX6) is advertised as performing a full sync on
every fsync and automatically some delay time after each write (10s),
and it looks like it is really doing so. If it wouldn't sync on fsync,
or if fsync would just wait until the automatic sync is through, I would
surely open a ticket with QNX and have some stern words with their resident.
That said I would expect the dirsync to shorten the window of
unnecessary rollback by an average of 5s in our case, which may or may
not be relevant to other users. On other filesystems with longer times
until it syncs anyway (I remember 30s from my first Linux box) the
reduction might be more substantial.
> There's certainly a surprising result here:
>
> if (sqlite3_exec("COMMIT") == SQLITE_OK) {
> /* post-commit logic */
> }
>
> Logically, the code in the if block can reasonably assume that the
> transaction will not rollback.
And that's where the potential for a tradeoff lies. Should SQLite
guarantee that in post-commit the transaction is done, durably, even for
the cost of the additional? Or should it not guarantee durability an be
a bit (or much, I don't know) faster?
Personally I tend to assume a database is made for guaranteeing ACID
properties, except when the user explicitly decides otherwise. So in my
eyes an fsync plus possibly an #ifdef for explicitly disabling it would
be the cleanest thing. OTOH it might surprise users that are used to the
higher performance/lower load and don't care for durability, so an
#ifdef for explicitly enabling the fsync instead might still be a good
compromise.
> But as you have discovered this is not
> always true with JOURNAL_MODE=DELETE unless the dirsync is performed. To be
> fair I don't think there are many power-failure scenarios where the
> post-commit logic would have a chance to do anything significant, so the
> incorrect assumption will usually be moot.
Yes, in a normal power failure you ?only? win more time, a few seconds
or more, depending on the filesystem's details.
BTW., as you have mentioned JOURNAL_MODE=DELETE?I have not examined what
happens in the other journal modes, i.e. I'm not sure whether
invalidating the journal by other means has the same lack of sync or
not. A proper patch might need to involve the other modes, too.
> In your case it sounds like a controlled shutdown - is there a reason you
> don't do a full disk sync before that?
Yes, it is a controlled shutdown, so in my case the /* post-commit logic
*/ basically pulls the plug.
Trouble is that I only control the database, not the shutdown procedure
(this is a commercial product with several hundred people working on
different aspects of the system). So while I can try to ask the shutdown
crew to sync like any real computer would do, I ultimately have no
saying in that but still need to ensure survival of data.
Without the patch (which I do have applied locally, of course), me and
everyone with a similar usecase get into trouble for relying on the
quite excellent reputation of SQLite. Actually I first fingerpointed to
the flash hardware or its driver, because ?SQLite is well-tested and
doesn't have this kind of bugs? :-)
S.M.
--
Dipl.-Phys. (Univ) Stefan Meinlschmidt, Senior Software Engineer
Am Wolfsmantel 46, 91058 Tennenlohe, Germany
Tel: +49-8458-3332-531 stefan.meinlschmidt at esolutions.de
Fax: +49-8458-3332-20-531
[sqlite] Can't open database via symlink
I have discovered that opening a database via a symlink works only if the symlink is in the current directory. Otherwise, this happens: ; mkdir x y ; sqlite3 x/foo.sqlite 'create table foo(x);' ; sqlite3 x/foo.sqlite 'select count(*) from foo;' 0 ; ln -s ../x/foo.sqlite y ; cmp x/foo.sqlite y/foo.sqlite ; sqlite3 y/foo.sqlite 'select count(*) from foo;' Error: unable to open database "y/foo.sqlite": unable to open database file The problem seems to have been introduced with this revision: uuid: ? ? ? ? c7c8105099c0412ac6c605f98987092c10bde57c 2015-10-31 17:58:33 UTC parent: ? ? ? 9f19420b0a79dff65fc3a9d548f4b3fc4955f9f9 2015-10-30 20:54:25 UTC merged-into: ?6d5ce3ede4c7038c19a77268a5a7b9d5650933c2 2015-11-02 15:08:56 UTC tags: ? ? ? ? follow-symlinks comment: ? ? ?On unix, if a file is opened via a symlink, create, read and ? ? ? ? ? ? ? write journal and wal files based on the name of the actual db ? ? ? ? ? ? ? file, not the symlink. (user: dan) Reading the source code for?unixFullPathname in src/os_unix.c, I see that the filename is created by appending a slash, then the contents of the symlink, to the current working directory. This is wrong, unless the symlink is in the current working directory. I think the proper algorithm should go like this, if the given filename is a symlink: Read the contents of the symlink. If it begins with a slash, replace the original filename by this. Otherwise, split up the given filename as ?dirname/filename? where filename containsno slashes, but dirname may. (Do nothing if the given filename contains no slashes to begin with). Append a slash and the contents of the symlink to ?dirname?. This is the new filename. (If the given filename contained no slash, just use the contents of the symlink.) Finally, don?t forget that the symlink may point to a symlink! If so, repeat the whole procedure. Repeat until the target is not an existing symlink, or some counter signals a likely symlink loop. (Aim to use the same limit as the built-in symlink resolution algorithm.) Only after all this is done, and only if absolute filenames are required, should the resulting filename be appended to the current working directory. One final parenthetical remark: I assume the code creates an absolute filename because the program might change working directory, and the sqlite code must be able to create new temporary files in the directory of the database file, is that right? But this can backfire if getcwd fails: ; mkdir -p /tmp/foo/bar ; cd /tmp/foo/bar ; chmod 000 /tmp/foo ; sqlite3 foo.sqlite 'create table foo (x);' Error: unable to open database "foo.sqlite": unable to open database file This error will surely be mystifying to whoever encounters it. At least, it would be good if the inability to run getcwd makes it into the error message somehow. ? Harald
[sqlite] Bug: Successfully committed transaction rolled back after power failure
On 1/19/16, Meinlschmidt Stefan wrote: > > Shutting down power right after a successfully committed > transaction rolls back that transaction on next startup. As you observe, this is a file-system dependent thing, and probably only happens on the QNX filesystem. I will see if we can add a compile-time option to the next release that will allow you to force a directory sync after the rollback journal commits. But this won't be on by default because it would do nothing but slow down commits on the overwhelming majority of SQLite users. Meanwhile, a good work-around for you might be to use either PRAGMA journal_mode=TRUNCATE; PRAGMA journal_mode=PERSIST; Both of which sync the rollback-journal upon commit. Or, use: PRAGMA journal_mode=WAL; PRAGMA synchronous=FULL; which causes the write-ahead log to be synced following every commit. Note that the default behavior for WAL mode is that the WAL is not synced, and hence committed transactions might rollback following a power loss. This is the behavior most people desire (not that their transactions roll back but rather they are willing to endure that in exchange for fewer fsyncs.) You must set "PRAGMA synchronous=FULL" to cause the WAL to be synced after each transaction. -- D. Richard Hipp drh at sqlite.org
[sqlite] wrong convertion to absolute path
Hi,
I'm testing version 310.
and I'm quite sure that commit:
On unix, if a file is opened via a symlink, create, read and write
journal and wal files based on the name of the actual db file, not the
symlink.
introduce a bug in the following condition:
1. create a realive symlink
2. use sqlite with current working dir different from where the symlink
is created.
the problem came from that the relative path is converted to absolute
path using the current working directory. But this is not corrected if
the original file is a symbolc link. For a symlink relative means
relative from the position of symlink.
I'm testing the following patch that seams to fix the issue.
Regards
Michele Dionisio
diff --git a/src/os_unix.c b/src/os_unix.c
index ee9b556..cfd904e 100644
--- a/src/os_unix.c
+++ b/src/os_unix.c
@@ -5927,6 +5927,28 @@ static int unixAccess(
return SQLITE_OK;
}
+/*
+** Turn an offset inside the given path to the last end of the last
+** directory name
+**
+** return in valid only if bigger than 0
+*/
+static int dirnameLastChrPos(
+const char *path
+){
+ const char *last_slash = strrchr (path, '/');
+
+ if (last_slash != NULL) {
+ for (; last_slash != path; --last_slash) {
+ if (last_slash[-1] != '/') {
+break;
+ }
+ }
+ return (last_slash - path);
+ }
+
+ return -1;
+}
/*
** Turn a relative pathname into a full pathname. The relative path
@@ -5970,6 +5992,15 @@ static int unixFullPathname(
nByte = sqlite3Strlen30(zOut);
}else{
zOut[nByte] = '\0';
+if( zOut[0]!='/' && zPath[0]=='/' ) { // if symlink is not
relative and original file name in not relative
+int nDir = dirnameLastChrPos(zPath);
+if (nDir>0 && (nDir+nByte+1)http://www.powersoft.it>
*Michele Dionisio |*Embedded System Manager
*skype:* m.dionisio *| email:* michele.dionisio at powersoft.it
*HQ Italy:* Via E. Conti, 5 *|* 50018 Scandicci (FI) Italy
*direct phone:* +39 055 735 0230 *| Fax:* +39 055 735 6235
*web site:* www.powersoft.it
Green Audio Power
[sqlite] Bug: Successfully committed transaction rolled back after power failure
On 1/19/16, Meinlschmidt Stefan wrote: > > Shutting down power right after a successfully committed > transaction rolls back that transaction on next startup. Patches checked in: https://www.sqlite.org/src/info/30671345b1c1ee55 https://www.sqlite.org/draft/compile.html#extra_durable -- D. Richard Hipp drh at sqlite.org
[sqlite] Bug: Successfully committed transaction rolled back after power failure
On 25/01/16 14:14, Richard Hipp wrote: > On 1/19/16, Meinlschmidt Stefan wrote: >> >> Shutting down power right after a successfully committed >> transaction rolls back that transaction on next startup. > > Patches checked in: > > https://www.sqlite.org/src/info/30671345b1c1ee55 > https://www.sqlite.org/draft/compile.html#extra_durable Does this mean that if I use SQLite SQLITE_EXTRA_DURABLE=0, PRAGMA journal_mode=DELETE and PRAGMA synchronous=FULL, SQLite could loose a transaction that it said to be committed depending on the VFS? If so, why isn't SQLITE_EXTRA_DURABLE=1 the default? Should correctness be more important than performance, except when the constraints are such that correctness has to be sacrificed for performance? The trade-off that is described in the description of SQLite SQLITE_EXTRA_DURABLE reads like an excerpt from the MySQL manual when MyISAM was still widely used. Perhaps I'm also too irritated by discussions with advocates of MySQL who would argue against the fact that proper transactions were necessary because the DBMS would be faster without them. That is not to say that the ACID properties and transactions solve every concurrency or correctness problem but they help significantly. - Matthias-Christian
[sqlite] SQLite crashing
Hi, Clemens, On Mon, Jan 25, 2016 at 2:42 AM, Clemens Ladisch wrote: > Igor Korot wrote: >> Upon exiting from the application > > When exactly? Is DllMain() involved in any way? The DLL where SQLite pointer is alocated there is a DLLMain. Then this pointer is brought back to the main application (wxWidgets-based). At this point the DLL is unloaded. When I close the main frame I am trying to disconnect from the database and delete the pointer. But the crash occur when disconnecting. Thank you. > > > Regards, > Clemens > ___ > sqlite-users mailing list > sqlite-users at mailinglists.sqlite.org > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
[sqlite] Bug: Successfully committed transaction rolled back after power failure
On 1/25/16, Matthias-Christian Ott wrote: > > Does this mean that if I use SQLite SQLITE_EXTRA_DURABLE=0, PRAGMA > journal_mode=DELETE and PRAGMA synchronous=FULL, SQLite could loose a > transaction that it said to be committed depending on the VFS? Sort of. This appears to be true if you are running on QNX and you lose power (or do a hard reset) shortly after the transaction commits. It might be the case on other OSes/filesystems but it has never before been observed. This is not new behavior. This is apparently what SQLite has been doing for 15 years, across quadrillions of transactions on many billions of devices, and it has never before caused any issues, until just recently when Mr. Meinlschmidt upgraded to a newer version of QNX. > > If so, why isn't SQLITE_EXTRA_DURABLE=1 the default? Should correctness > be more important than performance, except when the constraints are such > that correctness has to be sacrificed for performance? > > The trade-off that is described in the description of SQLite > SQLITE_EXTRA_DURABLE reads like an excerpt from the MySQL manual when > MyISAM was still widely used. Perhaps I'm also too irritated by > discussions with advocates of MySQL who would argue against the fact > that proper transactions were necessary because the DBMS would be faster > without them. That is not to say that the ACID properties and > transactions solve every concurrency or correctness problem but they > help significantly. > As you point out, it is an engineering tradeoff. The feedback I receive is that most users of SQLite would much rather avoid the extra directory syncs, even if it means having the last transaction rollback following a power loss. Most developers do not care that much about durability, at least not enough to want to take the performance hit of syncing the directory after every unlink. Non-durable commits on power-loss have long been the default in WAL mode (run-time fixable by setting PRAGMA synchronous=FULL) and nobody has before ever complained. Most people consider this a feature. In fact, if I recall correctly, we first made synchronous=NORMAL the default in WAL mode by popular request. WAL mode used to default to power-loss durable but people requested the change for performance reasons. Note especially that this is about durability, not consistency. SQLite guarantees consistency regardless. People care about consistency. Durability, not so much. I'm not a MySQL expert, but I think the historical complaints about MyISAM had more to do with consistency than with durability, did they not? -- D. Richard Hipp drh at sqlite.org
[sqlite] SQLite crashing
Igor, You can't safely pass a SQLite handle between different SQL DLLs that way if they're both built with their own copy of the amalgamation (or link to things built with different copies). SQLite uses a handful of global variables, but each DLL has its own copy of each of these global variables and they can and will have different values, which can mess things up. I ran into a version of this problem when I tried to load a 2nd DLL built with its own copy of the sqlite3.c amalgamation. I fixed that by exposing the SQLite3 entrypoints in the first DLL and linking the second DLL against it so there was only one copy of the amalgamation used for that SQLite3 handle. Peter On 1/24/2016 10:18 PM, Igor Korot wrote: > Hi, ALL, > I have a strange problem. > > I am trying to use sqlite in my program. It has a main application and > couplef DLLs. > > I am getting the connection in one of the DLL, then the pointer is passed up > to the main application. > > Upon exiting from the application I'm trying to close the connection and > delete all the memory. > > Unfortunately upon exiting the application it crashes inside > sqlite3_mutex_enter(). > The comment above the function says: > > [quote] > /* > ** Obtain the mutex p. If some other thread already has the mutex, block > ** until it can be obtained. > */ > [/quote] > > The DLL does not start any threads, in fact the application will be 1 > thread only. > So is there some compile-time switch I should use to mitigate the issue? > > Moreover I don't understand why am I getting the assertion - there is no MT > involved. > > Can someone shed some lights? > > Thank you. > ___ > sqlite-users mailing list > sqlite-users at mailinglists.sqlite.org > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users >
[sqlite] Bug: Successfully committed transaction rolled back after power failure
On Mon, Jan 25, 2016 at 10:08 AM, Matthias-Christian Ott wrote: > > If so, why isn't SQLITE_EXTRA_DURABLE=1 the default? Should correctness > be more important than performance, except when the constraints are such > that correctness has to be sacrificed for performance? > I wouldn't want that, as that would entirely depend on the application. Maybe I'm an edge case, but I've written apps where the database is 'production', but not 'valuable' enough to have performance slapped. The app didn't care if the data was destroyed. It'd go and recompile what it needed from the external sources from which the now-destroyed database was created on. It basically was a cache type implementation, but changes to the database then were pushed to whatever the external source was. SQLite is also mostly looking at devices that are embeded, limited on performance, and not a lot of wiggle room where performance would seriously degrade the "worth-while-ness" of the device. ... which is why I no longer have a Rogers PVR as it'd take 5+ seconds to do anything...ahem... Anyways.. You also have to look at balance across many millions (or is it billions?) of devices out there that use SQLite for their primary operations. Slapping a serious performance decrease on devices where time and performance is essential BY DEFAULT seems a little strict. And all at the cost of one transaction? To some, maybe a worthwhile trade off, sure, but I'm not sure I'm be overly outraged if my wifes music box (iPod) suddenly had to recompile its music list.
[sqlite] Bug: Successfully committed transaction rolled back after power failure
On 1/25/16, Stephen Chrzanowski wrote: > > You also have to look at balance across many millions (or is it billions?) > of devices out there that use SQLite for their primary operations. Billions and billions. > Slapping a serious performance decrease on devices where time and > performance is essential BY DEFAULT seems a little strict. And all at the > cost of one transaction? To some, maybe a worthwhile trade off, sure, but > I'm not sure I'm be overly outraged if my wifes music box (iPod) suddenly > had to recompile its music list. I don't think it is even that serious. This problem is that if you (for example) set a new bookmark on your browser just as the cat is tripping over the power cord, then after reboot the bookmark disappears. The bookmark database is still completely intact - it just went backwards in time a little. -- D. Richard Hipp drh at sqlite.org
[sqlite] Bug: Successfully committed transaction rolled back after power failure
On Mon, Jan 25, 2016 at 5:08 PM, Richard Hipp wrote: > On 1/25/16, Stephen Chrzanowski wrote: > > > > You also have to look at balance across many millions (or is it > billions?) > > of devices out there that use SQLite for their primary operations. > > Billions and billions. > https://en.wikipedia.org/wiki/Billions_and_Billions > I don't think it is even that serious. This problem is that if you > (for example) set a new bookmark on your browser just as the cat is > tripping over the power cord, then after reboot the bookmark > disappears. The bookmark database is still completely intact - it > just went backwards in time a little. > Or, alternately, it _never went forward_ in time. -- - stephan beal http://wanderinghorse.net/home/stephan/ http://gplus.to/sgbeal "Freedom is sloppy. But since tyranny's the only guaranteed byproduct of those who insist on a perfect world, freedom will have to do." -- Bigby Wolf
[sqlite] Bug: Successfully committed transaction rolled back after power failure
On 2016/01/25 5:08 PM, Matthias-Christian Ott wrote: > On 25/01/16 14:14, Richard Hipp wrote: >> On 1/19/16, Meinlschmidt Stefan wrote: >>> Shutting down power right after a successfully committed >>> transaction rolls back that transaction on next startup. >> Patches checked in: >> >> https://www.sqlite.org/src/info/30671345b1c1ee55 >> https://www.sqlite.org/draft/compile.html#extra_durable > Does this mean that if I use SQLite SQLITE_EXTRA_DURABLE=0, PRAGMA > journal_mode=DELETE and PRAGMA synchronous=FULL, SQLite could loose a > transaction that it said to be committed depending on the VFS? > > If so, why isn't SQLITE_EXTRA_DURABLE=1 the default? Should correctness > be more important than performance, except when the constraints are such > that correctness has to be sacrificed for performance? I understand your concern, but don't confuse the loss of a transaction with any form of inconsistency or broken database integrity. It essentially means that on some OSes (to knowledge, only QNX currently) in the event of power loss or system breakdown right at the moment you were editing and saving some value, you might suffer the inconvenience of having to add that one song to your music list again, or have to add the name of that one contact again. This is not really an end-of-the-World-OMG-we-all-gonna-die! problem in general application terms. It's quite acceptable to 99% of databases. Sure, if your database is critical and needs the durability to be absolute, you HAVE to go full synchronous regardless of OS, but that must already be obvious at the very onset of your design and everything you do or decide must keep this in mind. Such a developer would never simply roll with the defaults of any DB system - I'd hope. The very idea of defaults is to cater for the majority, not the rarest of use-cases.
[sqlite] SQLite crashing
Hi, Peter,
On Mon, Jan 25, 2016 at 10:50 AM, Peter Aronson wrote:
> Igor,
>
> You can't safely pass a SQLite handle between different SQL DLLs that way if
> they're both built with their own copy of the amalgamation (or link to
> things built with different copies). SQLite uses a handful of global
> variables, but each DLL has its own copy of each of these global variables
> and they can and will have different values, which can mess things up. I
> ran into a version of this problem when I tried to load a 2nd DLL built with
> its own copy of the sqlite3.c amalgamation. I fixed that by exposing the
> SQLite3 entrypoints in the first DLL and linking the second DLL against it
> so there was only one copy of the amalgamation used for that SQLite3 handle.
The SQLite is built only once and with just one version of the code.
Consider following pseudo-code:
In DLL:
BOOL APIENTRY DLLMain()
{
}
extern "C" __declspec(dllexport) Database *CreateObject(Database *db)
{
db = new SQLiteDatabase();
db->Connect();
return db;
}
In the main application:
mainframe.h:
class MainFrame
{
public:
MainFrame();
~MainFrame();
void ConnectToDb();
private:
Database *m_db;
};
mainframe.cpp:
void MainFrame::ConnectToDb()
{
Database *db = NULL;
LoadLibrary();
func = GetProcAddress();
m_db = func( db );
}
MainFrame::~MainFrame()
{
delete m_db; // this is where the crash happens
}
The pointer address are the same in DLL and main application MainFrame class.
And as I said the crash occurs when it tries to acquire the mutex lock.
Thank you.
>
> Peter
>
>
>
>
> On 1/24/2016 10:18 PM, Igor Korot wrote:
>>
>> Hi, ALL,
>> I have a strange problem.
>>
>> I am trying to use sqlite in my program. It has a main application and
>> couplef DLLs.
>>
>> I am getting the connection in one of the DLL, then the pointer is passed
>> up
>> to the main application.
>>
>> Upon exiting from the application I'm trying to close the connection and
>> delete all the memory.
>>
>> Unfortunately upon exiting the application it crashes inside
>> sqlite3_mutex_enter().
>> The comment above the function says:
>>
>> [quote]
>> /*
>> ** Obtain the mutex p. If some other thread already has the mutex, block
>> ** until it can be obtained.
>> */
>> [/quote]
>>
>> The DLL does not start any threads, in fact the application will be 1
>> thread only.
>> So is there some compile-time switch I should use to mitigate the issue?
>>
>> Moreover I don't understand why am I getting the assertion - there is no
>> MT
>> involved.
>>
>> Can someone shed some lights?
>>
>> Thank you.
>> ___
>> sqlite-users mailing list
>> sqlite-users at mailinglists.sqlite.org
>> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>>
>
> ___
> sqlite-users mailing list
> sqlite-users at mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
[sqlite] SQLite crashing
Where do you pass to the dll something that goes to sqlite3_close(db); ?
( https://www.sqlite.org/c3ref/close.html )
When that happens, does m_db get set to NULL (or now refers to memory
that is now NULL)
Do you check for m_db == NULL before deleting it?
regards,
Adam DeVita
On Mon, Jan 25, 2016 at 11:16 AM, Igor Korot wrote:
> Hi, Peter,
>
> On Mon, Jan 25, 2016 at 10:50 AM, Peter Aronson wrote:
>> Igor,
>>
>> You can't safely pass a SQLite handle between different SQL DLLs that way if
>> they're both built with their own copy of the amalgamation (or link to
>> things built with different copies). SQLite uses a handful of global
>> variables, but each DLL has its own copy of each of these global variables
>> and they can and will have different values, which can mess things up. I
>> ran into a version of this problem when I tried to load a 2nd DLL built with
>> its own copy of the sqlite3.c amalgamation. I fixed that by exposing the
>> SQLite3 entrypoints in the first DLL and linking the second DLL against it
>> so there was only one copy of the amalgamation used for that SQLite3 handle.
>
> The SQLite is built only once and with just one version of the code.
>
> Consider following pseudo-code:
>
> In DLL:
>
> BOOL APIENTRY DLLMain()
> {
> }
>
> extern "C" __declspec(dllexport) Database *CreateObject(Database *db)
> {
> db = new SQLiteDatabase();
> db->Connect();
> return db;
> }
>
> In the main application:
>
> mainframe.h:
>
> class MainFrame
> {
> public:
> MainFrame();
> ~MainFrame();
> void ConnectToDb();
> private:
> Database *m_db;
> };
>
> mainframe.cpp:
>
> void MainFrame::ConnectToDb()
> {
> Database *db = NULL;
> LoadLibrary();
> func = GetProcAddress();
> m_db = func( db );
> }
>
> MainFrame::~MainFrame()
> {
> delete m_db; // this is where the crash happens
> }
>
> The pointer address are the same in DLL and main application MainFrame class.
> And as I said the crash occurs when it tries to acquire the mutex lock.
>
> Thank you.
>
>>
>> Peter
>>
>>
>>
>>
>> On 1/24/2016 10:18 PM, Igor Korot wrote:
>>>
>>> Hi, ALL,
>>> I have a strange problem.
>>>
>>> I am trying to use sqlite in my program. It has a main application and
>>> couplef DLLs.
>>>
>>> I am getting the connection in one of the DLL, then the pointer is passed
>>> up
>>> to the main application.
>>>
>>> Upon exiting from the application I'm trying to close the connection and
>>> delete all the memory.
>>>
>>> Unfortunately upon exiting the application it crashes inside
>>> sqlite3_mutex_enter().
>>> The comment above the function says:
>>>
>>> [quote]
>>> /*
>>> ** Obtain the mutex p. If some other thread already has the mutex, block
>>> ** until it can be obtained.
>>> */
>>> [/quote]
>>>
>>> The DLL does not start any threads, in fact the application will be 1
>>> thread only.
>>> So is there some compile-time switch I should use to mitigate the issue?
>>>
>>> Moreover I don't understand why am I getting the assertion - there is no
>>> MT
>>> involved.
>>>
>>> Can someone shed some lights?
>>>
>>> Thank you.
>>> ___
>>> sqlite-users mailing list
>>> sqlite-users at mailinglists.sqlite.org
>>> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>>>
>>
>> ___
>> sqlite-users mailing list
>> sqlite-users at mailinglists.sqlite.org
>> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
> ___
> sqlite-users mailing list
> sqlite-users at mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
--
--
VerifEye Technologies Inc.
151 Whitehall Dr. Unit 2
Markham, ON
L3R 9T1
[sqlite] SQLite crashing
Igor Korot wrote:
> The DLL where SQLite pointer is alocated there is a DLLMain.
And what does it do?
> BOOL APIENTRY DLLMain()
> {
> }
Is it really empty?
> MainFrame::~MainFrame()
> {
> delete m_db; // this is where the crash happens
> }
When exactly is this destructor called?
Can you show a stack trace?
Regards,
Clemens
[sqlite] SQLite crashing
How are you building the DLL and the executable? Are they both
including a copy of sqlite3.obj? Or are you using a sqlite3.dll?
Peter
On 1/25/2016 9:16 AM, Igor Korot wrote:
> Hi, Peter,
>
> On Mon, Jan 25, 2016 at 10:50 AM, Peter Aronson wrote:
>> Igor,
>>
>> You can't safely pass a SQLite handle between different SQL DLLs that way if
>> they're both built with their own copy of the amalgamation (or link to
>> things built with different copies). SQLite uses a handful of global
>> variables, but each DLL has its own copy of each of these global variables
>> and they can and will have different values, which can mess things up. I
>> ran into a version of this problem when I tried to load a 2nd DLL built with
>> its own copy of the sqlite3.c amalgamation. I fixed that by exposing the
>> SQLite3 entrypoints in the first DLL and linking the second DLL against it
>> so there was only one copy of the amalgamation used for that SQLite3 handle.
> The SQLite is built only once and with just one version of the code.
>
> Consider following pseudo-code:
>
> In DLL:
>
> BOOL APIENTRY DLLMain()
> {
> }
>
> extern "C" __declspec(dllexport) Database *CreateObject(Database *db)
> {
> db = new SQLiteDatabase();
> db->Connect();
> return db;
> }
>
> In the main application:
>
> mainframe.h:
>
> class MainFrame
> {
> public:
> MainFrame();
> ~MainFrame();
> void ConnectToDb();
> private:
> Database *m_db;
> };
>
> mainframe.cpp:
>
> void MainFrame::ConnectToDb()
> {
> Database *db = NULL;
> LoadLibrary();
> func = GetProcAddress();
> m_db = func( db );
> }
>
> MainFrame::~MainFrame()
> {
> delete m_db; // this is where the crash happens
> }
>
> The pointer address are the same in DLL and main application MainFrame class.
> And as I said the crash occurs when it tries to acquire the mutex lock.
>
> Thank you.
>
>> Peter
>>
>>
>>
>>
>> On 1/24/2016 10:18 PM, Igor Korot wrote:
>>> Hi, ALL,
>>> I have a strange problem.
>>>
>>> I am trying to use sqlite in my program. It has a main application and
>>> couplef DLLs.
>>>
>>> I am getting the connection in one of the DLL, then the pointer is passed
>>> up
>>> to the main application.
>>>
>>> Upon exiting from the application I'm trying to close the connection and
>>> delete all the memory.
>>>
>>> Unfortunately upon exiting the application it crashes inside
>>> sqlite3_mutex_enter().
>>> The comment above the function says:
>>>
>>> [quote]
>>> /*
>>> ** Obtain the mutex p. If some other thread already has the mutex, block
>>> ** until it can be obtained.
>>> */
>>> [/quote]
>>>
>>> The DLL does not start any threads, in fact the application will be 1
>>> thread only.
>>> So is there some compile-time switch I should use to mitigate the issue?
>>>
>>> Moreover I don't understand why am I getting the assertion - there is no
>>> MT
>>> involved.
>>>
>>> Can someone shed some lights?
>>>
>>> Thank you.
>>> ___
>>> sqlite-users mailing list
>>> sqlite-users at mailinglists.sqlite.org
>>> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>>>
>> ___
>> sqlite-users mailing list
>> sqlite-users at mailinglists.sqlite.org
>> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
> ___
> sqlite-users mailing list
> sqlite-users at mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
[sqlite] Running Sqlite under Linux Terminal Server
Hello, Anyone with any experiences with this (good or bad)? Idea to have Sqlite database on server, accessed from a few ultra-thin Terminal clients. Best regards
[sqlite] Bug: Successfully committed transaction rolled back after power failure
On 2016-01-25 16:47, Richard Hipp wrote: > On 1/25/16, Matthias-Christian Ott wrote: >> >> Does this mean that if I use SQLite SQLITE_EXTRA_DURABLE=0, PRAGMA >> journal_mode=DELETE and PRAGMA synchronous=FULL, SQLite could loose a >> transaction that it said to be committed depending on the VFS? > > Sort of. This appears to be true if you are running on QNX and you > lose power (or do a hard reset) shortly after the transaction commits. > It might be the case on other OSes/filesystems but it has never before > been observed. > > This is not new behavior. This is apparently what SQLite has been > doing for 15 years, across quadrillions of transactions on many > billions of devices, and it has never before caused any issues, until > just recently when Mr. Meinlschmidt upgraded to a newer version of > QNX. So it would make sense to add a note that you should check whether your target VFS and target operating environment needs an fsync after a journal commit if you want to use this journal mode. Would it be possible to make SQLITE_EXTRA_DURABLE a pragma? Some GNU/Linux distributions package SQLite and therefore not every application can compile SQLite with different options. > Note especially that this is about durability, not consistency. > SQLite guarantees consistency regardless. People care about > consistency. Durability, not so much. I'm not a MySQL expert, but I > think the historical complaints about MyISAM had more to do with > consistency than with durability, did they not? That's also my understanding. It's unarguably an odd comparison but I was referring to the ACID properties in general. - Matthias-Christian
[sqlite] Pre-compiled x64 dll
Hello, What is the difference between the dll included in: http://sqlite.org/2015/sqlite-dll-win64-x64-3090200.zip And the one included in: http://sqlite.org/2016/sqlite-uap-3100200.vsix Redist/Retail/x64/sqlite3.dll ? The first one works with libffi, the second doesn't. Is there a sqlite-dll-win64-x64-3100200.zip somewhere ? Thanks.
[sqlite] SQLite crashing
Hello Igor,
MainFrame::~MainFrame()
{
delete m_db; // this is where the crash happens
}
I suspect you need to add a "Close" or destroy function to the DLL and
pass the handle back to the DLL to let it get deleted in the DLL
context and not in the context of the caller.
extern "C" __declspec(dllexport) void DestroyObject(Database *db)
{
delete db;
}
It was my impression that each DLL got it's own heap so, memory
allocated inside the DLL needs to be free'd inside the DLL. I use
Sqlite in a static lib in my applications.
I treat memory allocated in DLL's as being owned by the DLL so, I
typically pass it back to the DLL to be cleaned up. It believe it
depends on what run time library you're using though. If you're using
an RTL where all the DLL's end up using a DLL supplied allocator, this
probably isn't an issue. I tend to dynamic load my DLL's so they don't
all use the same allocator.
C
Monday, January 25, 2016, 11:16:57 AM, you wrote:
IK> Hi, Peter,
IK> On Mon, Jan 25, 2016 at 10:50 AM, Peter Aronson
wrote:
>> Igor,
>>
>> You can't safely pass a SQLite handle between different SQL DLLs that way if
>> they're both built with their own copy of the amalgamation (or link to
>> things built with different copies). SQLite uses a handful of global
>> variables, but each DLL has its own copy of each of these global variables
>> and they can and will have different values, which can mess things up. I
>> ran into a version of this problem when I tried to load a 2nd DLL built with
>> its own copy of the sqlite3.c amalgamation. I fixed that by exposing the
>> SQLite3 entrypoints in the first DLL and linking the second DLL against it
>> so there was only one copy of the amalgamation used for that SQLite3 handle.
IK> The SQLite is built only once and with just one version of the code.
IK> Consider following pseudo-code:
IK> In DLL:
IK> BOOL APIENTRY DLLMain()
IK> {
IK> }
IK> extern "C" __declspec(dllexport) Database *CreateObject(Database *db)
IK> {
IK> db = new SQLiteDatabase();
IK> db->Connect();
IK> return db;
IK> }
IK> In the main application:
IK> mainframe.h:
IK> class MainFrame
IK> {
IK> public:
IK> MainFrame();
IK> ~MainFrame();
IK> void ConnectToDb();
IK> private:
IK> Database *m_db;
IK> };
IK> mainframe.cpp:
IK> void MainFrame::ConnectToDb()
IK> {
IK> Database *db = NULL;
IK> LoadLibrary();
IK> func = GetProcAddress();
IK> m_db = func( db );
IK> }
IK> MainFrame::~MainFrame()
IK> {
IK> delete m_db; // this is where the crash happens
IK> }
IK> The pointer address are the same in DLL and main application MainFrame
class.
IK> And as I said the crash occurs when it tries to acquire the mutex lock.
IK> Thank you.
>>
>> Peter
>>
>>
>>
>>
>> On 1/24/2016 10:18 PM, Igor Korot wrote:
>>>
>>> Hi, ALL,
>>> I have a strange problem.
>>>
>>> I am trying to use sqlite in my program. It has a main application and
>>> couplef DLLs.
>>>
>>> I am getting the connection in one of the DLL, then the pointer is passed
>>> up
>>> to the main application.
>>>
>>> Upon exiting from the application I'm trying to close the connection and
>>> delete all the memory.
>>>
>>> Unfortunately upon exiting the application it crashes inside
>>> sqlite3_mutex_enter().
>>> The comment above the function says:
>>>
>>> [quote]
>>> /*
>>> ** Obtain the mutex p. If some other thread already has the mutex, block
>>> ** until it can be obtained.
>>> */
>>> [/quote]
>>>
>>> The DLL does not start any threads, in fact the application will be 1
>>> thread only.
>>> So is there some compile-time switch I should use to mitigate the issue?
>>>
>>> Moreover I don't understand why am I getting the assertion - there is no
>>> MT
>>> involved.
>>>
>>> Can someone shed some lights?
>>>
>>> Thank you.
>>> ___
>>> sqlite-users mailing list
>>> sqlite-users at mailinglists.sqlite.org
>>> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>>>
>>
>> ___
>> sqlite-users mailing list
>> sqlite-users at mailinglists.sqlite.org
>> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
IK> ___
IK> sqlite-users mailing list
IK> sqlite-users at mailinglists.sqlite.org
IK> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
--
Tegmailto:Teg at djii.com
[sqlite] SQLite crashing
On Mon, Jan 25, 2016 at 8:31 AM, Teg wrote:
> Hello Igor,
>
> extern "C" __declspec(dllexport) void DestroyObject(Database *db)
> {
> delete db;
> }
>
> It was my impression that each DLL got it's own heap so, memory
> allocated inside the DLL needs to be free'd inside the DLL. I use
> Sqlite in a static lib in my applications.
>
That's not the general case. DLLs all have the same memory spaces...
although in your next part, it may be true that if you're loading DLLs
that are linked to different runtimes they may have used different
allocators and require different deallocations... but typically (and
in programs with least issues) they will use multi-threaded shared
library runtime so they all use the same allocation functions.
> I treat memory allocated in DLL's as being owned by the DLL so, I
> typically pass it back to the DLL to be cleaned up. It believe it
> depends on what run time library you're using though. If you're using
> an RTL where all the DLL's end up using a DLL supplied allocator, this
> probably isn't an issue. I tend to dynamic load my DLL's so they don't
> all use the same allocator.
>
Are you sure you're not somehow double-freeing the sqlite handle?
Especially at close I've seen exit() end up calling atexit() methods
multiple times in some circumstances...
> C
>
>
>
> Monday, January 25, 2016, 11:16:57 AM, you wrote:
>
> IK> Hi, Peter,
>
> IK> On Mon, Jan 25, 2016 at 10:50 AM, Peter Aronson
> wrote:
>>> Igor,
>>>
>>> You can't safely pass a SQLite handle between different SQL DLLs that way if
>>> they're both built with their own copy of the amalgamation (or link to
>>> things built with different copies). SQLite uses a handful of global
>>> variables, but each DLL has its own copy of each of these global variables
>>> and they can and will have different values, which can mess things up. I
>>> ran into a version of this problem when I tried to load a 2nd DLL built with
>>> its own copy of the sqlite3.c amalgamation. I fixed that by exposing the
>>> SQLite3 entrypoints in the first DLL and linking the second DLL against it
>>> so there was only one copy of the amalgamation used for that SQLite3 handle.
>
> IK> The SQLite is built only once and with just one version of the code.
>
> IK> Consider following pseudo-code:
>
> IK> In DLL:
>
> IK> BOOL APIENTRY DLLMain()
> IK> {
> IK> }
>
> IK> extern "C" __declspec(dllexport) Database *CreateObject(Database *db)
> IK> {
> IK> db = new SQLiteDatabase();
> IK> db->Connect();
> IK> return db;
> IK> }
>
> IK> In the main application:
>
> IK> mainframe.h:
>
> IK> class MainFrame
> IK> {
> IK> public:
> IK> MainFrame();
> IK> ~MainFrame();
> IK> void ConnectToDb();
> IK> private:
> IK> Database *m_db;
> IK> };
>
> IK> mainframe.cpp:
>
> IK> void MainFrame::ConnectToDb()
> IK> {
> IK> Database *db = NULL;
> IK> LoadLibrary();
> IK> func = GetProcAddress();
> IK> m_db = func( db );
> IK> }
>
> IK> MainFrame::~MainFrame()
> IK> {
> IK> delete m_db; // this is where the crash happens
> IK> }
>
> IK> The pointer address are the same in DLL and main application MainFrame
> class.
> IK> And as I said the crash occurs when it tries to acquire the mutex lock.
>
> IK> Thank you.
>
>>>
>>> Peter
>>>
>>>
>>>
>>>
>>> On 1/24/2016 10:18 PM, Igor Korot wrote:
Hi, ALL,
I have a strange problem.
I am trying to use sqlite in my program. It has a main application and
couplef DLLs.
I am getting the connection in one of the DLL, then the pointer is passed
up
to the main application.
Upon exiting from the application I'm trying to close the connection and
delete all the memory.
Unfortunately upon exiting the application it crashes inside
sqlite3_mutex_enter().
The comment above the function says:
[quote]
/*
** Obtain the mutex p. If some other thread already has the mutex, block
** until it can be obtained.
*/
[/quote]
The DLL does not start any threads, in fact the application will be 1
thread only.
So is there some compile-time switch I should use to mitigate the issue?
Moreover I don't understand why am I getting the assertion - there is no
MT
involved.
Can someone shed some lights?
Thank you.
___
sqlite-users mailing list
sqlite-users at mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>>>
>>> ___
>>> sqlite-users mailing list
>>> sqlite-users at mailinglists.sqlite.org
>>> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
> IK> ___
> IK> sqlite-users mailing list
> IK> sqlite-users at mailinglists.sqlite.org
> IK> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
>
>
> -
[sqlite] SQLite crashing
Hi, Teg,
On Mon, Jan 25, 2016 at 11:31 AM, Teg wrote:
> Hello Igor,
>
> MainFrame::~MainFrame()
> {
> delete m_db; // this is where the crash happens
> }
>
> I suspect you need to add a "Close" or destroy function to the DLL and
> pass the handle back to the DLL to let it get deleted in the DLL
> context and not in the context of the caller.
Does not make a difference.
I added that function and call it from the MainFrame destructor.
It still crashed.
Thank you.
>
>
> extern "C" __declspec(dllexport) void DestroyObject(Database *db)
> {
> delete db;
> }
>
> It was my impression that each DLL got it's own heap so, memory
> allocated inside the DLL needs to be free'd inside the DLL. I use
> Sqlite in a static lib in my applications.
>
> I treat memory allocated in DLL's as being owned by the DLL so, I
> typically pass it back to the DLL to be cleaned up. It believe it
> depends on what run time library you're using though. If you're using
> an RTL where all the DLL's end up using a DLL supplied allocator, this
> probably isn't an issue. I tend to dynamic load my DLL's so they don't
> all use the same allocator.
>
> C
>
>
>
> Monday, January 25, 2016, 11:16:57 AM, you wrote:
>
> IK> Hi, Peter,
>
> IK> On Mon, Jan 25, 2016 at 10:50 AM, Peter Aronson
> wrote:
>>> Igor,
>>>
>>> You can't safely pass a SQLite handle between different SQL DLLs that way if
>>> they're both built with their own copy of the amalgamation (or link to
>>> things built with different copies). SQLite uses a handful of global
>>> variables, but each DLL has its own copy of each of these global variables
>>> and they can and will have different values, which can mess things up. I
>>> ran into a version of this problem when I tried to load a 2nd DLL built with
>>> its own copy of the sqlite3.c amalgamation. I fixed that by exposing the
>>> SQLite3 entrypoints in the first DLL and linking the second DLL against it
>>> so there was only one copy of the amalgamation used for that SQLite3 handle.
>
> IK> The SQLite is built only once and with just one version of the code.
>
> IK> Consider following pseudo-code:
>
> IK> In DLL:
>
> IK> BOOL APIENTRY DLLMain()
> IK> {
> IK> }
>
> IK> extern "C" __declspec(dllexport) Database *CreateObject(Database *db)
> IK> {
> IK> db = new SQLiteDatabase();
> IK> db->Connect();
> IK> return db;
> IK> }
>
> IK> In the main application:
>
> IK> mainframe.h:
>
> IK> class MainFrame
> IK> {
> IK> public:
> IK> MainFrame();
> IK> ~MainFrame();
> IK> void ConnectToDb();
> IK> private:
> IK> Database *m_db;
> IK> };
>
> IK> mainframe.cpp:
>
> IK> void MainFrame::ConnectToDb()
> IK> {
> IK> Database *db = NULL;
> IK> LoadLibrary();
> IK> func = GetProcAddress();
> IK> m_db = func( db );
> IK> }
>
> IK> MainFrame::~MainFrame()
> IK> {
> IK> delete m_db; // this is where the crash happens
> IK> }
>
> IK> The pointer address are the same in DLL and main application MainFrame
> class.
> IK> And as I said the crash occurs when it tries to acquire the mutex lock.
>
> IK> Thank you.
>
>>>
>>> Peter
>>>
>>>
>>>
>>>
>>> On 1/24/2016 10:18 PM, Igor Korot wrote:
Hi, ALL,
I have a strange problem.
I am trying to use sqlite in my program. It has a main application and
couplef DLLs.
I am getting the connection in one of the DLL, then the pointer is passed
up
to the main application.
Upon exiting from the application I'm trying to close the connection and
delete all the memory.
Unfortunately upon exiting the application it crashes inside
sqlite3_mutex_enter().
The comment above the function says:
[quote]
/*
** Obtain the mutex p. If some other thread already has the mutex, block
** until it can be obtained.
*/
[/quote]
The DLL does not start any threads, in fact the application will be 1
thread only.
So is there some compile-time switch I should use to mitigate the issue?
Moreover I don't understand why am I getting the assertion - there is no
MT
involved.
Can someone shed some lights?
Thank you.
___
sqlite-users mailing list
sqlite-users at mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>>>
>>> ___
>>> sqlite-users mailing list
>>> sqlite-users at mailinglists.sqlite.org
>>> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
> IK> ___
> IK> sqlite-users mailing list
> IK> sqlite-users at mailinglists.sqlite.org
> IK> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
>
>
> --
> Tegmailto:Teg at djii.com
>
[sqlite] SQLite crashing
Hi, Peter,
On Mon, Jan 25, 2016 at 11:37 AM, Peter Aronson wrote:
> How are you building the DLL and the executable? Are they both including a
> copy of sqlite3.obj? Or are you using a sqlite3.dll?
I'm building SQLite from source.
The DLL and executable both do not reference sqlite3.{obj,dll}.
They rather link to the .lib file which has all exported functions.
Thank you.
>
> Peter
>
>
> On 1/25/2016 9:16 AM, Igor Korot wrote:
>>
>> Hi, Peter,
>>
>> On Mon, Jan 25, 2016 at 10:50 AM, Peter Aronson wrote:
>>>
>>> Igor,
>>>
>>> You can't safely pass a SQLite handle between different SQL DLLs that way
>>> if
>>> they're both built with their own copy of the amalgamation (or link to
>>> things built with different copies). SQLite uses a handful of global
>>> variables, but each DLL has its own copy of each of these global
>>> variables
>>> and they can and will have different values, which can mess things up. I
>>> ran into a version of this problem when I tried to load a 2nd DLL built
>>> with
>>> its own copy of the sqlite3.c amalgamation. I fixed that by exposing the
>>> SQLite3 entrypoints in the first DLL and linking the second DLL against
>>> it
>>> so there was only one copy of the amalgamation used for that SQLite3
>>> handle.
>>
>> The SQLite is built only once and with just one version of the code.
>>
>> Consider following pseudo-code:
>>
>> In DLL:
>>
>> BOOL APIENTRY DLLMain()
>> {
>> }
>>
>> extern "C" __declspec(dllexport) Database *CreateObject(Database *db)
>> {
>> db = new SQLiteDatabase();
>> db->Connect();
>> return db;
>> }
>>
>> In the main application:
>>
>> mainframe.h:
>>
>> class MainFrame
>> {
>> public:
>> MainFrame();
>> ~MainFrame();
>> void ConnectToDb();
>> private:
>> Database *m_db;
>> };
>>
>> mainframe.cpp:
>>
>> void MainFrame::ConnectToDb()
>> {
>> Database *db = NULL;
>> LoadLibrary();
>> func = GetProcAddress();
>> m_db = func( db );
>> }
>>
>> MainFrame::~MainFrame()
>> {
>> delete m_db; // this is where the crash happens
>> }
>>
>> The pointer address are the same in DLL and main application MainFrame
>> class.
>> And as I said the crash occurs when it tries to acquire the mutex lock.
>>
>> Thank you.
>>
>>> Peter
>>>
>>>
>>>
>>>
>>> On 1/24/2016 10:18 PM, Igor Korot wrote:
Hi, ALL,
I have a strange problem.
I am trying to use sqlite in my program. It has a main application and
couplef DLLs.
I am getting the connection in one of the DLL, then the pointer is
passed
up
to the main application.
Upon exiting from the application I'm trying to close the connection and
delete all the memory.
Unfortunately upon exiting the application it crashes inside
sqlite3_mutex_enter().
The comment above the function says:
[quote]
/*
** Obtain the mutex p. If some other thread already has the mutex, block
** until it can be obtained.
*/
[/quote]
The DLL does not start any threads, in fact the application will be 1
thread only.
So is there some compile-time switch I should use to mitigate the issue?
Moreover I don't understand why am I getting the assertion - there is no
MT
involved.
Can someone shed some lights?
Thank you.
___
sqlite-users mailing list
sqlite-users at mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>>> ___
>>> sqlite-users mailing list
>>> sqlite-users at mailinglists.sqlite.org
>>> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>>
>> ___
>> sqlite-users mailing list
>> sqlite-users at mailinglists.sqlite.org
>> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>>
>
> ___
> sqlite-users mailing list
> sqlite-users at mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
[sqlite] Running Sqlite under Linux Terminal Server
On 25 Jan 2016, at 5:02pm, Bernard McNeill wrote: > Idea to have Sqlite database on server, accessed from a few ultra-thin > Terminal clients. SQLite is not a server/client system. You would have to write your own server/client system. Simon.
[sqlite] Running Sqlite under Linux Terminal Server
On Mon, Jan 25, 2016 at 12:22 PM, Simon Slavin wrote: > > On 25 Jan 2016, at 5:02pm, Bernard McNeill wrote: > > > Idea to have Sqlite database on server, accessed from a few ultra-thin > > Terminal clients. > ?I would guess that this harks back to the X terminal days. That is, a very small machine which basically just drives a keyboard, mouse, and display; with all the real work & data being on the remote server. > > SQLite is not a server/client system. You would have to write your own > server/client system. > ?I was thinking that too. Of course, the OP could write his own C/S database server using SQLite as the core, if there was a good need. But, personally, I'd go with a database which is designed and tested for this, myself. And, for me, that would be PostgreSQL. It is "heavy", but it can do amazing things. And I just personally like it a bit better than MariaDB/MySQL.? > > Simon. > -- Werner Heisenberg is driving down the autobahn. A police officer pulls him over. The officer says, "Excuse me, sir, do you know how fast you were going?" "No," replies Dr. Heisenberg, "but I know where I am." Computer Science is the only discipline in which we view adding a new wing to a building as being maintenance -- Jim Horning Schrodinger's backup: The condition of any backup is unknown until a restore is attempted. He's about as useful as a wax frying pan. Maranatha! <>< John McKown
[sqlite] Running Sqlite under Linux Terminal Server
On 2016/01/25 7:02 PM, Bernard McNeill wrote: > Hello, > > Anyone with any experiences with this (good or bad)? > > Idea to have Sqlite database on server, accessed from a few ultra-thin > Terminal clients. Terminal clients? You mean the kind where the terminal client is basically just peripherals to a session that runs on an actual server? If this is the case and the app runs on the session on the server - then the DB files would be local to the server and the sharing would work as planned. If however the clients somehow run this software (inside the thin client, or any client) and/or you plan any future expansion where it will become a client-run system, then they will gain file access via the network file system, in which case you most certainly do not wish to use an SQLite DB. (It's still ok if you only read from the DB, but bad if you try to write anything since network file locking doesn't work). Might I suggest MariaDB/MySQL or PostGres for this purpose? There exists an SQLite network server system, but it is commercial I believe.
[sqlite] Updating a filed in table a with contents from table b
anyone?
[sqlite] SQLite crashing
Hi, Adam,
On Mon, Jan 25, 2016 at 11:27 AM, Adam Devita wrote:
> Where do you pass to the dll something that goes to sqlite3_close(db); ?
> ( https://www.sqlite.org/c3ref/close.html )
> When that happens, does m_db get set to NULL (or now refers to memory
> that is now NULL)
> Do you check for m_db == NULL before deleting it?
SQLiteDatabase class is just a wrapper around the SQLite interface.
The constructor is empty, but in the Connect() function of the class I call
sqlite3_open().
And here is the code that you are asking for:
void MainFrame::ConnectToDb()
{
Database *db = NULL;
LoadLibrary();
func = GetProcAddress();
m_db = func( db );
}
Also, in C++ delete'ing NULL is perfectly normal operation.
Thank you.
>
> regards,
> Adam DeVita
>
> On Mon, Jan 25, 2016 at 11:16 AM, Igor Korot wrote:
>> Hi, Peter,
>>
>> On Mon, Jan 25, 2016 at 10:50 AM, Peter Aronson wrote:
>>> Igor,
>>>
>>> You can't safely pass a SQLite handle between different SQL DLLs that way if
>>> they're both built with their own copy of the amalgamation (or link to
>>> things built with different copies). SQLite uses a handful of global
>>> variables, but each DLL has its own copy of each of these global variables
>>> and they can and will have different values, which can mess things up. I
>>> ran into a version of this problem when I tried to load a 2nd DLL built with
>>> its own copy of the sqlite3.c amalgamation. I fixed that by exposing the
>>> SQLite3 entrypoints in the first DLL and linking the second DLL against it
>>> so there was only one copy of the amalgamation used for that SQLite3 handle.
>>
>> The SQLite is built only once and with just one version of the code.
>>
>> Consider following pseudo-code:
>>
>> In DLL:
>>
>> BOOL APIENTRY DLLMain()
>> {
>> }
>>
>> extern "C" __declspec(dllexport) Database *CreateObject(Database *db)
>> {
>> db = new SQLiteDatabase();
>> db->Connect();
>> return db;
>> }
>>
>> In the main application:
>>
>> mainframe.h:
>>
>> class MainFrame
>> {
>> public:
>> MainFrame();
>> ~MainFrame();
>> void ConnectToDb();
>> private:
>> Database *m_db;
>> };
>>
>> mainframe.cpp:
>>
>> void MainFrame::ConnectToDb()
>> {
>> Database *db = NULL;
>> LoadLibrary();
>> func = GetProcAddress();
>> m_db = func( db );
>> }
>>
>> MainFrame::~MainFrame()
>> {
>> delete m_db; // this is where the crash happens
>> }
>>
>> The pointer address are the same in DLL and main application MainFrame class.
>> And as I said the crash occurs when it tries to acquire the mutex lock.
>>
>> Thank you.
>>
>>>
>>> Peter
>>>
>>>
>>>
>>>
>>> On 1/24/2016 10:18 PM, Igor Korot wrote:
Hi, ALL,
I have a strange problem.
I am trying to use sqlite in my program. It has a main application and
couplef DLLs.
I am getting the connection in one of the DLL, then the pointer is passed
up
to the main application.
Upon exiting from the application I'm trying to close the connection and
delete all the memory.
Unfortunately upon exiting the application it crashes inside
sqlite3_mutex_enter().
The comment above the function says:
[quote]
/*
** Obtain the mutex p. If some other thread already has the mutex, block
** until it can be obtained.
*/
[/quote]
The DLL does not start any threads, in fact the application will be 1
thread only.
So is there some compile-time switch I should use to mitigate the issue?
Moreover I don't understand why am I getting the assertion - there is no
MT
involved.
Can someone shed some lights?
Thank you.
___
sqlite-users mailing list
sqlite-users at mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>>>
>>> ___
>>> sqlite-users mailing list
>>> sqlite-users at mailinglists.sqlite.org
>>> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>> ___
>> sqlite-users mailing list
>> sqlite-users at mailinglists.sqlite.org
>> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
>
>
> --
> --
> VerifEye Technologies Inc.
> 151 Whitehall Dr. Unit 2
> Markham, ON
> L3R 9T1
> ___
> sqlite-users mailing list
> sqlite-users at mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
[sqlite] SQLite crashing
Hi Igor,
I don't think you understood what I was trying to get at. Please
allow me to rephrase: There isn't enough information about how things
are being cleaned up to point out a problem, only make general
suggestions about good practice.
This is why I was asking about where you would ensure you are properly
shutting down the database before deleting your wrapper. I had
expected to see something of the form:
//private: sqlite3 *db;
BOOL SomeWrapperClass::Closedb()
{
if(db==NULL){
b_sqlite_db_open = false;//Ensure state variable of wrapper is correct
return TRUE; // db wasn't open, so Closed is OK
}
int ret=sqlite3_close(db);
if (ret==SQLITE_OK){
db = NULL;
b_sqlite_db_open = false;
return TRUE; //properly closed my db connection, no errors
}
return FALSE; //if we got here, there is an error; break-point hit
; What didn't get finalized? How did this happen?
}
I would have expect you to, before delete m_db to do something like
if(m_db != NULL) {
if(!m_db->Closedb() ){ //assumes that queries have already been
finalized ;
;//error handle code, break point, how did we fail to close?
}
delete m_db;
}
or a method of your dll that is some exported DestroyObject(Database *db) ;
or at least tell us that you are ensuring sqlite_close is called via
the destructor of m_db;
While in C++ one may choose to not check if the thing being deleted is
NULL before doing it, it is a clutter/style thing. For debugging
purposes finding out that the thing that should have been null isn't
as expected. It can often lead to one saying "hey, that should have
got deleted already!... oh somehow delete x instead of safe_delete(x)
got used so while it got deleted earlier and x should have been null
at this point"
Are you unit testing the trivial cases?
Create+Destroy
Create+Connect+Destroy
Create+Connect+DoBasicQuery+Destroy
regards,
Adam
On Mon, Jan 25, 2016 at 2:02 PM, Igor Korot wrote:
> Hi, Adam,
>
> On Mon, Jan 25, 2016 at 11:27 AM, Adam Devita wrote:
>> Where do you pass to the dll something that goes to sqlite3_close(db); ?
>> ( https://www.sqlite.org/c3ref/close.html )
>> When that happens, does m_db get set to NULL (or now refers to memory
>> that is now NULL)
>> Do you check for m_db == NULL before deleting it?
>
> SQLiteDatabase class is just a wrapper around the SQLite interface.
> The constructor is empty, but in the Connect() function of the class I call
>
> sqlite3_open().
>
> And here is the code that you are asking for:
>
> void MainFrame::ConnectToDb()
> {
> Database *db = NULL;
> LoadLibrary();
> func = GetProcAddress();
> m_db = func( db );
> }
>
> Also, in C++ delete'ing NULL is perfectly normal operation.
>
> Thank you.
>
>>
>> regards,
>> Adam DeVita
>>
>> On Mon, Jan 25, 2016 at 11:16 AM, Igor Korot wrote:
>>> Hi, Peter,
>>>
>>> On Mon, Jan 25, 2016 at 10:50 AM, Peter Aronson
>>> wrote:
Igor,
You can't safely pass a SQLite handle between different SQL DLLs that way
if
they're both built with their own copy of the amalgamation (or link to
things built with different copies). SQLite uses a handful of global
variables, but each DLL has its own copy of each of these global variables
and they can and will have different values, which can mess things up. I
ran into a version of this problem when I tried to load a 2nd DLL built
with
its own copy of the sqlite3.c amalgamation. I fixed that by exposing the
SQLite3 entrypoints in the first DLL and linking the second DLL against it
so there was only one copy of the amalgamation used for that SQLite3
handle.
>>>
>>> The SQLite is built only once and with just one version of the code.
>>>
>>> Consider following pseudo-code:
>>>
>>> In DLL:
>>>
>>> BOOL APIENTRY DLLMain()
>>> {
>>> }
>>>
>>> extern "C" __declspec(dllexport) Database *CreateObject(Database *db)
>>> {
>>> db = new SQLiteDatabase();
>>> db->Connect();
>>> return db;
>>> }
>>>
>>> In the main application:
>>>
>>> mainframe.h:
>>>
>>> class MainFrame
>>> {
>>> public:
>>> MainFrame();
>>> ~MainFrame();
>>> void ConnectToDb();
>>> private:
>>> Database *m_db;
>>> };
>>>
>>> mainframe.cpp:
>>>
>>> void MainFrame::ConnectToDb()
>>> {
>>> Database *db = NULL;
>>> LoadLibrary();
>>> func = GetProcAddress();
>>> m_db = func( db );
>>> }
>>>
>>> MainFrame::~MainFrame()
>>> {
>>> delete m_db; // this is where the crash happens
>>> }
>>>
>>> The pointer address are the same in DLL and main application MainFrame
>>> class.
>>> And as I said the crash occurs when it tries to acquire the mutex lock.
>>>
>>> Thank you.
>>>
Peter
On 1/24/2016 10:18 PM, Igor Korot wrote:
>
> Hi, ALL,
> I have a strange problem.
>
> I am trying to use sqlite in my program. It has a main application and
> couplef DLLs.
>
> I am getti
[sqlite] Bug: Successfully committed transaction rolled back after power failure
Matthias-Christian Ott wrote: > On 2016-01-25 16:47, Richard Hipp wrote: >> On 1/25/16, Matthias-Christian Ott wrote: >>> >>> Does this mean that if I use SQLite SQLITE_EXTRA_DURABLE=0, PRAGMA >>> journal_mode=DELETE and PRAGMA synchronous=FULL, SQLite could loose a >>> transaction that it said to be committed depending on the VFS? >> >> Sort of. This appears to be true if you are running on QNX and you >> lose power (or do a hard reset) shortly after the transaction commits. >> It might be the case on other OSes/filesystems but it has never before >> been observed. >> >> This is not new behavior. This is apparently what SQLite has been >> doing for 15 years, across quadrillions of transactions on many >> billions of devices, and it has never before caused any issues, until >> just recently when Mr. Meinlschmidt upgraded to a newer version of >> QNX. > > So it would make sense to add a note that you should check whether your > target VFS and target operating environment needs an fsync after a > journal commit if you want to use this journal mode. Would it be > possible to make SQLITE_EXTRA_DURABLE a pragma? Some GNU/Linux > distributions package SQLite and therefore not every application can > compile SQLite with different options. The question isn't just whether "an fsync is needed" with journal mode - the question is *which* fsync is needed? The issue here is that file creation/deletion/rename ops require an fsync *on the containing directory*. This is actually quite an unusual requirement; on older Unix systems you couldn't even *open* a directory, let alone obtain write access to it or fsync it. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
[sqlite] Bug: Successfully committed transaction rolled back after power failure
On 1/25/16, Howard Chu wrote: > > This is actually quite an unusual requirement; on older Unix systems you > couldn't even *open* a directory, let alone obtain write access to it or > fsync it. Yeah. When the SQLITE_DISABLE_DIRSYNC compile-time option is present, we disable the directory sync logic for this reason. Some unixes (HP/UX) require -DSQLITE_DISABLE_DIRSYNC in order to work. But Linux, MacOS, and *BSD all work without it, so I thought I'd just not bring that up... -- D. Richard Hipp drh at sqlite.org
[sqlite] SQLite crashing
Hi, Adam,
On Mon, Jan 25, 2016 at 2:46 PM, Adam Devita wrote:
> Hi Igor,
> I don't think you understood what I was trying to get at. Please
> allow me to rephrase: There isn't enough information about how things
> are being cleaned up to point out a problem, only make general
> suggestions about good practice.
>
> This is why I was asking about where you would ensure you are properly
> shutting down the database before deleting your wrapper. I had
> expected to see something of the form:
>
>
> //private: sqlite3 *db;
> BOOL SomeWrapperClass::Closedb()
> {
> if(db==NULL){
> b_sqlite_db_open = false;//Ensure state variable of wrapper is correct
> return TRUE; // db wasn't open, so Closed is OK
> }
>
> int ret=sqlite3_close(db);
> if (ret==SQLITE_OK){
> db = NULL;
> b_sqlite_db_open = false;
> return TRUE; //properly closed my db connection, no errors
> }
> return FALSE; //if we got here, there is an error; break-point hit
> ; What didn't get finalized? How did this happen?
> }
>
> I would have expect you to, before delete m_db to do something like
> if(m_db != NULL) {
> if(!m_db->Closedb() ){ //assumes that queries have already been
> finalized ;
> ;//error handle code, break point, how did we fail to close?
>}
> delete m_db;
> }
>
> or a method of your dll that is some exported DestroyObject(Database *db) ;
>
> or at least tell us that you are ensuring sqlite_close is called via
> the destructor of m_db;
Yes, sqlite3_close() _is_ called from the m_db destructor.
Moreover, for now I am just opening the db thru the loading the DLL
dynamically and
calling the DLL function which creates an object and connects to the db:
SQLiteDatabase::SQLiteDatabase()
{
}
SQLiteDatabase::~SQLiteDatabase()
{
sqlite3_close();
}
int SQLiteDatabase::Connect()
{
int result = sqlite3_open();
if( result != SQLITE_OK )
{
// handling error
}
}
In the MainFrame:
void MainFrame::ConnectToDb()
{
Database *db = NULL;
LoadLibrary();
func = GetProcAddress( "CreateObject" );
m_db = func( db );
}
In the DLL:
extern "C" __declspec(dllexport) Database *CreateObject(Database *db)
{
db = new SQLiteDatabase();
int res = db->Connect();
if( res )
delete db;
return db;
}
and then exiting the main application, i.e. closing the connection.
No queries are issued for the database.
Moreover all pointers are the same: from the point I call new in DLL'
CreateObject(), then
in ConnectToDB() in MainFrame' ConnectToDb() function and finally in
the MainFrame' destructor.
Also it does not matter whether I'm trying to do "delete m_db" in the
MainFrame destructor or try to
close the connection from an additional exported function - it still crashes.
Thank you.
>
> While in C++ one may choose to not check if the thing being deleted is
> NULL before doing it, it is a clutter/style thing. For debugging
> purposes finding out that the thing that should have been null isn't
> as expected. It can often lead to one saying "hey, that should have
> got deleted already!... oh somehow delete x instead of safe_delete(x)
> got used so while it got deleted earlier and x should have been null
> at this point"
>
> Are you unit testing the trivial cases?
> Create+Destroy
> Create+Connect+Destroy
> Create+Connect+DoBasicQuery+Destroy
>
>
> regards,
> Adam
>
> On Mon, Jan 25, 2016 at 2:02 PM, Igor Korot wrote:
>> Hi, Adam,
>>
>> On Mon, Jan 25, 2016 at 11:27 AM, Adam Devita
>> wrote:
>>> Where do you pass to the dll something that goes to sqlite3_close(db); ?
>>> ( https://www.sqlite.org/c3ref/close.html )
>>> When that happens, does m_db get set to NULL (or now refers to memory
>>> that is now NULL)
>>> Do you check for m_db == NULL before deleting it?
>>
>> SQLiteDatabase class is just a wrapper around the SQLite interface.
>> The constructor is empty, but in the Connect() function of the class I call
>>
>> sqlite3_open().
>>
>> And here is the code that you are asking for:
>>
>> void MainFrame::ConnectToDb()
>> {
>> Database *db = NULL;
>> LoadLibrary();
>> func = GetProcAddress();
>> m_db = func( db );
>> }
>>
>> Also, in C++ delete'ing NULL is perfectly normal operation.
>>
>> Thank you.
>>
>>>
>>> regards,
>>> Adam DeVita
>>>
>>> On Mon, Jan 25, 2016 at 11:16 AM, Igor Korot wrote:
Hi, Peter,
On Mon, Jan 25, 2016 at 10:50 AM, Peter Aronson
wrote:
> Igor,
>
> You can't safely pass a SQLite handle between different SQL DLLs that way
> if
> they're both built with their own copy of the amalgamation (or link to
> things built with different copies). SQLite uses a handful of global
> variables, but each DLL has its own copy of each of these global variables
> and they can and will have different values, which can mess things up. I
> ran into a version of this problem when I tried to load a 2nd DLL built
> with
> its own copy of
[sqlite] Updating a filed in table a with contents from table b
On 2016/01/24 10:15 PM, audio muze wrote: > I have a large table with ~350k records for which I'm in the process of > standardising data. 350k records is not exactly a "large" table, A table scan would complete in a couple of seconds on a normal system. 350 million rows are more substantial and would require more care. The rest of your explanation is a bit hard to follow, but if I understand correct, You have a field (albumartist) which you have somehow corrected from the original by creating a new table (albumartists) with the old and new fields. Once the field has been corrected and saved (in newalbumartist) you would like to update these new values to the original database but fear messing things up (a good fear) and so would like to know whether it is safe to run the query, which you've already made and which seemingly works, on the original DB without making mistakes and you want us to tell you whether we see any problems that might bite you looking at your update query. Right? I have some weird news for you - We don't really know, nobody here is above making mistakes, so what we all do is make backups and then try the big updates and restore where necessary, until it works. There are however better ways to do that update more directly, but first things first: You need to back-up the database. Luckily in SQLite you can simply close all open connections to it and copy the files to another folder. 350k records should be a small file. You may optionally use a DB Manager to back it up - I am not familiar with SQLite Studio but I'm sure they have it. If not, you could try SQLitespeed (which definitely has it and you can back up multiple versions and test-run scripts before committing). Your update query seems fine - if I may add one comment, please qualify all the field references so you can't refer a wrong field that might end up in scope. I doubt your query will have that problem, but it is a safer practice. Something like this perhaps: UPDATE audio SET audio.albumartist = ( SELECT A.newalbumartist FROM albumartists AS A WHERE A.albumartist = audio.albumartist ) WHERE audio.albumartist IN (SELECT B.albumartist FROM albumartists AS B); Should do the trick - but find a way to check after the run completes, and restore a backup if needed and retry or ask again if things seem awry. Cheers, Ryan
[sqlite] SQLite crashing
Hi, Teg,
On Mon, Jan 25, 2016 at 4:51 PM, Teg wrote:
> Hello Igor,
>
> Then I'd note the address of the object (make a copy of the pointer
> right as it's allocated) and then verify that the address you're
> deleting is the same as the address that was allocated. I've verify
> that the correct calling convention is being used throughout too.
Yes, pointers (addresses) are the same.
>
>
> I'd single step into the destructor and see what's actually happening to
> make it crash. I'd pay attention to the "this" pointer and see if it
> makes sense.
>
> Assuming you're using visual studio and have the source to the wrapper
> if should be easy to simply step in and see exactly what line of code
> makes it crash.
Yes, MSVC 2010 and the code is mine. ;-)
Thank you.
>
> C
>
>
> Monday, January 25, 2016, 1:00:53 PM, you wrote:
>
> IK> Hi, Teg,
>
> IK> On Mon, Jan 25, 2016 at 11:31 AM, Teg wrote:
>>> Hello Igor,
>>>
>>> MainFrame::~MainFrame()
>>> {
>>> delete m_db; // this is where the crash happens
>>> }
>>>
>>> I suspect you need to add a "Close" or destroy function to the DLL and
>>> pass the handle back to the DLL to let it get deleted in the DLL
>>> context and not in the context of the caller.
>
> IK> Does not make a difference.
> IK> I added that function and call it from the MainFrame destructor.
>
> IK> It still crashed.
>
> IK> Thank you.
>
>>>
>>>
>>> extern "C" __declspec(dllexport) void DestroyObject(Database *db)
>>> {
>>> delete db;
>>> }
>>>
>>> It was my impression that each DLL got it's own heap so, memory
>>> allocated inside the DLL needs to be free'd inside the DLL. I use
>>> Sqlite in a static lib in my applications.
>>>
>>> I treat memory allocated in DLL's as being owned by the DLL so, I
>>> typically pass it back to the DLL to be cleaned up. It believe it
>>> depends on what run time library you're using though. If you're using
>>> an RTL where all the DLL's end up using a DLL supplied allocator, this
>>> probably isn't an issue. I tend to dynamic load my DLL's so they don't
>>> all use the same allocator.
>>>
>>> C
>>>
>>>
>>>
>>> Monday, January 25, 2016, 11:16:57 AM, you wrote:
>>>
>>> IK> Hi, Peter,
>>>
>>> IK> On Mon, Jan 25, 2016 at 10:50 AM, Peter Aronson
>>> wrote:
> Igor,
>
> You can't safely pass a SQLite handle between different SQL DLLs that way
> if
> they're both built with their own copy of the amalgamation (or link to
> things built with different copies). SQLite uses a handful of global
> variables, but each DLL has its own copy of each of these global variables
> and they can and will have different values, which can mess things up. I
> ran into a version of this problem when I tried to load a 2nd DLL built
> with
> its own copy of the sqlite3.c amalgamation. I fixed that by exposing the
> SQLite3 entrypoints in the first DLL and linking the second DLL against it
> so there was only one copy of the amalgamation used for that SQLite3
> handle.
>>>
>>> IK> The SQLite is built only once and with just one version of the code.
>>>
>>> IK> Consider following pseudo-code:
>>>
>>> IK> In DLL:
>>>
>>> IK> BOOL APIENTRY DLLMain()
>>> IK> {
>>> IK> }
>>>
>>> IK> extern "C" __declspec(dllexport) Database *CreateObject(Database *db)
>>> IK> {
>>> IK> db = new SQLiteDatabase();
>>> IK> db->Connect();
>>> IK> return db;
>>> IK> }
>>>
>>> IK> In the main application:
>>>
>>> IK> mainframe.h:
>>>
>>> IK> class MainFrame
>>> IK> {
>>> IK> public:
>>> IK> MainFrame();
>>> IK> ~MainFrame();
>>> IK> void ConnectToDb();
>>> IK> private:
>>> IK> Database *m_db;
>>> IK> };
>>>
>>> IK> mainframe.cpp:
>>>
>>> IK> void MainFrame::ConnectToDb()
>>> IK> {
>>> IK> Database *db = NULL;
>>> IK> LoadLibrary();
>>> IK> func = GetProcAddress();
>>> IK> m_db = func( db );
>>> IK> }
>>>
>>> IK> MainFrame::~MainFrame()
>>> IK> {
>>> IK> delete m_db; // this is where the crash happens
>>> IK> }
>>>
>>> IK> The pointer address are the same in DLL and main application MainFrame
>>> class.
>>> IK> And as I said the crash occurs when it tries to acquire the mutex lock.
>>>
>>> IK> Thank you.
>>>
>
> Peter
>
>
>
>
> On 1/24/2016 10:18 PM, Igor Korot wrote:
>>
>> Hi, ALL,
>> I have a strange problem.
>>
>> I am trying to use sqlite in my program. It has a main application and
>> couplef DLLs.
>>
>> I am getting the connection in one of the DLL, then the pointer is passed
>> up
>> to the main application.
>>
>> Upon exiting from the application I'm trying to close the connection and
>> delete all the memory.
>>
>> Unfortunately upon exiting the application it crashes inside
>> sqlite3_mutex_enter().
>> The comment above the function says:
>>
>> [quote]
>> /*
>> ** Obtain the mutex p. If some other thread already has the mutex, block
>> ** until it c
[sqlite] SQLite crashing
Hi, Teg,
On Mon, Jan 25, 2016 at 5:21 PM, Igor Korot wrote:
> Hi, Teg,
>
> On Mon, Jan 25, 2016 at 4:51 PM, Teg wrote:
>> Hello Igor,
>>
>> Then I'd note the address of the object (make a copy of the pointer
>> right as it's allocated) and then verify that the address you're
>> deleting is the same as the address that was allocated. I've verify
>> that the correct calling convention is being used throughout too.
>
> Yes, pointers (addresses) are the same.
>
>>
>>
>> I'd single step into the destructor and see what's actually happening to
>> make it crash. I'd pay attention to the "this" pointer and see if it
>> makes sense.
I just ran the program under the debugger step-by-step and everything
looks "normal".
All addresses are the same. and the opening db executes successfully.
The stack trace look like this:
>sqlite.dll!sqlite3_mutex_enter(sqlite3_mutex * p) Line 19996 + 0xc bytes
> C
sqlite.dll!sqlite3Close(sqlite3 * db, int forceZombie) Line 726
+ 0xc bytesC
sqlite.dll!sqlite3_close(sqlite3 * db) Line 772 + 0xe bytesC
sqlite.dll!SQLiteDatabase::Disconnect(std::vector,std::allocator
>,std::allocator,std::allocator
> > > & errorMsg) Line 49 + 0xc bytesC++
dialogs.dll!DisconnectFromDb(Database * db) Line 108 + 0x13 bytesC++
docview.exe!MainFrame::~MainFrame() Line 73 + 0xf bytesC++
The DisconnectFromDb() function is a function from DLL. It calls
SQLiteDatabase::Disconnect().
Then everything is sqlite.
Any idea of what is going on?
Thank you.
>>
>> Assuming you're using visual studio and have the source to the wrapper
>> if should be easy to simply step in and see exactly what line of code
>> makes it crash.
>
> Yes, MSVC 2010 and the code is mine. ;-)
>
> Thank you.
>>
>> C
>>
>>
>> Monday, January 25, 2016, 1:00:53 PM, you wrote:
>>
>> IK> Hi, Teg,
>>
>> IK> On Mon, Jan 25, 2016 at 11:31 AM, Teg wrote:
Hello Igor,
MainFrame::~MainFrame()
{
delete m_db; // this is where the crash happens
}
I suspect you need to add a "Close" or destroy function to the DLL and
pass the handle back to the DLL to let it get deleted in the DLL
context and not in the context of the caller.
>>
>> IK> Does not make a difference.
>> IK> I added that function and call it from the MainFrame destructor.
>>
>> IK> It still crashed.
>>
>> IK> Thank you.
>>
extern "C" __declspec(dllexport) void DestroyObject(Database *db)
{
delete db;
}
It was my impression that each DLL got it's own heap so, memory
allocated inside the DLL needs to be free'd inside the DLL. I use
Sqlite in a static lib in my applications.
I treat memory allocated in DLL's as being owned by the DLL so, I
typically pass it back to the DLL to be cleaned up. It believe it
depends on what run time library you're using though. If you're using
an RTL where all the DLL's end up using a DLL supplied allocator, this
probably isn't an issue. I tend to dynamic load my DLL's so they don't
all use the same allocator.
C
Monday, January 25, 2016, 11:16:57 AM, you wrote:
IK> Hi, Peter,
IK> On Mon, Jan 25, 2016 at 10:50 AM, Peter Aronson
wrote:
>> Igor,
>>
>> You can't safely pass a SQLite handle between different SQL DLLs that
>> way if
>> they're both built with their own copy of the amalgamation (or link to
>> things built with different copies). SQLite uses a handful of global
>> variables, but each DLL has its own copy of each of these global
>> variables
>> and they can and will have different values, which can mess things up. I
>> ran into a version of this problem when I tried to load a 2nd DLL built
>> with
>> its own copy of the sqlite3.c amalgamation. I fixed that by exposing the
>> SQLite3 entrypoints in the first DLL and linking the second DLL against
>> it
>> so there was only one copy of the amalgamation used for that SQLite3
>> handle.
IK> The SQLite is built only once and with just one version of the code.
IK> Consider following pseudo-code:
IK> In DLL:
IK> BOOL APIENTRY DLLMain()
IK> {
IK> }
IK> extern "C" __declspec(dllexport) Database *CreateObject(Database *db)
IK> {
IK> db = new SQLiteDatabase();
IK> db->Connect();
IK> return db;
IK> }
IK> In the main application:
IK> mainframe.h:
IK> class MainFrame
IK> {
IK> public:
IK> MainFrame();
IK> ~MainFrame();
IK> void ConnectToDb();
IK> private:
IK> Database *m_db;
IK> };
IK> mainframe.cpp:
IK> void MainFrame::ConnectToDb()
IK> {
IK> Database *db = NULL;
IK> LoadLibrary();
IK> func = GetProcAddress();
IK> m_db = fun
[sqlite] Slight problem with sqlite3_compileoption_get
On Sun, 24 Jan 2016 21:26:41 + Bart Smissaert wrote: > http://www.cdecl.org/ > > It doesn't work though with complex arguments like this: > void (*xFunc)(sqlite3_context*,int,sqlite3_value**) In isolation, that's a syntax error because e.g. sqlite3_value is not defined (on that line, which is the only one being parsed). This works: void (*xFunc)(long*,int,long**) declare xFunc as pointer to function (pointer to long, int, pointer to pointer to long) returning void but it's hard to say the output is clearer than the input. --jkl
[sqlite] SQLite crashing
Teg,
On Mon, Jan 25, 2016 at 9:25 PM, Igor Korot wrote:
> Hi, Teg,
>
> On Mon, Jan 25, 2016 at 5:21 PM, Igor Korot wrote:
>> Hi, Teg,
>>
>> On Mon, Jan 25, 2016 at 4:51 PM, Teg wrote:
>>> Hello Igor,
>>>
>>> Then I'd note the address of the object (make a copy of the pointer
>>> right as it's allocated) and then verify that the address you're
>>> deleting is the same as the address that was allocated. I've verify
>>> that the correct calling convention is being used throughout too.
>>
>> Yes, pointers (addresses) are the same.
>>
>>>
>>>
>>> I'd single step into the destructor and see what's actually happening to
>>> make it crash. I'd pay attention to the "this" pointer and see if it
>>> makes sense.
>
> I just ran the program under the debugger step-by-step and everything
> looks "normal".
> All addresses are the same. and the opening db executes successfully.
>
> The stack trace look like this:
>
>>sqlite.dll!sqlite3_mutex_enter(sqlite3_mutex * p) Line 19996 + 0xc bytes
>>C
> sqlite.dll!sqlite3Close(sqlite3 * db, int forceZombie) Line 726
> + 0xc bytesC
> sqlite.dll!sqlite3_close(sqlite3 * db) Line 772 + 0xe bytesC
>
> sqlite.dll!SQLiteDatabase::Disconnect(std::vector,std::allocator
>>,std::allocator,std::allocator
>> > > & errorMsg) Line 49 + 0xc bytesC++
> dialogs.dll!DisconnectFromDb(Database * db) Line 108 + 0x13 bytesC++
> docview.exe!MainFrame::~MainFrame() Line 73 + 0xf bytesC++
>
> The DisconnectFromDb() function is a function from DLL. It calls
> SQLiteDatabase::Disconnect().
> Then everything is sqlite.
>
> Any idea of what is going on?
[code]
SQLITE_API void SQLITE_STDCALL sqlite3_mutex_enter(sqlite3_mutex *p){
if( p ){
assert( sqlite3GlobalConfig.mutex.xMutexEnter );// This is the
line that fails
sqlite3GlobalConfig.mutex.xMutexEnter(p);
}
}
[/code]
Thank you.
>
> Thank you.
>
>>>
>>> Assuming you're using visual studio and have the source to the wrapper
>>> if should be easy to simply step in and see exactly what line of code
>>> makes it crash.
>>
>> Yes, MSVC 2010 and the code is mine. ;-)
>>
>> Thank you.
>>>
>>> C
>>>
>>>
>>> Monday, January 25, 2016, 1:00:53 PM, you wrote:
>>>
>>> IK> Hi, Teg,
>>>
>>> IK> On Mon, Jan 25, 2016 at 11:31 AM, Teg wrote:
> Hello Igor,
>
> MainFrame::~MainFrame()
> {
> delete m_db; // this is where the crash happens
> }
>
> I suspect you need to add a "Close" or destroy function to the DLL and
> pass the handle back to the DLL to let it get deleted in the DLL
> context and not in the context of the caller.
>>>
>>> IK> Does not make a difference.
>>> IK> I added that function and call it from the MainFrame destructor.
>>>
>>> IK> It still crashed.
>>>
>>> IK> Thank you.
>>>
>
>
> extern "C" __declspec(dllexport) void DestroyObject(Database *db)
> {
> delete db;
> }
>
> It was my impression that each DLL got it's own heap so, memory
> allocated inside the DLL needs to be free'd inside the DLL. I use
> Sqlite in a static lib in my applications.
>
> I treat memory allocated in DLL's as being owned by the DLL so, I
> typically pass it back to the DLL to be cleaned up. It believe it
> depends on what run time library you're using though. If you're using
> an RTL where all the DLL's end up using a DLL supplied allocator, this
> probably isn't an issue. I tend to dynamic load my DLL's so they don't
> all use the same allocator.
>
> C
>
>
>
> Monday, January 25, 2016, 11:16:57 AM, you wrote:
>
> IK> Hi, Peter,
>
> IK> On Mon, Jan 25, 2016 at 10:50 AM, Peter Aronson att.net> wrote:
>>> Igor,
>>>
>>> You can't safely pass a SQLite handle between different SQL DLLs that
>>> way if
>>> they're both built with their own copy of the amalgamation (or link to
>>> things built with different copies). SQLite uses a handful of global
>>> variables, but each DLL has its own copy of each of these global
>>> variables
>>> and they can and will have different values, which can mess things up.
>>> I
>>> ran into a version of this problem when I tried to load a 2nd DLL built
>>> with
>>> its own copy of the sqlite3.c amalgamation. I fixed that by exposing
>>> the
>>> SQLite3 entrypoints in the first DLL and linking the second DLL against
>>> it
>>> so there was only one copy of the amalgamation used for that SQLite3
>>> handle.
>
> IK> The SQLite is built only once and with just one version of the code.
>
> IK> Consider following pseudo-code:
>
> IK> In DLL:
>
> IK> BOOL APIENTRY DLLMain()
> IK> {
> IK> }
>
> IK> extern "C" __declspec(dllexport) Database *CreateObject(Database *db)
> IK> {
> IK> db = new SQLiteDatabase();
> IK> db->Connect();
> IK> return db;
[sqlite] SQLite crashing
Hello J, I don't typically use "new" in my C++ applications other than in startup to allocate things that last the life of the application. So, I don't have any issues with double-freeing. I never manually "new" objects after startup. Instead I use containers to contains my objects and leave the allocation and de-allocation to them. >>Are you sure you're not somehow double-freeing the sqlite handle? >>Especially at close I've seen exit() end up calling atexit() methods >>multiple times in some circumstances... "newed" objects never fires the destructor unless you manually destruct it or use an auto-pointer even when you exit. In fact I use this as a method to get around destructor order issues when my applications exit. I essentially just abandon the "new"ed startup objects. It also makes applications exit faster just to abandon memory and depend on the OS to reclaim all the memory.
[sqlite] SQLite crashing
Hello Igor,
Then I'd note the address of the object (make a copy of the pointer
right as it's allocated) and then verify that the address you're
deleting is the same as the address that was allocated. I've verify
that the correct calling convention is being used throughout too.
I'd single step into the destructor and see what's actually happening to
make it crash. I'd pay attention to the "this" pointer and see if it
makes sense.
Assuming you're using visual studio and have the source to the wrapper
if should be easy to simply step in and see exactly what line of code
makes it crash.
C
Monday, January 25, 2016, 1:00:53 PM, you wrote:
IK> Hi, Teg,
IK> On Mon, Jan 25, 2016 at 11:31 AM, Teg wrote:
>> Hello Igor,
>>
>> MainFrame::~MainFrame()
>> {
>> delete m_db; // this is where the crash happens
>> }
>>
>> I suspect you need to add a "Close" or destroy function to the DLL and
>> pass the handle back to the DLL to let it get deleted in the DLL
>> context and not in the context of the caller.
IK> Does not make a difference.
IK> I added that function and call it from the MainFrame destructor.
IK> It still crashed.
IK> Thank you.
>>
>>
>> extern "C" __declspec(dllexport) void DestroyObject(Database *db)
>> {
>> delete db;
>> }
>>
>> It was my impression that each DLL got it's own heap so, memory
>> allocated inside the DLL needs to be free'd inside the DLL. I use
>> Sqlite in a static lib in my applications.
>>
>> I treat memory allocated in DLL's as being owned by the DLL so, I
>> typically pass it back to the DLL to be cleaned up. It believe it
>> depends on what run time library you're using though. If you're using
>> an RTL where all the DLL's end up using a DLL supplied allocator, this
>> probably isn't an issue. I tend to dynamic load my DLL's so they don't
>> all use the same allocator.
>>
>> C
>>
>>
>>
>> Monday, January 25, 2016, 11:16:57 AM, you wrote:
>>
>> IK> Hi, Peter,
>>
>> IK> On Mon, Jan 25, 2016 at 10:50 AM, Peter Aronson
>> wrote:
Igor,
You can't safely pass a SQLite handle between different SQL DLLs that way
if
they're both built with their own copy of the amalgamation (or link to
things built with different copies). SQLite uses a handful of global
variables, but each DLL has its own copy of each of these global variables
and they can and will have different values, which can mess things up. I
ran into a version of this problem when I tried to load a 2nd DLL built
with
its own copy of the sqlite3.c amalgamation. I fixed that by exposing the
SQLite3 entrypoints in the first DLL and linking the second DLL against it
so there was only one copy of the amalgamation used for that SQLite3
handle.
>>
>> IK> The SQLite is built only once and with just one version of the code.
>>
>> IK> Consider following pseudo-code:
>>
>> IK> In DLL:
>>
>> IK> BOOL APIENTRY DLLMain()
>> IK> {
>> IK> }
>>
>> IK> extern "C" __declspec(dllexport) Database *CreateObject(Database *db)
>> IK> {
>> IK> db = new SQLiteDatabase();
>> IK> db->Connect();
>> IK> return db;
>> IK> }
>>
>> IK> In the main application:
>>
>> IK> mainframe.h:
>>
>> IK> class MainFrame
>> IK> {
>> IK> public:
>> IK> MainFrame();
>> IK> ~MainFrame();
>> IK> void ConnectToDb();
>> IK> private:
>> IK> Database *m_db;
>> IK> };
>>
>> IK> mainframe.cpp:
>>
>> IK> void MainFrame::ConnectToDb()
>> IK> {
>> IK> Database *db = NULL;
>> IK> LoadLibrary();
>> IK> func = GetProcAddress();
>> IK> m_db = func( db );
>> IK> }
>>
>> IK> MainFrame::~MainFrame()
>> IK> {
>> IK> delete m_db; // this is where the crash happens
>> IK> }
>>
>> IK> The pointer address are the same in DLL and main application MainFrame
>> class.
>> IK> And as I said the crash occurs when it tries to acquire the mutex lock.
>>
>> IK> Thank you.
>>
Peter
On 1/24/2016 10:18 PM, Igor Korot wrote:
>
> Hi, ALL,
> I have a strange problem.
>
> I am trying to use sqlite in my program. It has a main application and
> couplef DLLs.
>
> I am getting the connection in one of the DLL, then the pointer is passed
> up
> to the main application.
>
> Upon exiting from the application I'm trying to close the connection and
> delete all the memory.
>
> Unfortunately upon exiting the application it crashes inside
> sqlite3_mutex_enter().
> The comment above the function says:
>
> [quote]
> /*
> ** Obtain the mutex p. If some other thread already has the mutex, block
> ** until it can be obtained.
> */
> [/quote]
>
> The DLL does not start any threads, in fact the application will be 1
> thread only.
> So is there some compile-time switch I should use to mitigate the issue?
>
> Moreover I don't understand why am I getting the assertion - there is no
> MT
> involved.
>
