Re: [sqlite] make fails on Solaris
See if your installation has "nawk" or "gawk". Then search through Makefile.in and change the "awk" invocations to "nawk" or "gawk" (three changes). It might work then. --- H S <[EMAIL PROTECTED]> wrote: > You are right. opcodes.h is corrupted. It has 0 byte.. > > I have all other files: lemon exists in the build directory as well > as parrse.c and parse.h > > Here is the messages from configure > > gums2-sun% ../sqlite-3.2.2/configure --prefix=/dssweb/sqlite > checking build system type... sparc-sun-solaris2.8 > checking host system type... sparc-sun-solaris2.8 > checking for gcc... gcc > checking for C compiler default output file name... a.out > checking whether the C compiler works... yes > checking whether we are cross compiling... no > checking for suffix of executables... > checking for suffix of object files... o > checking whether we are using the GNU C compiler... yes > checking whether gcc accepts -g... yes > checking for gcc option to accept ANSI C... none needed > checking for a sed that does not truncate output... /usr/bin/sed > checking for egrep... grep -E > checking for ld used by gcc... > /prod/cygnus/gnupro-03r1/H-sparc-sun-solaris2.8/sparc-sun-solaris2.\ > 8/bin/ld > checking if the linker > (/prod/cygnus/gnupro-03r1/H-sparc-sun-solaris2.8/sparc-sun-solaris2.8/bin/l\ > d) is GNU ld... yes > checking for > /prod/cygnus/gnupro-03r1/H-sparc-sun-solaris2.8/sparc-sun-solaris2.8/bin/ld > option to\ > reload object files... -r > checking for BSD-compatible nm... /usr/ccs/bin/nm -p > checking whether ln -s works... yes > checking how to recognise dependent libraries... pass_all > checking how to run the C preprocessor... gcc -E > checking for ANSI C header files... yes > checking for sys/types.h... yes > checking for sys/stat.h... yes > checking for stdlib.h... yes > checking for string.h... yes > checking for memory.h... yes > checking for strings.h... yes > checking for inttypes.h... yes > checking for stdint.h... no > checking for unistd.h... yes > checking dlfcn.h usability... yes > checking dlfcn.h presence... yes > checking for dlfcn.h... yes > checking for g++... g++ > checking whether we are using the GNU C++ compiler... yes > checking whether g++ accepts -g... yes > checking how to run the C++ preprocessor... g++ -E > checking for g77... no > checking for f77... no > checking for xlf... no > checking for frt... no > checking for pgf77... no > checking for fort77... no > checking for fl32... no > checking for af77... no > checking for f90... no > checking for xlf90... no > checking for pgf90... no > checking for epcf90... no > checking for f95... no > checking for fort... no > checking for xlf95... no > checking for ifc... no > checking for efc... no > checking for pgf95... no > checking for lf95... no > checking for gfortran... no > checking whether we are using the GNU Fortran 77 compiler... no > checking whether accepts -g... no > checking the maximum length of command line arguments... 262144 > checking command to parse /usr/ccs/bin/nm -p output from gcc object... ok > checking for objdir... .libs > checking for ar... ar > checking for ranlib... ranlib > checking for strip... strip > checking if gcc static flag works... yes > checking if gcc supports -fno-rtti -fno-exceptions... yes > checking for gcc option to produce PIC... -fPIC > checking if gcc PIC flag -fPIC works... yes > checking if gcc supports -c -o file.o... yes > checking whether the gcc linker > (/prod/cygnus/gnupro-03r1/H-sparc-sun-solaris2.8/sparc-sun-solaris\ > 2.8/bin/ld) supports shared libraries... yes > checking whether -lc should be explicitly linked in... yes > checking dynamic linker characteristics... solaris2.8 ld.so > checking how to hardcode library paths into programs... immediate > checking whether stripping libraries is possible... no > checking if libtool supports shared libraries... yes > checking whether to build shared libraries... yes > checking whether to build static libraries... yes > configure: creating libtool > appending configuration tag "CXX" to libtool > checking for ld used by g++... > /prod/cygnus/gnupro-03r1/H-sparc-sun-solaris2.8/sparc-sun-solaris2.\ > 8/bin/ld > checking if the linker > (/prod/cygnus/gnupro-03r1/H-sparc-sun-solaris2.8/sparc-sun-solaris2.8/bin/l\ > d) is GNU ld... yes > checking whether the g++ linker > (/prod/cygnus/gnupro-03r1/H-sparc-sun-solaris2.8/sparc-sun-solaris\ > 2.8/bin/ld) supports shared libraries... yes > checking for g++ option to produce PIC... -fPIC > checking if g++ PIC flag -fPIC works... yes > checking if g++ supports -c -o file.o... yes > checking whether the g++ linker > (/prod/cygnus/gnupro-03r1/H-sparc-sun-solaris2.8/sparc-sun-solaris\ > 2.8/bin/ld) supports shared libraries... yes > checking dynamic linker characteristics... solaris2.8 ld.so > checking how to hardcode library paths into programs... immediate > checking whether stripping li
Re: [sqlite] make fails on Solaris
You are right. opcodes.h is corrupted. It has 0 byte.. I have all other files: lemon exists in the build directory as well as parrse.c and parse.h Here is the messages from configure gums2-sun% ../sqlite-3.2.2/configure --prefix=/dssweb/sqlite checking build system type... sparc-sun-solaris2.8 checking host system type... sparc-sun-solaris2.8 checking for gcc... gcc checking for C compiler default output file name... a.out checking whether the C compiler works... yes checking whether we are cross compiling... no checking for suffix of executables... checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ANSI C... none needed checking for a sed that does not truncate output... /usr/bin/sed checking for egrep... grep -E checking for ld used by gcc... /prod/cygnus/gnupro-03r1/H-sparc-sun-solaris2.8/sparc-sun-solaris2.\ 8/bin/ld checking if the linker (/prod/cygnus/gnupro-03r1/H-sparc-sun-solaris2.8/sparc-sun-solaris2.8/bin/l\ d) is GNU ld... yes checking for /prod/cygnus/gnupro-03r1/H-sparc-sun-solaris2.8/sparc-sun-solaris2.8/bin/ld option to\ reload object files... -r checking for BSD-compatible nm... /usr/ccs/bin/nm -p checking whether ln -s works... yes checking how to recognise dependent libraries... pass_all checking how to run the C preprocessor... gcc -E checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... no checking for unistd.h... yes checking dlfcn.h usability... yes checking dlfcn.h presence... yes checking for dlfcn.h... yes checking for g++... g++ checking whether we are using the GNU C++ compiler... yes checking whether g++ accepts -g... yes checking how to run the C++ preprocessor... g++ -E checking for g77... no checking for f77... no checking for xlf... no checking for frt... no checking for pgf77... no checking for fort77... no checking for fl32... no checking for af77... no checking for f90... no checking for xlf90... no checking for pgf90... no checking for epcf90... no checking for f95... no checking for fort... no checking for xlf95... no checking for ifc... no checking for efc... no checking for pgf95... no checking for lf95... no checking for gfortran... no checking whether we are using the GNU Fortran 77 compiler... no checking whether accepts -g... no checking the maximum length of command line arguments... 262144 checking command to parse /usr/ccs/bin/nm -p output from gcc object... ok checking for objdir... .libs checking for ar... ar checking for ranlib... ranlib checking for strip... strip checking if gcc static flag works... yes checking if gcc supports -fno-rtti -fno-exceptions... yes checking for gcc option to produce PIC... -fPIC checking if gcc PIC flag -fPIC works... yes checking if gcc supports -c -o file.o... yes checking whether the gcc linker (/prod/cygnus/gnupro-03r1/H-sparc-sun-solaris2.8/sparc-sun-solaris\ 2.8/bin/ld) supports shared libraries... yes checking whether -lc should be explicitly linked in... yes checking dynamic linker characteristics... solaris2.8 ld.so checking how to hardcode library paths into programs... immediate checking whether stripping libraries is possible... no checking if libtool supports shared libraries... yes checking whether to build shared libraries... yes checking whether to build static libraries... yes configure: creating libtool appending configuration tag "CXX" to libtool checking for ld used by g++... /prod/cygnus/gnupro-03r1/H-sparc-sun-solaris2.8/sparc-sun-solaris2.\ 8/bin/ld checking if the linker (/prod/cygnus/gnupro-03r1/H-sparc-sun-solaris2.8/sparc-sun-solaris2.8/bin/l\ d) is GNU ld... yes checking whether the g++ linker (/prod/cygnus/gnupro-03r1/H-sparc-sun-solaris2.8/sparc-sun-solaris\ 2.8/bin/ld) supports shared libraries... yes checking for g++ option to produce PIC... -fPIC checking if g++ PIC flag -fPIC works... yes checking if g++ supports -c -o file.o... yes checking whether the g++ linker (/prod/cygnus/gnupro-03r1/H-sparc-sun-solaris2.8/sparc-sun-solaris\ 2.8/bin/ld) supports shared libraries... yes checking dynamic linker characteristics... solaris2.8 ld.so checking how to hardcode library paths into programs... immediate checking whether stripping libraries is possible... no appending configuration tag "F77" to libtool checking for a BSD-compatible install... ../sqlite-3.2.2/install-sh -c Version set to 3.2 Release set to 3.2.2 Version number set to 3002002 checking for gcc... (cached) gcc checking whether we are using the GNU C compiler... (cached) yes checking whether gcc accepts -g... (cached) yes checking for gcc option to accept ANSI C... (cached) none needed checkin
Re: [sqlite] make fails on Solaris
> Hi, > > Could someone please shed some light what I should do? > I've got the following errors. > > Thank you, > Isarin > > gums-sun% make > ./libtool --mode=compile gcc -g -O2 -DOS_UNIX=1 -DHAVE_USLEEP=1 -I. > -I../sqlite-3.2.2/src -DNDEBUG -DSQLITE_OM\ > IT_CURSOR -c ../sqlite-3.2.2/src/alter.c > mkdir .libs > gcc -g -O2 -DOS_UNIX=1 -DHAVE_USLEEP=1 -I. -I../sqlite-3.2.2/src > -DNDEBUG -DSQLITE_OMIT_CURSOR -c ../sqlite-3\ > .2.2/src/alter.c -fPIC -DPIC -o .libs/alter.o > ../sqlite-3.2.2/src/alter.c: In function `reloadTableSchema': > ../sqlite-3.2.2/src/alter.c:220: `OP_DropTrigger' undeclared (first the file opcodes.h is missing or broken. This header file is made by awk from parse.h. I can't imagine there is a unix system that doesn't have an awk installed. Does parse.h exist? If not, parse.h and parse.c is made from parse.y by lemon. Lemon should have been build before you got this error message. Does lemon exist in the sqlite build directory? Gerald
[sqlite] make fails on Solaris
Hi, Could someone please shed some light what I should do? I've got the following errors. Thank you, Isarin gums-sun% make ./libtool --mode=compile gcc -g -O2 -DOS_UNIX=1 -DHAVE_USLEEP=1 -I. -I../sqlite-3.2.2/src -DNDEBUG -DSQLITE_OM\ IT_CURSOR -c ../sqlite-3.2.2/src/alter.c mkdir .libs gcc -g -O2 -DOS_UNIX=1 -DHAVE_USLEEP=1 -I. -I../sqlite-3.2.2/src -DNDEBUG -DSQLITE_OMIT_CURSOR -c ../sqlite-3\ .2.2/src/alter.c -fPIC -DPIC -o .libs/alter.o ../sqlite-3.2.2/src/alter.c: In function `reloadTableSchema': ../sqlite-3.2.2/src/alter.c:220: `OP_DropTrigger' undeclared (first use in this function) ../sqlite-3.2.2/src/alter.c:220: (Each undeclared identifier is reported only once ../sqlite-3.2.2/src/alter.c:220: for each function it appears in.) ../sqlite-3.2.2/src/alter.c:225: `OP_DropTable' undeclared (first use in this function) ../sqlite-3.2.2/src/alter.c:230: `OP_ParseSchema' undeclared (first use in this function) ../sqlite-3.2.2/src/alter.c: In function `sqlite3AlterFinishAddColumn': ../sqlite-3.2.2/src/alter.c:468: `OP_ReadCookie' undeclared (first use in this function) ../sqlite-3.2.2/src/alter.c:469: `OP_Integer' undeclared (first use in this function) ../sqlite-3.2.2/src/alter.c:470: `OP_Ge' undeclared (first use in this function) ../sqlite-3.2.2/src/alter.c:472: `OP_SetCookie' undeclared (first use in this function) *** Error code 1 make: Fatal error: Command failed for target `alter.lo' gums-sun% uname -snrvmapiX SunOS gums-sun 5.8 Generic_108528-27 sun4u sparc SUNW,Netra-T12System = SunOS Node = gums2-sun Release = 5.8 KernelID = Generic_108528-27 Machine = sun4u BusType = Serial = Users = OEM# = 0 Origin# = 1 NumCPU = 8
Re: [sqlite] SQLITE3 database open from multiple processes
On 7/15/05, Jay Sprenkle <[EMAIL PROTECTED]> wrote: > Make sure you use and observe locks and it works fine. > You can only have one process writing at a specific instant. > What do you know, the problem was actually that I was starting another query before having called sqlite3_finalize() on the old one. But, adding transactions helped me to actually get an error from the database that was meaningful. I think I'll keep the transactions there since they will improve performance. ttyl, --buddy
Re: [sqlite] SQLITE3 database open from multiple processes
> Is it safe to have the database open by two separate processes? > > I have an application that uses the database somewhat often and runs > 24/7. I have another application that runs once / day. The second > application inserts some data into a table that is not coming up in > the database. I have confirmed that the query works fine from the > sqlite3 command line client. It does not return any error codes. > > Any ideas what I might need to do to make this work properly? Make sure you use and observe locks and it works fine. You can only have one process writing at a specific instant.
[sqlite] SQLITE3 database open from multiple processes
Hi, Is it safe to have the database open by two separate processes? I have an application that uses the database somewhat often and runs 24/7. I have another application that runs once / day. The second application inserts some data into a table that is not coming up in the database. I have confirmed that the query works fine from the sqlite3 command line client. It does not return any error codes. Any ideas what I might need to do to make this work properly? Thanks, --buddy
[sqlite] Newbie Help Please
Heyas, I tried to do the quick-start example and I could not get it to work. It is not explicit enough for some like me. First off, the quickstart doesn't specify that you need to include the source (just the external interface header). But since it does not come with a *.lib I tried to include the source files and I get a bunch of unresolved external dependencies (can't find some function definitions). I even removed the tcl*.c files and still got the errors. I'm compiling in Visual C++ 7.1. Also, I tried to build with the *.dll but the *.dll file doesn't come with a header for it or static *.lib to link against. If I use the *.dll do I have to fn* every function I wish to use? Sorry for the newb questions. Thank you for the help, Lee
Re: [sqlite] Multi-threading.
On Fri, Jul 15, 2005 at 01:04:50PM -0400, Paul G wrote: > the issue wasn't necessarily the thread implementation per se, but the fact > that threads were treated as processes for scheduling purposes and hence > scheduled with the regular process scheduler, which was not efficient for a > large number of processes. these problems went away when ingo molnar's O(1) > scheduler was adopted (not sure when it was merged into linus' 2.4, but Interesting. Then that may explain why I never heard any first hand reports of notable Linux threading problems. The people I tended to talk to were probably all running with well under 100 threads per process, and only 2 or 3 such processes at most. Presumably even the earlier lousy process scheduler could handle that ok. -- Andrew Piskorski <[EMAIL PROTECTED]> http://www.piskorski.com/
Re: [sqlite] Multi-threading.
D. Richard Hipp wrote: On Fri, 2005-07-15 at 16:41 +0530, Roushan Ali wrote: Hello all, Can we use single sqlite_open handle(global) across threads( if all database operations are serialized by using semaphore) ? Please help. Opening a database connection in one thread and using it in another will work on some operating systems but not on others. You are advised not to do it. See http://www.sqlite.org/cvstrac/tktview?tn=1272 and http://www.sqlite.org/cvstrac/chngview?cn=2521. Good sound advice, to a point. Multiple threads accessing the same connection *can* be done, its a design time issue that needs to be addressed before even a single line of code is written. I do it with my mail server using SQLite for logging purposes, I use mutex semaphores to handle the nitty gritty details. Its the usual issue of knowing what you are stepping into before you step, because some of what you step into stinks. -- Craig Morrison =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= http://www.mtsprofessional.com/ A Win32 email server that works for You.
Re: [sqlite] Multi-threading.
Andrew Piskorski wrote: On Fri, Jul 15, 2005 at 04:21:05PM +0300, Cariotoglou Mike wrote: memory and cpu-wise. on Linux, this is nothing, it can handle it easily. otoh, 500 threads for windows is business as usual, but threading on Linux, is , I hear, iffy at best. Linux runs multi-threaded apps (e.g., AOLserver) quite well, and has for many years - since at least 2000 or so, probably earlier. My understanding is that the old LinuxThreads implementation had some pretty ugly bits, but it worked. NPTL is much better, and is standard with the Linux 2.6.x kernels. Some architectures permit, or even encourage, multi-threaded design. It can be done obviously. However, Dr. Hipp still has a point. One thread can trash another's address space. They share code, global data, the heap (generally) and system object handles (files, sockets, IPC devices ( and weird crap like "Desktop" and "Mutants" on windows). The only non-shared things are the stack, TLS (thread local storage) and per-thread CPU context. Even then all of those things can be trashed by other threads in the same process. Unless you can _prove_ that your code won't do this (and all code that you call, including system DLLs / SOs) then you are taking a risk. Personally, I prefer multi-threaded code. I like to write it and I like to debug it. I ship it to customers. Your millage may vary. And yes, Linux threads used to be very unstable. I've only used Linux threads once, and it was on a recent 2.6 kernel, so I never experienced the problem(s).
Re: [sqlite] Multi-threading.
- Original Message - From: "Andrew Piskorski" <[EMAIL PROTECTED]> To: Sent: Friday, July 15, 2005 1:05 PM Subject: Re: [sqlite] Multi-threading. > On Fri, Jul 15, 2005 at 04:21:05PM +0300, Cariotoglou Mike wrote: > > > memory and cpu-wise. on Linux, this is nothing, it can handle it easily. > > otoh, 500 threads for windows is business as usual, but threading on > > Linux, is , I hear, iffy at best. > > Linux runs multi-threaded apps (e.g., AOLserver) quite well, and has > for many years - since at least 2000 or so, probably earlier. My > understanding is that the old LinuxThreads implementation had some > pretty ugly bits, but it worked. NPTL is much better, and is standard > with the Linux 2.6.x kernels. the issue wasn't necessarily the thread implementation per se, but the fact that threads were treated as processes for scheduling purposes and hence scheduled with the regular process scheduler, which was not efficient for a large number of processes. these problems went away when ingo molnar's O(1) scheduler was adopted (not sure when it was merged into linus' 2.4, but distros adopted it quite fast). -p
Re: [sqlite] Multi-threading.
On Fri, Jul 15, 2005 at 04:21:05PM +0300, Cariotoglou Mike wrote: > memory and cpu-wise. on Linux, this is nothing, it can handle it easily. > otoh, 500 threads for windows is business as usual, but threading on > Linux, is , I hear, iffy at best. Linux runs multi-threaded apps (e.g., AOLserver) quite well, and has for many years - since at least 2000 or so, probably earlier. My understanding is that the old LinuxThreads implementation had some pretty ugly bits, but it worked. NPTL is much better, and is standard with the Linux 2.6.x kernels. -- Andrew Piskorski <[EMAIL PROTECTED]> http://www.piskorski.com/
Re: [sqlite] Multi-threading.
D. Richard Hipp wrote: Actually, this seems like a good opportunity to repeat my oft-ignored advice to not use more than one thread in a single address space. If you need multiple threads, create multiple processes. I think its not really an acceptable option for those who are on Windows :( best regards, Alex
Re: [sqlite] Multi-threading.
Roushan Ali wrote: Hi, Thanks for your response. I don't have any idea how multiple connection objects work. Can you please tell us something about that. I wrappered the C interface to SQLite3 via a C++ Class called "CSqlite3". The constructor does NOT open the database, it just allocates the sqlite struct. I declared 4 global instances of this class. The constructors get called before my WinMain(). In my initialization code (called before any threads are created), I open the database 4 times. I do an integrity check (and some other logic) after the first open. Like this: g_DbMain.Open(szFilename); CheckDatabase(g_DbMain); // "pragma integrity_check, create missing tables / schema updates, vacuum" g_DbTimer.Open(szFilename); g_DbThread2.Open(szFilename); g_DbThread3.Open(szFilename); I then create the worker threads. One of my threads does NOT use any database, so we'll ignore it. Another thread (main / gui) already exists, so I am really only creating threads #2 and #3. The thread function uses the database object as needed. After the worker threads terminate, the main thread closes all four database objects. The object's destructor is called when the application exits. I do not create new connections to the database while the executing. Please note that my solution is NOT appropriate if I wanted to create arbitrary threads at arbitrary times. If I were doing that, then each thread would create it's own database object on it's own TLS (thread local storage) or stack. I created all of my database "Open()" code into the main thread just to keep it all together. Each of my threads does a very specific function that is totally unique to that thread: 1. The main thread uses it's database connection to respond to user initiated GUI events. 2. The main thread also uses the "timer" database connection to handle WM_TIMER messages to update a status display synchronously (kinda). Because this function can be invoked while the thread has a transaction on the main connection, I need to use a different connection. One thread, but it must be fully re-entrant. 3. Thread #2 is a producer. It gathers data and inserts it into the database. 4. Thread #3 is a consumer. It takes data from the database and does stuff with them. It updates those rows. The timer connection only executes "select" to update the GUI. The main connection is used to query the database, update the database and to delete from the database. The application is what it is. I make no public claims about it being the best designed thing ever, but it does work well under stress.
RE: [sqlite] How to delete all rows in table (TRUNCATE) without creating journal file(s)?
Yes to both questions. -Randy -Original Message- From: Jay Sprenkle [mailto:[EMAIL PROTECTED] Sent: Friday, July 15, 2005 6:37 AM To: sqlite-users@sqlite.org Subject: Re: [sqlite] How to delete all rows in table (TRUNCATE) without creating journal file(s)? > Disabling the journal would roughly half the amount > of disk (flash) I/O required. If it currently takes > 2 minutes to delete, disabling the journal would > reduce that to about 1 minutes. Still a long time. You were aware that flash memory has a limited number of write cycles before it fails right? Is your flash disk backed with RAM?
Re: [sqlite] Multi-threading.
Hi, Thanks for your response. I don't have any idea how multiple connection objects work. Can you please tell us something about that. Thanks, Roushan On Fri, 2005-07-15 at 20:15, Dennis Jenkins wrote: > Roushan Ali wrote: > > >Thanks Richard for your reply. > > > >Actually, we have written a windows application which uses four threads. > >Each thread may have to add/delete thousands of entries in the database( > >for performance reason , we don't want to open/close the database for > >each insertion/deletion) .If we use different sqlite_open handle for > >each thread , then one thread has to do busy looping until other threads > >complete their operation, which is not desirable according to the > >application requirement. That's why we opened global database handle for > >the lifetime of the application and each thread used the handle serially > >and it worked. > > > > > > > We have a multi-threaded windows application with four threads. Three > threads need access to the database (all three are producers and > consumers), but one thread is the GUI thread and wants to access the > database while handling WM_TIMER messages (re-entrency issues). So we > allocate 4 database connections during initialization. Each section of > our code uses its own connection. We have a special "stress test" mode > that we can enable. The program remains stable after hours of operation > under the stress test. The program will slow down because of the > database locking mechanism (especially during large transactions), but > it has never crashed due to multiple threads accessing the database used > _different_ connection objects. > > If you are going to be multi-threaded, then why not just use multiple > connection objects (structs - ours are wrapped in a C++ class)? > >
Re: [sqlite] Multi-threading.
On Fri, Jul 15, 2005 at 08:27:14AM -0400, D. Richard Hipp wrote: > [...] > I am constantly amazed at the prevailing idea (exemplified > by Java) that software should be strongly typed and should > not use goto statement or pointers - all in the name of > reducing bugs - but that it is OK to use multiple threads > within the same address space. Strong typing helps prevent > only bugs that are trivially easy to locate and fix. The > use of goto statements and pointers likewise results in > deterministic problems that are easy to test for and > relatively easy to track down and correct. But threading > bugs tend to manifest themselves as timing-dependent > glitches and lock-ups that are hardware and platform > dependent, that never happen the same way twice, and that > only appear for customers after deployment and never in a > testing environment. My quote of the week :-) -- Gerhard -- Gerhard Häring - [EMAIL PROTECTED] - Python, web & database development signature.asc Description: Digital signature
Re: [sqlite] Multi-threading.
Roushan Ali wrote: Thanks Richard for your reply. Actually, we have written a windows application which uses four threads. Each thread may have to add/delete thousands of entries in the database( for performance reason , we don't want to open/close the database for each insertion/deletion) .If we use different sqlite_open handle for each thread , then one thread has to do busy looping until other threads complete their operation, which is not desirable according to the application requirement. That's why we opened global database handle for the lifetime of the application and each thread used the handle serially and it worked. We have a multi-threaded windows application with four threads. Three threads need access to the database (all three are producers and consumers), but one thread is the GUI thread and wants to access the database while handling WM_TIMER messages (re-entrency issues). So we allocate 4 database connections during initialization. Each section of our code uses its own connection. We have a special "stress test" mode that we can enable. The program remains stable after hours of operation under the stress test. The program will slow down because of the database locking mechanism (especially during large transactions), but it has never crashed due to multiple threads accessing the database used _different_ connection objects. If you are going to be multi-threaded, then why not just use multiple connection objects (structs - ours are wrapped in a C++ class)?
Re: [sqlite] Multi-threading.
Thanks Richard for your reply. Actually, we have written a windows application which uses four threads. Each thread may have to add/delete thousands of entries in the database( for performance reason , we don't want to open/close the database for each insertion/deletion) .If we use different sqlite_open handle for each thread , then one thread has to do busy looping until other threads complete their operation, which is not desirable according to the application requirement. That's why we opened global database handle for the lifetime of the application and each thread used the handle serially and it worked. Thanking you again, Roushan On Fri, 2005-07-15 at 17:57, D. Richard Hipp wrote: > On Fri, 2005-07-15 at 16:41 +0530, Roushan Ali wrote: > > Hello all, > > Can we use single sqlite_open handle(global) across threads( > > if all database operations are serialized by using semaphore) ? Please > > help. > > > > Opening a database connection in one thread and using it in another > will work on some operating systems but not on others. You are > advised not to do it. See http://www.sqlite.org/cvstrac/tktview?tn=1272 > and http://www.sqlite.org/cvstrac/chngview?cn=2521. > > Actually, this seems like a good opportunity to repeat my > oft-ignored advice to not use more than one thread in a single > address space. If you need multiple threads, create multiple > processes. This has nothing to do with SQLite = it is just > good programming advice. I have worked on countless multi- > threaded programs over the years, and I have yet to see a > single one that didn't contain subtle, hard to reproduce, > and very hard to troubleshoot bugs related to threading issues. > > I am constantly amazed at the prevailing idea (exemplified > by Java) that software should be strongly typed and should > not use goto statement or pointers - all in the name of > reducing bugs - but that it is OK to use multiple threads > within the same address space. Strong typing helps prevent > only bugs that are trivially easy to locate and fix. The > use of goto statements and pointers likewise results in > deterministic problems that are easy to test for and > relatively easy to track down and correct. But threading > bugs tend to manifest themselves as timing-dependent > glitches and lock-ups that are hardware and platform > dependent, that never happen the same way twice, and that > only appear for customers after deployment and never in a > testing environment.
Re: [sqlite] How to delete all rows in table (TRUNCATE) without creating journal file(s)?
> Disabling the journal would roughly half the amount > of disk (flash) I/O required. If it currently takes > 2 minutes to delete, disabling the journal would > reduce that to about 1 minutes. Still a long time. You were aware that flash memory has a limited number of write cycles before it fails right? Is your flash disk backed with RAM?
[sqlite] NOCASE COLLATION + INDEX - Any known issue?
Hi, I have ported Sqlite 3.0.8 source onto Arm platform and it is having some problems when I create INDICES for the columns that have COLLATE NOCASE conditions. Is there any known issue and if so any fixes already for such a problem? I find that when I insert into the table with one of its column which have COLLATE NOCASE defined, after around 400 inserts into the table, the database image gets corrupted for that table. The pragma integrity check (when run through sqlite3Explorer) returns SQL LOGIC ERROR OR MISSING DATABASE as the error. My schema is CREATE TABLE albumTable( iAlbumId INTEGER PRIMARY KEY, cAlbumTitle VARCHAR(512) COLLATE NOCASE) CREATE INDEX album_cAlbumTitle ON albumTable (cAlbumTitle) If I remove the index and insert, I could even insert more than 5000 records (the maximum I tested with the limited memory available in my target) and I could query too on this table. I request you to please provide me any known issues in this area and in the mean time I am verifying the changes I did during the porting of the library. In case any one of you require any details please contact me. Your inputs are highly appreciated as we need to fix this problem at the earliest. Thank you, With Regards, Sankara Narayanan "Utthistatha Jaagrata Praapya Varaan Nibodhatha"
RE: [sqlite] Multi-threading.
which gives me the opportunity to repear my oft-ignored reply :) what you say is true, provided the OS is geared towards multiple processes. which is not true for windows, but is true for *ix, as400 and other environments. if you need, say, a server that handles 500 sessions, and attempt to do this with spawning processes on windows, you are probably dead, memory and cpu-wise. on Linux, this is nothing, it can handle it easily. otoh, 500 threads for windows is business as usual, but threading on Linux, is , I hear, iffy at best. so, although this is good advice, it is not unconditionally applicable. > -Original Message- > From: D. Richard Hipp [mailto:[EMAIL PROTECTED] > Sent: Friday, July 15, 2005 3:27 PM > To: sqlite-users@sqlite.org > Subject: Re: [sqlite] Multi-threading. > > On Fri, 2005-07-15 at 16:41 +0530, Roushan Ali wrote: > > Hello all, > > Can we use single sqlite_open handle(global) > across threads( > > if all database operations are serialized by using > semaphore) ? Please > > help. > > > > Opening a database connection in one thread and using it in > another will work on some operating systems but not on > others. You are advised not to do it. See > http://www.sqlite.org/cvstrac/tktview?tn=1272 > and http://www.sqlite.org/cvstrac/chngview?cn=2521. > > Actually, this seems like a good opportunity to repeat my > oft-ignored advice to not use more than one thread in a > single address space. If you need multiple threads, create > multiple processes. This has nothing to do with SQLite = it > is just good programming advice. I have worked on countless > multi- threaded programs over the years, and I have yet to > see a single one that didn't contain subtle, hard to > reproduce, and very hard to troubleshoot bugs related to > threading issues. > > I am constantly amazed at the prevailing idea (exemplified by > Java) that software should be strongly typed and should not > use goto statement or pointers - all in the name of reducing > bugs - but that it is OK to use multiple threads within the > same address space. Strong typing helps prevent only bugs > that are trivially easy to locate and fix. The use of goto > statements and pointers likewise results in deterministic > problems that are easy to test for and relatively easy to > track down and correct. But threading bugs tend to manifest > themselves as timing-dependent glitches and lock-ups that are > hardware and platform dependent, that never happen the same > way twice, and that only appear for customers after > deployment and never in a testing environment. > -- > D. Richard Hipp <[EMAIL PROTECTED]> > > > >
[sqlite] sqlite3_errmsg
Have I to free the string buffer returned by sqlite3_errmsg? If I write something like: char *err; err = (char *) sqlite3_errmsg(db); if (err) sqlite3_free(err); or if (err) free(err); then, OSX gives me a "Deallocation of a pointer not malloced" error... So it seems that I haven't to free it, right? Any help? Thanks, Marco Bambini
Re: [sqlite] Multi-threading.
On Fri, 2005-07-15 at 16:41 +0530, Roushan Ali wrote: > Hello all, > Can we use single sqlite_open handle(global) across threads( > if all database operations are serialized by using semaphore) ? Please > help. > Opening a database connection in one thread and using it in another will work on some operating systems but not on others. You are advised not to do it. See http://www.sqlite.org/cvstrac/tktview?tn=1272 and http://www.sqlite.org/cvstrac/chngview?cn=2521. Actually, this seems like a good opportunity to repeat my oft-ignored advice to not use more than one thread in a single address space. If you need multiple threads, create multiple processes. This has nothing to do with SQLite = it is just good programming advice. I have worked on countless multi- threaded programs over the years, and I have yet to see a single one that didn't contain subtle, hard to reproduce, and very hard to troubleshoot bugs related to threading issues. I am constantly amazed at the prevailing idea (exemplified by Java) that software should be strongly typed and should not use goto statement or pointers - all in the name of reducing bugs - but that it is OK to use multiple threads within the same address space. Strong typing helps prevent only bugs that are trivially easy to locate and fix. The use of goto statements and pointers likewise results in deterministic problems that are easy to test for and relatively easy to track down and correct. But threading bugs tend to manifest themselves as timing-dependent glitches and lock-ups that are hardware and platform dependent, that never happen the same way twice, and that only appear for customers after deployment and never in a testing environment. -- D. Richard Hipp <[EMAIL PROTECTED]>
[sqlite] Multi-threading.
Hello all, Can we use single sqlite_open handle(global) across threads( if all database operations are serialized by using semaphore) ? Please help. Regards, Roushan
