Re: [sqlite] Towards SQLite version 3.5.0
2007/8/29, [EMAIL PROTECTED] [EMAIL PROTECTED]: I think you will be much much better off to put every thread it is own private address space. In other words, turn each thread into a separate process. Threads are an invention of the devil. Stay as far away from these fiendish abominations as you can get. In Unix processes are rather cheap (fork, etc.), and as threads got added late (mid 90ies) to the unix (posix) standard, several concurrent threading implementation are spreading around, and pthread is just one of them. Famous book Advanced Programming in Unix Environment by Stevens (1992) doesn't cover threads at all, but processes. Butenhof's Programming with POSIX Threads (1997) is one of the first and best book that explains the new standard unix threads. In Win32, process are more expensive (more management overhead), and threads, fibers or even I/O complettion ports are the prefered way to go. - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Towards SQLite version 3.5.0
Klemens Friedl wrote: 2007/8/29, [EMAIL PROTECTED] [EMAIL PROTECTED]: I think you will be much much better off to put every thread it is own private address space. In other words, turn each thread into a separate process. Threads are an invention of the devil. Stay as far away from these fiendish abominations as you can get. In Unix processes are rather cheap (fork, etc.), and as threads got added late (mid 90ies) to the unix (posix) standard, several concurrent threading implementation are spreading around, and pthread is just one of them. Famous book Advanced Programming in Unix Environment by Stevens (1992) doesn't cover threads at all, but processes. Butenhof's Programming with POSIX Threads (1997) is one of the first and best book that explains the new standard unix threads. In Win32, process are more expensive (more management overhead), and threads, fibers or even I/O complettion ports are the prefered way to go. Well, I'm already using boost::thread for this application on win32. Since the application will not be finished before sqlite 3.5 I just though I could develop using 3.5 and help finding sqlites multithread-bugs at the same time. This is why I originally asked the question about the best practice when going multithread on SQLite 3.5. - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Towards SQLite version 3.5.0
Hi! The new multithread-features will be great. Do you think that it will be better to share one connection between all theads in an application or is better to have each thread open a new connection and use the sqlite3_enable_shared_cache? Best regards Daniel [EMAIL PROTECTED] wrote: The transition from 3.4.2 to 3.5.0 will perhaps be the largest single change to SQLite since 2.8-3.0. There will not be that many visible changes, but a lot is changing behind the scenes. Some less frequently used interfaces will be changing in slightly incompatible ways. Users who have build customized OS intereface layers or backends for SQLite will find that they are going to need to do some rework. SQLite version 3.5.0 is not close to being ready yet. But it is to the point where the source code will compile and pass many tests. And so I would like to take this opportunity to encourage people in the community to download the CVS HEAD and give it a whirl in their applications. Please let me know about any serious issues you run across. I have *started* to prepare documentation describing the changes in 3.5.0. This is draft documentation. But for those who are interested, please visit http://www.sqlite.org/34to35.html http://www.sqlite.org/capi350ref.html In particular, if your application uses a customized OS interface for SQLite, you should read the 34to35.html document to see exactly what will be involved in porting your application to run with version 3.5.0. The SQLite code currently in CVS HEAD is not ready for production use. We know that. We know what many of the problems are and Dan and I are working long hours to fix them. It's the problems that we *do not* know about that are scary. So that is why I am inviting the larger community to have an early look and perhaps bring our attention to issues sooner rather than later. -- D. Richard Hipp [EMAIL PROTECTED] - To unsubscribe, send email to [EMAIL PROTECTED] - - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Towards SQLite version 3.5.0
=?ISO-8859-1?Q?Daniel_=D6nnerby?= [EMAIL PROTECTED] wrote: The new multithread-features will be great. Do you think that it will be better to share one connection between all theads in an application or is better to have each thread open a new connection and use the sqlite3_enable_shared_cache? I think you will be much much better off to put every thread it is own private address space. In other words, turn each thread into a separate process. Threads are an invention of the devil. Stay as far away from these fiendish abominations as you can get. -- D. Richard Hipp [EMAIL PROTECTED] - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Towards SQLite version 3.5.0
D R H said: Threads are an invention of the devil. I rather think human beings were to blame; but they had probably read a lot about the Spanish Inquisition. Stay as far away from these fiendish abominations as you can get. That's the best bit of programming advice on threads I have ever seen. Thank you for not pretending that in fact it's all very straightforward, which would make the rest of us feel inadequate. Michael Hooker On 29/08/2007 19:52:11, [EMAIL PROTECTED] wrote: =?ISO-8859-1?Q?Daniel_=D6nnerby?= [EMAIL PROTECTED] wrote: The new multithread-features will be great. Do you think that it will be better to share one connection between all theads in an application or is better to have each thread open a new connection and use the sqlite3_enable_shared_cache? I think you will be much much better off to put every thread it is own private address space. In other words, turn each thread into a separate process. Threads are an invention of the devil. Stay as far away from these fiendish abominations as you can get. -- D. Richard Hipp [EMAIL PROTECTED] - To unsubscribe, send email to [EMAIL PROTECTED] - - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Towards SQLite version 3.5.0
--- [EMAIL PROTECTED] wrote: =?ISO-8859-1?Q?Daniel_=D6nnerby?= [EMAIL PROTECTED] wrote: The new multithread-features will be great. Do you think that it will be better to share one connection between all theads in an application or is better to have each thread open a new connection and use the sqlite3_enable_shared_cache? I think you will be much much better off to put every thread it is own private address space. In other words, turn each thread into a separate process. Threads are an invention of the devil. Stay as far away from these fiendish abominations as you can get. :-) From a library maker's point of view, I can see your point. But from a library user's perspective, it's often more convenient to use threads over a multi-process approach. Event-based asynchronous coding tends to turn your code inside out. Many users like the simpler linear style of coding within a thread that keeps business logic together - the Java way. Multi-threading headaches typically stem from manual shared memory coordination. Here's one alternative to mutexes that's getting some attention, but probably needs built-in language support to be effective: http://en.wikipedia.org/wiki/Software_transactional_memory Yahoo! oneSearch: Finally, mobile search that gives answers, not web links. http://mobile.yahoo.com/mobileweb/onesearch?refer=1ONXIC - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Towards SQLite version 3.5.0
Joe Wilson [EMAIL PROTECTED] wrote: --- [EMAIL PROTECTED] wrote: The transition from 3.4.2 to 3.5.0 will perhaps be the The SQLite code currently in CVS HEAD is not ready for production use. We know that. We know what many of the problems are and Dan and I are working long hours to fix them. It's the problems that we *do not* know about that are scary. So that is why I am inviting the larger community to have an early look and perhaps bring our attention to issues sooner rather than later. Just to clarify, you don't want to see make test failure tickets at this time? Correct. I can figure out for myself when make test fails. Those failures are examples of things I know about. I'm much more interested in errors that I don't know about. -- D. Richard Hipp [EMAIL PROTECTED] - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Towards SQLite version 3.5.0
--- [EMAIL PROTECTED] wrote: Joe Wilson [EMAIL PROTECTED] wrote: --- [EMAIL PROTECTED] wrote: The transition from 3.4.2 to 3.5.0 will perhaps be the The SQLite code currently in CVS HEAD is not ready for production use. We know that. We know what many of the problems are and Dan and I are working long hours to fix them. It's the problems that we *do not* know about that are scary. So that is why I am inviting the larger community to have an early look and perhaps bring our attention to issues sooner rather than later. Just to clarify, you don't want to see make test failure tickets at this time? Correct. I can figure out for myself when make test fails. Those failures are examples of things I know about. I'm much more interested in errors that I don't know about. http://en.wikipedia.org/wiki/Unknown_unknown Choose the right car based on your needs. Check out Yahoo! Autos new Car Finder tool. http://autos.yahoo.com/carfinder/ - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Towards SQLite version 3.5.0
[EMAIL PROTECTED] wrote: The transition from 3.4.2 to 3.5.0 will perhaps be the largest single change to SQLite since 2.8-3.0. I have *started* to prepare documentation describing the changes in 3.5.0. This is draft documentation. But for those who are interested, please visit http://www.sqlite.org/34to35.html http://www.sqlite.org/capi350ref.html Richard, The API function sqlite3_vfs_find() is documented as: The *sqlite3_vfs_find()* API is used to locate a particular VFS by name. Its prototype is as follows: sqlite3_vfs *sqlite3_vfs_find(const char *zVfsName); The argument is the symbolic name for the desired VFS. If the argument is a NULL pointer, then the default VFS is returned. The function returns a pointer to the *sqlite3_vfs* object that implements the VFS. Or it returns a NULL pointer if no object could be found that matched the search criteria. I wonder if it might not be better to change this API to accept an empty string, in addition to a NULL pointer, to find the default VFS. It seems to me this might make life easier for those writing wrappers in languages that don't have a concept of a NULL pointer. Dennis Cote - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Towards SQLite version 3.5.0
--- Dennis Cote [EMAIL PROTECTED] wrote: sqlite3_vfs *sqlite3_vfs_find(const char *zVfsName); The argument is the symbolic name for the desired VFS. If the argument is a NULL pointer, then the default VFS is returned. The function returns a pointer to the *sqlite3_vfs* object that implements the VFS. Or it returns a NULL pointer if no object could be found that matched the search criteria. I wonder if it might not be better to change this API to accept an empty string, in addition to a NULL pointer, to find the default VFS. It seems to me this might make life easier for those writing wrappers in languages that don't have a concept of a NULL pointer. That's an issue for the wrapper language binding, not the language itself. The wrapper language binding can call a null pointer whatever it wants. Take the Internet to Go: Yahoo!Go puts the Internet in your pocket: mail, news, photos more. http://mobile.yahoo.com/go?refer=1GNXIC - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Towards SQLite version 3.5.0
Dennis Cote [EMAIL PROTECTED] wrote: I wonder if it might not be better to change this API to accept an empty string, in addition to a NULL pointer, to find the default VFS. It seems to me this might make life easier for those writing wrappers in languages that don't have a concept of a NULL pointer. I'll meet you half way. I have changed the sqlite3_vfs_register() documentation to make the behavior undefined if you try to register a VFS with a name that an empty string. This way, you can be certain that no valid VFS has an empty string as its name. And your wrapper can map empty string into NULL. -- D. Richard Hipp [EMAIL PROTECTED] - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Towards SQLite version 3.5.0
The sqlite3_exec function has resided in the legacy.c file since shift to version 3. The function is still around in current v3.5 draft as http://www.sqlite.org/capi350ref.html says. That function is still kept around for compatibility reasons, as it was a important function in SQLite v2 days. On the other side it is quite handy to execute e.g. create, and generally for queries that modify the database and don't return data. Will it stay around for the whole life-span of version 3? Currently, I prefer to avoid sqlite3_exec function, as it's in the legacy.c file and I am unsure about its future. The official SQLite 3 API documentation says nothing about the legacy status of the sqlite3_exec function: http://www.sqlite.org/capi3ref.html#sqlite3_exec ... nor any info in the source code, and I haven't found any information about it beside a short footnote in The Definitive Guide to SQLite book and the legacy.c filename in the source code. My propose, either move function out of legacy file or add some legacy information text to API documentation (or at least to the source code) and about its planned life span. About mutex usage, the following page http://www.sqlite.org/34to35.html mentions : [...] The SQLite source code provides multiple implementations of these APIs, suitable for varying environments. [...] [...] Embedded applications may wish to provide their own mutex implementation. [...] For Unix, the POSIX Pthreads specification supports mutexes. The four basic functions are as follows: * pthread_mutex_init * pthread_mutex_destory * ptrhead_mutex_lock * pthread_mutex_unlock In Win32 world, the same can be solved by using Win32 Mutex functions or CRITICAL_SECTIONS (beside others): * CreateMutex * ReleaseMutex * OpenMutex SQLite 3 already use Win32 Mutex for WinCE (file) locking (os_win.c file). Most of that code could be reused for WinNT Win32 API (with the help of some macros, because of the wince part of the WinAPI function names). Will SQLite 3.5 use operating system related mutex API or does it ship with its own mutex implemented functions (I have seen some related code, but haven't investigated further)? - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Towards SQLite version 3.5.0
On Aug 28, 2007, at 4:51 PM, Dennis Cote wrote: I wonder if it might not be better to change this API to accept an empty string, in addition to a NULL pointer, to find the default VFS. It seems to me this might make life easier for those writing wrappers in languages that don't have a concept of a NULL pointer. Dennis Cote Just pass 0 in that case. --- Marco Bambini http://www.sqlabs.net http://www.sqlabs.net/blog/ http://www.sqlabs.net/realsqlserver/ - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Towards SQLite version 3.5.0
Would this be a good time to replace the older sqlite3_prepare() and sqlite3_prepare16() interfaces with what is currently the newer sqlite3_prepare_v2() and sqlite3_prepare16_v2() interfaces respectively? Admittedly they are definitely not among the less frequently used interfaces, but such an incompatible change to existing interfaces would be best done in such a version number jump if it is ever intended. Regards, Eugene Wee [EMAIL PROTECTED] wrote: The transition from 3.4.2 to 3.5.0 will perhaps be the largest single change to SQLite since 2.8-3.0. There will not be that many visible changes, but a lot is changing behind the scenes. Some less frequently used interfaces will be changing in slightly incompatible ways. Users who have build customized OS intereface layers or backends for SQLite will find that they are going to need to do some rework. SQLite version 3.5.0 is not close to being ready yet. But it is to the point where the source code will compile and pass many tests. And so I would like to take this opportunity to encourage people in the community to download the CVS HEAD and give it a whirl in their applications. Please let me know about any serious issues you run across. I have *started* to prepare documentation describing the changes in 3.5.0. This is draft documentation. But for those who are interested, please visit http://www.sqlite.org/34to35.html http://www.sqlite.org/capi350ref.html In particular, if your application uses a customized OS interface for SQLite, you should read the 34to35.html document to see exactly what will be involved in porting your application to run with version 3.5.0. The SQLite code currently in CVS HEAD is not ready for production use. We know that. We know what many of the problems are and Dan and I are working long hours to fix them. It's the problems that we *do not* know about that are scary. So that is why I am inviting the larger community to have an early look and perhaps bring our attention to issues sooner rather than later. -- D. Richard Hipp [EMAIL PROTECTED] - To unsubscribe, send email to [EMAIL PROTECTED] - - To unsubscribe, send email to [EMAIL PROTECTED] -
[sqlite] Towards SQLite version 3.5.0
The transition from 3.4.2 to 3.5.0 will perhaps be the largest single change to SQLite since 2.8-3.0. There will not be that many visible changes, but a lot is changing behind the scenes. Some less frequently used interfaces will be changing in slightly incompatible ways. Users who have build customized OS intereface layers or backends for SQLite will find that they are going to need to do some rework. SQLite version 3.5.0 is not close to being ready yet. But it is to the point where the source code will compile and pass many tests. And so I would like to take this opportunity to encourage people in the community to download the CVS HEAD and give it a whirl in their applications. Please let me know about any serious issues you run across. I have *started* to prepare documentation describing the changes in 3.5.0. This is draft documentation. But for those who are interested, please visit http://www.sqlite.org/34to35.html http://www.sqlite.org/capi350ref.html In particular, if your application uses a customized OS interface for SQLite, you should read the 34to35.html document to see exactly what will be involved in porting your application to run with version 3.5.0. The SQLite code currently in CVS HEAD is not ready for production use. We know that. We know what many of the problems are and Dan and I are working long hours to fix them. It's the problems that we *do not* know about that are scary. So that is why I am inviting the larger community to have an early look and perhaps bring our attention to issues sooner rather than later. -- D. Richard Hipp [EMAIL PROTECTED] - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Towards SQLite version 3.5.0
The documentation looks great! I look forward to the release of 3.5.0. A couple of questions: (1) In 3.5.0, a database cache can be shared by two different threads having two different database connections. I infer from this that two different threads can now share a cache without having to utilize a client/server mechanism. True? (2) The presence of the new mutex subsystem, and the absence of xReadLock(), xWriteLock(), and xUnlock() methods in the new VFS object, would seem to indicate that SQLite has abandoned the use of file locking to synchronize access to a shared database. True? (I hope so; it means that we embedded developers don't have to spend precious development time emulating those file locking calls.) All in all, 3.5.0 looks to be a major improvement over the current design. Any idea when it might be ready? Regards, - Richard [EMAIL PROTECTED] wrote: The transition from 3.4.2 to 3.5.0 will perhaps be the largest single change to SQLite since 2.8-3.0. There will not be that many visible changes, but a lot is changing behind the scenes. Some less frequently used interfaces will be changing in slightly incompatible ways. Users who have build customized OS intereface layers or backends for SQLite will find that they are going to need to do some rework. SQLite version 3.5.0 is not close to being ready yet. But it is to the point where the source code will compile and pass many tests. And so I would like to take this opportunity to encourage people in the community to download the CVS HEAD and give it a whirl in their applications. Please let me know about any serious issues you run across. I have *started* to prepare documentation describing the changes in 3.5.0. This is draft documentation. But for those who are interested, please visit http://www.sqlite.org/34to35.html http://www.sqlite.org/capi350ref.html In particular, if your application uses a customized OS interface for SQLite, you should read the 34to35.html document to see exactly what will be involved in porting your application to run with version 3.5.0. The SQLite code currently in CVS HEAD is not ready for production use. We know that. We know what many of the problems are and Dan and I are working long hours to fix them. It's the problems that we *do not* know about that are scary. So that is why I am inviting the larger community to have an early look and perhaps bring our attention to issues sooner rather than later. -- D. Richard Hipp [EMAIL PROTECTED] - To unsubscribe, send email to [EMAIL PROTECTED] - - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Towards SQLite version 3.5.0
ranlib .libs/libsqlite3.a creating libsqlite3.la (cd .libs rm -f libsqlite3.la ln -s ../libsqlite3.la libsqlite3.la) ./libtool --mode=link gcc -g -O2 -I. -I../src -DNDEBUG -DTHREADSAFE=1 -DSQLITE_THREAD_OVERRIDE_LOCK=-1 -DSQLITE_OMIT_LOAD_EXTENSION=1 -DHAVE_READLINE=1 -I/usr/include/readline -lpthread \ -o sqlite3 ../src/shell.c libsqlite3.la \ -lreadline -lreadline gcc -g -O2 -I. -I../src -DNDEBUG -DTHREADSAFE=1 -DSQLITE_THREAD_OVERRIDE_LOCK=-1 -DSQLITE_OMIT_LOAD_EXTENSION=1 -DHAVE_READLINE=1 -I/usr/include/readline -o .libs/sqlite3 ../src/shell.c ./.libs/libsqlite3.so -lpthread -lreadline /tmp/ccFQJzGR.o: In function `dump_callback': ../src/shell.c:762: undefined reference to `sqlite3_free' /tmp/ccFQJzGR.o: In function `run_schema_dump_query': ../src/shell.c:837: undefined reference to `sqlite3_free' /tmp/ccFQJzGR.o: In function `do_meta_command': ../src/shell.c:1431: undefined reference to `sqlite3_free' ../src/shell.c:1491: undefined reference to `sqlite3_free' ../src/shell.c:1147: undefined reference to `sqlite3_free' /tmp/ccFQJzGR.o:../src/shell.c:1670: more undefined references to `sqlite3_free' follow ./.libs/libsqlite3.so: undefined reference to `sqlite3_mutex_leave' ./.libs/libsqlite3.so: undefined reference to `sqlite3_mutex_try' ./.libs/libsqlite3.so: undefined reference to `sqlite3_mutex_free' ./.libs/libsqlite3.so: undefined reference to `sqlite3_memory_alarm' ./.libs/libsqlite3.so: undefined reference to `sqlite3_malloc' ./.libs/libsqlite3.so: undefined reference to `sqlite3_realloc' ./.libs/libsqlite3.so: undefined reference to `sqlite3_memory_used' ./.libs/libsqlite3.so: undefined reference to `sqlite3_mutex_alloc' ./.libs/libsqlite3.so: undefined reference to `sqlite3_mutex_enter' collect2: ld returned 1 exit status make: *** [sqlite3] Error 1 [EMAIL PROTECTED] wrote: The transition from 3.4.2 to 3.5.0 will perhaps be the largest single change to SQLite since 2.8-3.0. There will not be that many visible changes, but a lot is changing behind the scenes. Some less frequently used interfaces will be changing in slightly incompatible ways. Users who have build customized OS intereface layers or backends for SQLite will find that they are going to need to do some rework. SQLite version 3.5.0 is not close to being ready yet. But it is to the point where the source code will compile and pass many tests. And so I would like to take this opportunity to encourage people in the community to download the CVS HEAD and give it a whirl in their applications. Please let me know about any serious issues you run across. I have *started* to prepare documentation describing the changes in 3.5.0. This is draft documentation. But for those who are interested, please visit http://www.sqlite.org/34to35.html http://www.sqlite.org/capi350ref.html In particular, if your application uses a customized OS interface for SQLite, you should read the 34to35.html document to see exactly what will be involved in porting your application to run with version 3.5.0. The SQLite code currently in CVS HEAD is not ready for production use. We know that. We know what many of the problems are and Dan and I are working long hours to fix them. It's the problems that we *do not* know about that are scary. So that is why I am inviting the larger community to have an early look and perhaps bring our attention to issues sooner rather than later. -- D. Richard Hipp - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Towards SQLite version 3.5.0
Richard Klein [EMAIL PROTECTED] wrote: The documentation looks great! I look forward to the release of 3.5.0. A couple of questions: (1) In 3.5.0, a database cache can be shared by two different threads having two different database connections. I infer from this that two different threads can now share a cache without having to utilize a client/server mechanism. True? Correct. (2) The presence of the new mutex subsystem, and the absence of xReadLock(), xWriteLock(), and xUnlock() methods in the new VFS object, would seem to indicate that SQLite has abandoned the use of file locking to synchronize access to a shared database. True? False. The names of the methods are xLock() and xUnlock() and they are found in the sqlite3_io_methods structure. -- D. Richard Hipp [EMAIL PROTECTED] - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Towards SQLite version 3.5.0
--- [EMAIL PROTECTED] wrote: The transition from 3.4.2 to 3.5.0 will perhaps be the ... The SQLite code currently in CVS HEAD is not ready for production use. We know that. We know what many of the problems are and Dan and I are working long hours to fix them. It's the problems that we *do not* know about that are scary. So that is why I am inviting the larger community to have an early look and perhaps bring our attention to issues sooner rather than later. Just to clarify, you don't want to see make test failure tickets at this time? Take the Internet to Go: Yahoo!Go puts the Internet in your pocket: mail, news, photos more. http://mobile.yahoo.com/go?refer=1GNXIC - To unsubscribe, send email to [EMAIL PROTECTED] -
