Markus, Unfortunately, I have no experience maintaining code for packages. The best I can do is a few lines in a Makefile, and I don't understand automake scripts at all. I can fix all sorts of things in c source code, but I would likely break the test and build process. So I wouldn't trust me as a maintainer. Thanks for your work on libdbi over the years. I have been using it for well over a decade. Ethan... On Tue, 2019-11-19 at 00:20 +0100, Markus Hoenicka wrote: > Hi Ethan and anyone else who might be listening, > I'm sorry for the sluggish response time. I'm afraid I'm the one > you're looking for. I used to maintain libdbi and the drivers for a > couple of years while I was able to do some programming. I continued > as sort of a caretaker for another while even though there was no > time left for pretty much anything outside my dayjob. I still use > libdbi everyday but I'd have to read the manual how to check in a > patch these days. So long story short - if anyone is willing to take > over development, or at least to act as a caretaker, please step > forward. If you have the skills to check in and test your patch, let > me know. I can add you as a developer to the project. > regards,Markus > On 2019-11-16 21:09, Ethan Funk was heard to say: > > I found and tested a fix for the mysql thread safety problem I > > havebeen troubleshooting. > > Commenting out line 154 of dbd_mysql.c succeeds in fixing the > > problemwhen libdbi-mysql is build against recent libmysqlclient > > libraries.This is the were mysql_server_end() is called after each > > dbi instanceif finished. In a multi-threaded application, multiple > > dbi instancescan come and go in various threads. With recent > > libmysqlclient libraryimplementations, mysql_server_end() should > > only be called ONCE, at theend of the entire program/process, not > > multiple times when a thread isfinished with a dbi intsnace. > > libmysqlclient's mysql_server_end() function performs some > > finallibmysqlclient memory clean-up. So if it isn't called, and > > theapplication ends, it's not really a problem - the OS will clean > > up thememory allocation. > > A few other thread safety notes that might be helpful to others. To > > bethread safe: > > 1. Each program thread must use it's own libdbi instance > > (asdocumented in libdbi). > > 2. The dbi_conn_connect function must be protected by a mutex, > > sinceit results in an underlying libmysqlclient call that is not > > threadsafe, using memory objects allocated once by the > > libmysqlclientlibrary when it was first initialized. This is > > exactly the memory thatwas causing trouble (freed) when > > mysql_server_end() is called bylibdbi_shutdown_r in onw thread when > > the libdbi library is still inuse by other threads prior to program > > termination. > > So... is libdbi and the database drivers still maintained by > > someone?How can I get this fix included so it will be present when > > folksinstall the driver for mysql on their various systems? > > Thanks,Ethan... > > On Fri, 2019-11-15 at 09:35 -0700, Ethan Funk wrote: > > > I think I found the problem in the mysql dbd driver code.: > > > dbd_finalize() calls mysql_server_end(), which is deprecated in > > > the > > > current mysqlclient21 library, but results in a call to > > > areplacement > > > function which is intended to be called only once, at the end of > > > a > > > process. With the call being made inside dbd_finalize(), the > > > mysql_server_end() gets called each time my application > > > codefinished > > > with a libdbi instance. Reading between the lines in the MySQL > > > documentation, it looks think the mysql_server_end() call to > > > it'snew > > > counter part frees memory which is allocated by a pthread_once() > > > function triggered by mysql_library_init(), which itself is > > > calledby > > > mysql_init() via the libdbi driver on each connection > > > attempt. Somy > > > code's first libdbi connection results in the memory > > > beingallocated, > > > with subsequent calls in other threads being ignored. But > > > dbd_finalize() gets called at the end of EVERY libdbi > > > instanceclosure, > > > so the memory if freed over and over again. I don't see a > > > mechanism > > > for a once-per-process driver shutdown callback in libdbi, so I > > > wantto > > > try commenting out mysql_servefr_end() in dbd_finalize(). This > > > will > > > leak memory which is never freed at the end of the process, but > > > so > > > what. It is only allocated once per process, so the > > > operatingsystem > > > process shutdown will free the memory. > > > Unfortunately, I can't seem to build my modified code for > > > testingwith > > > the libdi_dbd project code. "./configure" is giving me the > > > following > > > error: > > > configure: error: Invalid libdbi directory - include files > > > notfound. > > > This is after configure has already indicated that yes, it > > > haslocated > > > the dbi header files, which I installed with the libdbi-dev > > > package > > > with the Ubuntu apt command. > > > How might I be able to fix this error, and proceed with testing > > > mycode > > > fix? > > > Thanks, > > > Ethan... > > > On Wed, 2019-11-13 at 14:19 -0700, Ethan Funk wrote: > > > > I am having a problem with libdbd_mysql on linux (Ubuntu 19.10) > > > > reagding thread fasety. > > > > There used to be two versions of libmysqlclient, one was > > > > threadsafe, > > > > one was not. However, > > > > now there seems to be only one client library, which I > > > > understoodwas > > > > thread safe: libmysqlclient.so.21. > > > > Any insight into the dump below would be greatly appreciated! > > > > Ethan... > > > > Here is a valgrind dump of a read-already-freed error that > > > > occursin > > > > what I will call thread #3 > > > > when it attempts to connect to MySQL. This thread has it's > > > > ownlibdbi > > > > instance, yet mysql_real_connect > > > > is trying to read memory allocated by the first thread (thread > > > > #1)that > > > > first called mysql_real_connect, > > > > after that same memory was freed by another thread (thread > > > > #2)that > > > > called mysql_server_end as it was > > > > finishing. Each thread has it's own libdbi instance, so why is > > > > libmysqlclient sharring memory between > > > > threads? It's looking like the libmysqlclient isn't thread > > > > safe. > > > > ==2828== Invalid read of size 1 > > > > ==2828== at 0x4A3A042: strlen (vg_replace_strmem.c:461) > > > > ==2828== by 0x141A46C1: ??? (in /usr/lib/x86_64-linux- > > > > gnu/libmysqlclient.so.21.1.17) > > > > ==2828== by 0x141A5468: ??? (in /usr/lib/x86_64-linux- > > > > gnu/libmysqlclient.so.21.1.17) > > > > ==2828== by 0x141A54A8: ??? (in /usr/lib/x86_64-linux- > > > > gnu/libmysqlclient.so.21.1.17) > > > > ==2828== by 0x141A373E: ??? (in /usr/lib/x86_64-linux- > > > > gnu/libmysqlclient.so.21.1.17) > > > > ==2828== by 0x4FE041E: __pthread_once_slow > > > > (pthread_once.c:116) > > > > ==2828== by 0x141A320D: ??? (in /usr/lib/x86_64-linux- > > > > gnu/libmysqlclient.so.21.1.17) > > > > ==2828== by 0x141A41B2: ??? (in /usr/lib/x86_64-linux- > > > > gnu/libmysqlclient.so.21.1.17) > > > > ==2828== by 0x141A429E: ??? (in /usr/lib/x86_64-linux- > > > > gnu/libmysqlclient.so.21.1.17) > > > > ==2828== by 0x1415472B: ??? (in /usr/lib/x86_64-linux- > > > > gnu/libmysqlclient.so.21.1.17) > > > > ==2828== by 0x1415492A: ??? (in /usr/lib/x86_64-linux- > > > > gnu/libmysqlclient.so.21.1.17) > > > > ==2828== by 0x141577A8: mysql_real_connect (in > > > > /usr/lib/x86_64- > > > > linux-gnu/libmysqlclient.so.21.1.17) > > > > ==2828== by 0x12F47435: dbd_connect (in /usr/lib/x86_64- > > > > linux- > > > > gnu/dbd/libdbdmysql.so) > > > > ==2828== by 0x4DC37AE: dbi_conn_connect (in/usr/lib/x86_64- > > > > linux- > > > > gnu/libdbi.so.1.1.0) > > > > ==2828== by 0x134BFA: dbSetupConnection (database.c:211) > > > > ==2828== by 0x1357F6: DeleteLogEntry (database.c:440) > > > > ==2828== by 0x131ECE: taskCallProcInThread (tasks.c:73) > > > > ==2828== by 0x4FD7668: start_thread (pthread_create.c:479) > > > > ==2828== by 0x567B322: clone (clone.S:95) > > > > Memory was freed in a thread (lets call it thread #2) that ran > > > > and > > > > completed. It created it's own > > > > dbi instance, did it's thing, and then closed it's > > > > MySQLconnection and > > > > shutdown the instance. > > > > So why is it freeing memory deep in the mysql driver that it > > > > didnot > > > > allocate? > > > > ==2828== Address 0x1343e698 is 424 bytes inside a block of > > > > size4,088 > > > > free'd > > > > ==2828== at 0x4A37F90: free (vg_replace_malloc.c:540) > > > > ==2828== by 0x141A9B8A: ??? (in /usr/lib/x86_64-linux- > > > > gnu/libmysqlclient.so.21.1.17) > > > > ==2828== by 0x141A8756: ??? (in /usr/lib/x86_64-linux- > > > > gnu/libmysqlclient.so.21.1.17) > > > > ==2828== by 0x14148982: mysql_server_end (in/usr/lib/x86_64- > > > > linux- > > > > gnu/libmysqlclient.so.21.1.17) > > > > ==2828== by 0x12F4684C: dbd_finalize (in /usr/lib/x86_64- > > > > linux- > > > > gnu/dbd/libdbdmysql.so) > > > > ==2828== by 0x4DC2E4D: dbi_shutdown_r (in/usr/lib/x86_64- > > > > linux- > > > > gnu/libdbi.so.1.1.0) > > > > ==2828== by 0x134440: dbi_instance_free (database.c:56) > > > > ==2828== by 0x4FD6600: __nptl_deallocate_tsd.part.0 > > > > (pthread_create.c:301) > > > > ==2828== by 0x4FD7689: > > > > __nptl_deallocate_tsd(pthread_create.c:256) > > > > ==2828== by 0x4FD7689: start_thread (pthread_create.c:490) > > > > ==2828== by 0x567B322: clone (clone.S:95) > > > > This is in the first thread in the program to make use > > > > oflibdbi/MySQL, > > > > and it is still > > > > running/connected to MySQL using a unique libdbi intance. > > > > ==2828== Block was alloc'd at > > > > ==2828== at 0x4A36E96: malloc (vg_replace_malloc.c:309) > > > > ==2828== by 0x141A9A30: ??? (in /usr/lib/x86_64-linux- > > > > gnu/libmysqlclient.so.21.1.17) > > > > ==2828== by 0x141A9AEB: ??? (in /usr/lib/x86_64-linux- > > > > gnu/libmysqlclient.so.21.1.17) > > > > ==2828== by 0x141A4E1C: ??? (in /usr/lib/x86_64-linux- > > > > gnu/libmysqlclient.so.21.1.17) > > > > ==2828== by 0x1419CFA0: ??? (in /usr/lib/x86_64-linux- > > > > gnu/libmysqlclient.so.21.1.17) > > > > ==2828== by 0x1419D95E: ??? (in /usr/lib/x86_64-linux- > > > > gnu/libmysqlclient.so.21.1.17) > > > > ==2828== by 0x1416FD29: ??? (in /usr/lib/x86_64-linux- > > > > gnu/libmysqlclient.so.21.1.17) > > > > ==2828== by 0x141A2A66: ??? (in /usr/lib/x86_64-linux- > > > > gnu/libmysqlclient.so.21.1.17) > > > > ==2828== by 0x141A3771: ??? (in /usr/lib/x86_64-linux- > > > > gnu/libmysqlclient.so.21.1.17) > > > > ==2828== by 0x4FE041E: __pthread_once_slow > > > > (pthread_once.c:116) > > > > ==2828== by 0x141A320D: ??? (in /usr/lib/x86_64-linux- > > > > gnu/libmysqlclient.so.21.1.17) > > > > ==2828== by 0x141A41B2: ??? (in /usr/lib/x86_64-linux- > > > > gnu/libmysqlclient.so.21.1.17) > > > > ==2828== by 0x141A429E: ??? (in /usr/lib/x86_64-linux- > > > > gnu/libmysqlclient.so.21.1.17) > > > > ==2828== by 0x1415472B: ??? (in /usr/lib/x86_64-linux- > > > > gnu/libmysqlclient.so.21.1.17) > > > > ==2828== by 0x1415492A: ??? (in /usr/lib/x86_64-linux- > > > > gnu/libmysqlclient.so.21.1.17) > > > > ==2828== by 0x141577A8: mysql_real_connect (in > > > > /usr/lib/x86_64- > > > > linux-gnu/libmysqlclient.so.21.1.17) > > > > ==2828== by 0x12F47435: dbd_connect (in /usr/lib/x86_64- > > > > linux- > > > > gnu/dbd/libdbdmysql.so) > > > > ==2828== by 0x4DC37AE: dbi_conn_connect (in/usr/lib/x86_64- > > > > linux- > > > > gnu/libdbi.so.1.1.0) > > > > ==2828== by 0x134BFA: dbSetupConnection (database.c:211) > > > > ==2828== by 0x13D2F4: GetItemMetaData (database.c:2960) > > > > ==2828== > > _______________________________________________libdbi-users mailing > > listlibdbi-us...@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/libdbi-users
signature.asc
Description: This is a digitally signed message part
_______________________________________________ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
