Re: [sr-dev] [kamailio] MS ODBC Driver crashes kamailio on boot (#501)

Its weird it works absolutely fine when i have TLS disabled in kamailio, and is a lot faster than freetds which works alright apart from an occasional hung connection which requires a kamailio restart to recover from (no kamailio processes hang when this happens). It also works fine with any

Re: [sr-dev] [kamailio] MS ODBC Driver crashes kamailio on boot (#501)

What triggers the issue could be in Kamailio, but it is hard to figure out with no clue of what the driver doesn't like. The parameters should be ok, as freetds goes on fine. Kamailio initiates libssl with its own shared memory manager, which then is reused by other libs linked to libssl, as

Re: [sr-dev] [kamailio] MS ODBC Driver crashes kamailio on boot (#501)

The first core is relevant, the other one is because kamailio tries to shut down due to other process crash, but ends in a deadlock, likely because the process that crashed didn't release a lock. In the fist core, the problem seems to be in the odbc driver - the backtrace leaves the code of

Re: [sr-dev] [kamailio] MS ODBC Driver crashes kamailio on boot (#501)

Ok, i have pulled the latest commits from master and rebuilt. Here is the backtrace, it appears that latest master no longer emits lock tracing information when debug is set to level 3. No errors or alerts are logged either. Im starting kamailio as root with the following command: `kamailio -f

Re: [sr-dev] [kamailio] MS ODBC Driver crashes kamailio on boot (#501)

Got distracted by traveling and other tasks... The last logs point to a crash in a kamailio process: ``` Feb 13 12:57:46 vs-kam-uat01 /usr/local/sbin/kamailio[51225]: ALERT: [main.c:739]: handle_sigs(): child process 51256 exited by a signal 11 ``` Can you do 'ulimit -c unlimited' before

Re: [sr-dev] [kamailio] MS ODBC Driver crashes kamailio on boot (#501)

Keen to resolve this issue, is there anything else i can do my end to try and trackdown where the deadlock is occurring? --- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/501#issuecomment-196551712___

Re: [sr-dev] [kamailio] MS ODBC Driver crashes kamailio on boot (#501)

Is there any additional analysis i can do on the log files to try and narrow down where the problem lies? --- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/501#issuecomment-187668002___ sr-dev mailing

Re: [sr-dev] [kamailio] MS ODBC Driver crashes kamailio on boot (#501)

You need to recreate the active watcher table and update its version number in 'version' table -- there is an error: ``` Feb 12 10:26:01 vs-kam-uat01 /usr/local/sbin/kamailio[35513]: ERROR: [db.c:439]: db_check_table_version(): invalid version 11 for table active_watchers found, expected 12

Re: [sr-dev] [kamailio] MS ODBC Driver crashes kamailio on boot (#501)

Doh, silly mistake to make, never mind, have rerun with updated table definitions, same behavior, redacted log is attached in a zip as it was over 15Mb uncompressed. [kam-tls-debugging.log.13-02-16.zip](https://github.com/kamailio/kamailio/files/129144/kam-tls-debugging.log.13-02-16.zip) ---

Re: [sr-dev] [kamailio] MS ODBC Driver crashes kamailio on boot (#501)

Sure no worries will post back the results, i have also tested changing the threading model used by UNIXODBC which has had no impact on the issue, so will see what the latest master shows :) --- Reply to this email directly or view it on GitHub:

Re: [sr-dev] [kamailio] MS ODBC Driver crashes kamailio on boot (#501)

Log file is too big for pastebin (8.5Mb) so i have attached it to this post [kam-tls-debugging.log.txt](https://github.com/kamailio/kamailio/files/127835/kam-tls-debugging.log.txt) --- Reply to this email directly or view it on GitHub:

Re: [sr-dev] [kamailio] MS ODBC Driver crashes kamailio on boot (#501)

It appears to be a deadlock -- somewhere, a component does two times lock on a mutex set for libcrypto. I added some debug messages in master branch, can you test with it? First set debug=3 in config file, then start and make the log message from syslog available somewhere (you can mask

Re: [sr-dev] [kamailio] MS ODBC Driver crashes kamailio on boot (#501)

Same issue with the ubuntu compiled v13 driver, everything works fine with the tls module disabled, hangs when its enabled. --- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/501#issuecomment-182319382___

Re: [sr-dev] [kamailio] MS ODBC Driver crashes kamailio on boot (#501)

Latest backtrace, made using the ubuntu compiled version of the driver, as frame 19 is the same as previous backtrace included the requested info below. ``` gdb kamailio /core GNU gdb (Debian 7.7.1+dfsg-5) 7.7.1 Copyright (C) 2014 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3

Re: [sr-dev] [kamailio] MS ODBC Driver crashes kamailio on boot (#501)

Just disabled the TLS module and its working, unfortunatly TLS is a requirement i cannot live without... I wonder if this could be caused by a symlink that has to be created to get this version of the driver to load, as its compiled for redhat (further details here:

Re: [sr-dev] [kamailio] MS ODBC Driver crashes kamailio on boot (#501)

You can always replace the sensitive information with generic tokens such as USER, HOST, PASSWORD, IP1, etc... I am more interested where is invalid data rather than valid values. Can you try without tls module? --- Reply to this email directly or view it on GitHub:

Re: [sr-dev] [kamailio] MS ODBC Driver crashes kamailio on boot (#501)

TLS module is loaded, and when i issue the commands you asked for above it displays the connection string, which i cant really share publically. I have also rebuilt the build box in case it was an issue of conflicting versions. Behavious is slightly different now rather than an outright crash

Re: [sr-dev] [kamailio] MS ODBC Driver crashes kamailio on boot (#501)

Can you do in gdb: ``` frame 19 p *id p ptr p *ptr p con_str info locals ``` Is tls module of kamailio loaded? --- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/501#issuecomment-181417562___ sr-dev

Re: [sr-dev] [kamailio] MS ODBC Driver crashes kamailio on boot (#501)

**Backtrace** ``` gdb kamailio /core GNU gdb (Debian 7.7.1+dfsg-5) 7.7.1 Copyright (C) 2014 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the

[sr-dev] [kamailio] MS ODBC Driver crashes kamailio on boot (#501)

Hi im currently using freetds via unixodbc to connect to a sql azure db which is ok but suffers from various connection busy errors. I would like to use the ms odbc driver as it supports MARS (multiple active result sets) which should stop the connection busy errors, and numerous other sql

Re: [sr-dev] [kamailio] MS ODBC Driver crashes kamailio on boot (#501)

Do you get a coredump file? If yes, paste here the backtrace taken with gdb. --- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/501#issuecomment-181085754___ sr-dev mailing list sr-dev@lists.sip-router.org