Re: [sr-dev] [kamailio/kamailio] TLSA Undefined Symbol __dlclose (#2815)
Additional notes with ubuntu 18.04 Thought i would try compiling against openssl 1.0.2u to see what effect that had. Although open ssl would compile correctly, tlsa would complain when trying to link against 1.02u. Solution was to re-rerun the openssl ./config script with the suggested compiler flag in the build error, then rebuild openssl and TLSA ``` test@kamtest:/usr/local/src/openssl-1.0.2u/openssl-1.0.2u$ ./config -fPIC test@kamtest:/usr/local/src/openssl-1.0.2u/openssl-1.0.2u$ make test@kamtest:/usr/local/src/openssl-1.0.2u/openssl-1.0.2u$ make test test@kamtest:/usr/local/src/openssl-1.0.2u/openssl-1.0.2u$ cd ../../kamailio test@kamtest:/usr/local/src/openssl-1.0.2u/openssl-1.0.2u$ make test@kamtest:/usr/local/src/openssl-1.0.2u/openssl-1.0.2u$ sudo make install ``` -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2815#issuecomment-898358632___ Kamailio (SER) - Development Mailing List sr-dev@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] ruxc automatic retry (#2820)
@miconda Agreed, have opened an issue in the ruxc repo, was more of an interim fix, but i think it makes absolute sense to have it at lib level -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2820#issuecomment-896769930___ Kamailio (SER) - Development Mailing List sr-dev@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] ruxc automatic retry (#2820)
Just had a bash at adding this myself, and it seems to work nicely for my usecase... Adding this changing calls to post or get, and defining a parameter for retry attempts ``` ruxc_http_get(_http_request, _http_response); //Retry enabled, either have a server error code or a connection error int attempt; attempt = 0; while(attempt < _ruxc_http_retry && (v_http_response.retcode < 0 || (v_http_response.rescode > 499 && v_http_response.rescode < 699)){ attempt++; LM_DBG("Retrying Request Attempt: %d - Response Code: %d - Return Code: %d - URI: %s\n", attempt, v_http_response.retcode, v_http_response.rescode, v_http_request.url); ruxc_http_get(_http_request, _http_response); } ``` Im loving ruxc lol -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2820#issuecomment-896719750___ Kamailio (SER) - Development Mailing List sr-dev@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
[sr-dev] [kamailio/kamailio] ruxc automatic retry (#2820)
### Description Could a module parameter be added to retry a request _n_ number of times in the event of a connection or server failure, without having to write a tedious bit of boiler plate config to catch failure response codes and retry. ### Expected behavior A http(s) post or get is performed and the returned code is < 0 but > -20 (null url return code in libruxc) or is a 5xx or 6xx http error code. The module will reattempt the request up to the number of attempts defined in a module parameter. By default this behaviour is disabled and would require explicitly defining the module parameter to enable it. Requests could be exempted from automatic retries by either adding an additional argument to the function called in config, or a http header could be supplied `X-RUXC-NORETRY` for example using the existing function signature to exclude the request. The header name could also be defined as a module parameter. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2820___ Kamailio (SER) - Development Mailing List sr-dev@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
[sr-dev] [kamailio/kamailio] ruxc connection debugging messages (#2819)
### Description Would it be possible to have the debug messages from libruxc entered into the kamailio logs, as thats much more useful than -1 for diagnosing connection failures, as well as the uri of the failing request? ### Expected behavior Some diagnostic data passed up from libruxc detailing connection failure. Actual observed behavior `ERROR: ruxc [ruxc_mod.c:151]: ki_ruxc_http_get_helper(): failed to perform http get - retcode: -1 uri: https://someservice.somewhere.com` -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2819___ Kamailio (SER) - Development Mailing List sr-dev@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] TLSA Undefined Symbol __dlclose (#2815)
Unfortunatly the additional switches made no effect, i have however found a work around... If you set the `LIBSSL_STATIC_SRCLIB` to yes, and alter the path to where ubuntu stores the static libs from packages it compiles with no warnings and loads as expected. ``` # set to yes when wanting to link with static libraries compiled from source LIBSSL_STATIC_SRCLIB ?= yes # set to the path of the folder with static libraries compiled from source #LIBSSL_STATIC_SRCPATH ?= /usr/local/src/openssl LIBSSL_STATIC_SRCPATH ?= /usr/lib/x86_64-linux-gnu/ ``` As an aside the emergence of this module and ruxc couldnt be more timely as i have been having no end of problems with random TLS failures which seem to be related to the new "locking" model, so if needs be being able to use an older version of libssl this is perfect, as is an alternative to http_client which has also been having random TLS failures of either accessing priv key or failing while trying to initialise the connection. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2815#issuecomment-895854682___ Kamailio (SER) - Development Mailing List sr-dev@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
[sr-dev] [kamailio/kamailio] TLSA Undefined Symbol __dlclose (#2815)
### Description Building TLSA module on ubuntu 18.04 results in a compiler warning and an undefined symbol error when attempting to load module. ### Troubleshooting Reproduction Make module and attempt to load it Debugging Data ``` N/A ``` Log Messages Compiler output ``` CC (gcc) [M tlsa.so]tls_util.o CC (gcc) [M tlsa.so]tls_domain.o CC (gcc) [M tlsa.so]tls_locking.o CC (gcc) [M tlsa.so]tls_rand.o CC (gcc) [M tlsa.so]tls_map.o CC (gcc) [M tlsa.so]tls_cfg.o CC (gcc) [M tlsa.so]tls_select.o CC (gcc) [M tlsa.so]tls_server.o CC (gcc) [M tlsa.so]tls_dump_vf.o CC (gcc) [M tlsa.so]tls_init.o CC (gcc) [M tlsa.so]tls_rpc.o CC (gcc) [M tlsa.so]tls_config.o CC (gcc) [M tlsa.so]tls_bio.o CC (gcc) [M tlsa.so]tls_ct_wrq.o CC (gcc) [M tlsa.so]tlsa_mod.o CC (gcc) [M tlsa.so]tls_verify.o LD (gcc) [M tlsa.so]tlsa.so /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libcrypto.a(dso_dlfcn.o): In function `dlfcn_globallookup': (.text+0x11): warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking ``` Kamailio Log ``` Aug 9 11:21:04 kamtest kamailio: ERROR: [core/sr_module.c:570]: load_module(): could not open module : /usr/local/lib64/kamailio/modules/tlsa.so: undefined symbol: __dlclose Aug 9 11:21:04 kamtest kamailio: CRITICAL: [core/cfg.y:3683]: yyerror_at(): parse error in config file /etc/kamailio/parameters/modules.cfg, line 7, column 12-20: failed to load module Aug 9 11:21:04 kamtest kamailio: ERROR: [core/modparam.c:181]: set_mod_param_regex(): No module matching found ``` SIP Traffic ``` N/A ``` ### Possible Solutions ### Additional Information * **Kamailio Version** - output of `kamailio -v` ``` version: kamailio 5.6.0-dev0 (x86_64/linux) 0a5307-dirty flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: 0a5307 -dirty compiled on 10:36:16 Aug 9 2021 with gcc 7.5.0 ``` * **Operating System**: ``` Linux kamtest 4.15.0-144-generic #148-Ubuntu SMP Sat May 8 02:33:43 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux ``` -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2815___ Kamailio (SER) - Development Mailing List sr-dev@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] TLS: same TLSc is used for different outbound connection when it is switched with tls_set_connect_server_id() (#2760)
@miconda it was a very naïve modification so it just created a new connection and never reused which was a horrible solution Re-reading through tcp_main.c I had a spark of inspiration when I noticed that `tcpconn_rm` checked if `c->extra_data` had a value other than 0 to determine if `tls_tcpconn_clean` needed calling. I have currently hacked an extra property to indicate sni force connection into the `tls_extra_data` struct, which gets set by calling `ksr_tls_set_connect_server_id`, an additional check is added to `_tcpconn_find` which checks if the connection is TLS or WSS and that the sni force new connection `extra_data` property is 1, now only requests which have a call to `tls_set_connect_server_id` force a new connection, otherwise connections are reused (in my limited testing so far). I havnt investigated if the flags property might be more appropriate. One thing that occurs to me with this solution is that other than the changes to the `tls_extra_data` struct the behaviour could be explicitly triggered with an additional argument on `tls_set_connect_server_id(str serverId, bool forceNew)` thus wouldnt effect any existing behaviours? I shall dig out the PR guidelines and get one raised ASAP, i suspect my fix is a bit too hairy a solution to make it, but im happy to contribute to the discussion. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2760#issuecomment-858713772___ Kamailio (SER) - Development Mailing List sr-dev@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] TLS: same TLSc is used for different outbound connection when it is switched with tls_set_connect_server_id() (#2760)
I would agree its a connection reuse issue, #1107 is probably a more relevant discussion to have. As an experiment I tried a modified version of tcp_main.c with the connection lookup modified to not return an existing connection and the expected behaviour was observed. Thinking the further the ideal solution from my point of view would be a pair of config values, one to enable a enhanced matching mechanism, and a complementary value list of ips which need to be outbound sni aware and takes the tls profile into account when a configured ip is matched as the destination. Unfortunately this would be non trivial as correct me if I'm wrong but the tls profile is attached after the connection is created so a chicken and egg problem? I agree this isnt vast majority use case territory but would be invaluable for offering hosted services and allows me to keep the rest of the architecture and network config the same. As surely an outbound connection with a different servername specified should be treated as distinct connections even if the destination is the same? The issue I have with aliases is that I would swap the admin burden of adding sans to the ssl and doing a tls.reload to having to restart the service, and increase the number of open ports at the edge, basically its not scalable enough, I can just about cope with slow tls reload when you have more than one cert loaded (trivial load test with 30 certs took over 60 secs to reload) The whole impetus for trying to use sni with kamailio would be offer a scalable, automatable provisioning for direct routing customers, which doesn't have a restriction on number of sans defined on the cert (think simplicity of adding a vhost in nginx). Currently provisioning can be done live by expanding a cert, updating the dispatcher with new entries and reloading tls and dispatcher with kamcmd. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2760#issuecomment-858396771___ Kamailio (SER) - Development Mailing List sr-dev@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] TLS: same TLSc is used for different outbound connection when it is switched with tls_set_connect_server_id() (#2760)
Debug output attached, in the full version from about line 8050 is the dispatcher activity [kamailio-debug-dispatcher-sni-full.log](https://github.com/kamailio/kamailio/files/6623779/kamailio-debug-dispatcher-sni-full.log) [kamailio-debug-dispatcher-sni-dispatcher-activity.log](https://github.com/kamailio/kamailio/files/6623780/kamailio-debug-dispatcher-sni-dispatcher-activity.log) -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2760#issuecomment-857659048___ Kamailio (SER) - Development Mailing List sr-dev@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] TLS: same TLSc is used for different outbound connection when it is switched with tls_set_connect_server_id() (#2760)
Certainly, i spent yesterday debugging and going through the logs and im thinking the issue may be the (ab)use of dispatcher to ping the same destination presenting as different domains. The destinations are intentially the same in the sample dispatcher.list in the first post ```1 sip:sip1.host.com;transport=tls 0 1 socket=tls:111.222.233.11:5061;ping_from=sip:my-domain-01.com 1 sip:sip2.host.com;transport=tls 0 2 socket=tls:111.222.233.12:5061;ping_from=sip:my-domain-01.com 1 sip:sip3.host.com;transport=tls 0 3 socket=tls:111.222.233.13:5061;ping_from=sip:my-domain-01.com 2 sip:sip1.host.com;transport=tls 0 1 socket=tls:111.222.233.21:5061;ping_from=sip:my-domain-02.com 2 sip:sip2.host.com;transport=tls 0 2 socket=tls:111.222.233.22:5061;ping_from=sip:my-domain-02.com 2 sip:sip3.host.com;transport=tls 0 3 socket=tls:111.222.233.23:5061;ping_from=sip:my-domain-02.com``` What we are both trying to achieve is I believe integration with Teams using SNI to provide connectivity to multiple domain names. I have been running in production for a couple of years now a solution based on using a single cert and lots of SAN's but that i becoming a management nightmare and using a cert per domain served is far preferable. DR (Direct Routing) has a primary signalling domain and two failover domains (hence dispatcher is ideal for this). Connectivity is monitored by mutually verifying the TLS connection where the Server (kamailio) must present a valid cert whose SN or SAN must match the expected SNI server name from the client (DR SBC), for inbound traffic this works fine, but on outbound traffic the same rules apply meaning that each domain we wish to serve has to have its own connection which has been initialised to utilise the corresponding [client:any] entry with the appropriate server name configured. DR's second level connection state involves mutually pinging each end, but DR only sends pings on the receipt of valid ping from the kamailio, which doesnt happen as my-domain02.com presents as my-domain01.com. I added some additional LM_DBG statements to the TLS module as well as compiling with the TLS_WR_DEBUG and TLS_RD_DEBUG defines enabled to help track the flow through the code initially i noticed calls to `ksr_tls_set_connect_server_id` were not always setting `_ksr_tls_connect_server_id` so i commented out the `if(_ksr_tls_connect_server_id.len>=srvid->len)` block and the value was consistently set. However i was seeing the TLS connection initalise only 3 times, using the set server ID for the first dispatcher set, and never the second, even though i could see clearly the server id being set, the logs jump straight to writing over the existing connection. Which leads me to believe as the destination (and probably sending port) is the same for each dispatcher address set then the connection will be reused. (#1107 seems to be an accurate description of what i have observed). Which leads me to wonder; if an existing matching connection needs to be closed when the server id is set by `ksr_tls_set_connect_server_id` or if an additional connection matching mode could be implemented which took a from address into consideration whilst matching. I will revert my changes and post the log output here -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2760#issuecomment-857528638___ Kamailio (SER) - Development Mailing List sr-dev@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] TLS: same TLSc is used for different outbound connection when it is switched with tls_set_connect_server_id() (#2760)
@arkadiam Pretty sure we are both trying to do same thing judging by your dispatcher and routing config, multi domain telephony integration with a cloud comms platform? Its frustrating that SNI is being flakey as it works very nicely if you have a single cert with multiple SAN's defined but that less than ideal from a management point of view (but doesnt seem to have the performance hit loading multiple certs has #2312 ) For sake of completeness my virtually identical config is as follows: ```event_route[tm:local-request] { if(is_method("OPTIONS") && $ru =~ "vendor.com") { xlogl("L INFO", "Dispatcher Pinging ruri: $ru turi: $tu furi: $fu"); append_hf("Contact: \r\n"); tls_set_connect_server_id($fd); xlogl("L INFO","SNI ID: $fd"); } } ``` -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2760#issuecomment-854463465___ Kamailio (SER) - Development Mailing List sr-dev@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] TLS: same TLSc is used for different outbound connection when it is switched with tls_set_connect_server_id() (#2760)
I was half way through writing a less detailed version of a very similar issue i am having too, having to send options presenting a specific servername and cert same environment as well (ubuntu 18.04, kam 5.5.0) I have noticed that the xavp sni settings seems to only work on one thread as well if you loop the message back and set via request_route -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2760#issuecomment-854073385___ Kamailio (SER) - Development Mailing List sr-dev@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] Dispatcher SNI support (#2591)
No afraid not, I need to send all traffic to a destination using a specific tls client config, including dispatcher pings, that only sets the from header of the options message, in a request route I just set the tls xavp with suitable values to force a specific tls client profile. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2591#issuecomment-747739342___ Kamailio (SER) - Development Mailing List sr-dev@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
[sr-dev] [kamailio/kamailio] Dispatcher SNI support (#2591)
Is it possible to specify a specific server name/server id for dispatcher to use to send pings with (and in outgoing requests) It seems I can't (or haven't noticed in the docs) an event route I could use to set the xavp, as tm:local-request happens to late to use there. As an aside why isn't there a "last chance before send" event route? This is probably a feature request more than a bug, as the ideal solution from my point of view would be an additional dispatcher attrs value to be able to set it per destination in the underlying dispatcher data Judging by the output from tls module dbg the tls xavps are checked, before the ping is sent.. I've had a cursery glance at dispatch.c and it seems easy enough to add an extra attr parameter but I'm ata loss as to how I would preset the server name/id xavp... (I'm no c dev, but can Google quick enough to follow along mostly, and happy to fiddle away if anyone can be kind enough to give me some pointers in the right direction) -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2591___ Kamailio (SER) - Development Mailing List sr-dev@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] DB_UNIXODBC always changes language and context (#1993)
Originally i had everything being backed by ODBC tables, but could never get the hangs to stop, usually presenting it self as an inability to auth registrations randomly. I think i also stumbled upon some hard coded MySQL syntax specific queries in various modules typically using the MySQL specific REPLACE (upsert) keyword. Getting around the lack of USE keyword validity in SQL Azure was acheived like i said by specifying on the server login and user entities rather than in the connection string. As a result of the random nature of the hangs we re-architected to using intermediate API for our dynamic data access and for infrequently changing data (dispatcher for example) generated textdb files, when a change is submitted through the back end. At its simplest we just set a flag saying file has changed, and poll the api, if the flag is set local copy is overritten by the downloaded generated file, and the data gets reloaded via kamcmd. Since having done that all our relational data stays in sql azure, but is never directly accessed by kamailio, and have not had a single hang like we were experiencing. Like i said the odd thing was if i pointed the connection string to a local SQL server it would work perfectly, just undermined our long established SQL Azure consolidation project... -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/1993#issuecomment-505899274___ Kamailio (SER) - Development Mailing List sr-dev@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
Re: [sr-dev] [kamailio/kamailio] DB_UNIXODBC always changes language and context (#1993)
I worked around this issue (it's more a sql azure thing than kamailio) by creating a user on the dB and setting default language, db and schema on the login that was enough to get it to connect. Be warned though using sql azure with either freetds or the ms Linux driver I experienced random hangs of kamailio related to odbc handling, was unable to find root cause and it seemed unique to sql azure as a local sql server instance worked without issue. My working theory was that the sql azure drops a connection to a node from its load balancer somewhere inside that black box but kamailio attempts to reuse that connection and hangs. Also due to how sql azure works I found query time to variable as well and based on dB size the number of open connections seems limiting. To overcome this I ended up writing a simple proxy Web API in. Net core which ran on local host And cached results to a local redis instance which gave consistent sub 10ms responses -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/1993#issuecomment-505771401___ Kamailio (SER) - Development Mailing List sr-dev@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
[sr-dev] Additional headers on a dispatcher options ping
Hi Is there any mechanism to add a contact and rr headers to the option messages generated by dispatcher? I have a need for both headers to be present otherwise the message gets rejected at the proxy im trying to connect to, the content of the headers in my use case is the from address, so would seem trivial (in so far I know enough c to just about follow what's going on in the code, but wouldn't know where to start to add the functionality) to add using the existing from attribute and a pair of additional modparams to enable this functionality, if its not already possible Currently I have a workaround by generating the pings in asterisk and routing as usual, but would really prefer to not reinvent the wheel and use dispatcher as I do with other trunks and services mainly for consistency and keeping what is essentially a routing issue out of the application stack, as apart from the missing headers dispatchers functionality is exactly what I need. Any suggestions? Tim. ___ Kamailio (SER) - Development Mailing List sr-dev@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev