[OpenSIPS-Devel] [opensips] error in infinite loop io_watch_del: BUG - trying to del fd 34 with flags 2 1 (#591)
Hi Bogdan, Almost a month ago, I have raised the issue regarding the opensips log file filling up with the following message ERROR:core:io_watch_del: BUG - trying to del fd 38 with flags 2 1 Now I can reproduce it every time with the following scenario: 1) UAC and UAS are connected to opensips proxy with TCP transport 2) Connections looks good 3) Netstat shows the connection to UAC in ESTABLISHED state and the tcp_conn_lists of opensips also looks fine for that UAC 4) Now unplug the Ethernet cable on UAC 5) After tcp connection timeout (set to 5 mins in opensips.cfg), the tcp connection goes away from netstat as well as from tcp_conn_lists 6) UAS tries another tcp call to the UAC which is still unplugged 7) tcp_conn_lists shows the tcp connection to the UAC and netstat shows the connection in SYN_SENT state 8) After the tcp connection timeout (set to 5 mins in opensips.cfg), the connection goes away from netstat however it remains there in tcp_conn_lists and at that moment the “BUG - trying to del fd 38 with flags 2 1” starts printing in infinite loop. I looked at the source code and observed the following 1) When the network cable is plugged in The io_watch_add happens with flag IO_WATCH_READ in tcp_main.c when the command is CONN_NEW 2) When the network cable is unplugged The io_watch_add happens with flag IO_WATCH_WRITE in tcp_main.c when the command is ASYNC_CONNECT 3) While doing io_watch_del after timeout, from handle_tcpconn_ev method, io_watch_del always uses IO_WATCH_READ to delete the fd, which gives this error in case of unplugged cable. Please look into this and suggest how can I fix this issue ? Thanks Rahul Gupta --- Reply to this email directly or view it on GitHub: https://github.com/OpenSIPS/opensips/issues/591___ Devel mailing list Devel@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
Re: [OpenSIPS-Devel] [opensips] error in infinite loop io_watch_del: BUG - trying to del fd 34 with flags 2 1 (#591)
I have captured logs for the scenario and uploaded to zippyshare . Please let me know if you want me to upload to anyother location http://www76.zippyshare.com/v/njlgk4lW/file.html 1) netstat_debug.log --- netstat info for the particular endpoint (10.205.203.130 in this case) 2) opensipsctl_debug.log --- list_tcp_conns from opensips for the particular end point (10.205.203.130) 3) opensips.log --- opensips logs with debug level 6 Call flow and timestamps : 1) 15:45:02 --- created a TCP connection to send out SIP NOTIFY to 10.205.203.130 2) 15:46:30 --- unplugged the network cable from the endpoint 10.205.203.130 3) 15:47:11 --- tcp connection timed out (FYI tcp_connection_lifetime=120 in opensips.cfg) 4) 15:49:30 --- Tried another call to the same end point while the network cable was unplugged. opensips created another TCP connection to send another SIP NOTIFY, netstat showed this connection in SYN_SENT state 5) 15:50:31 --- the above connection went away from net stat and opensips started printing BUG - trying to del fd 34 with flags 2 1 in infinite loop FYI, I changed the debug level of this message from ERR to DBG in source code and recompiled so that it doesnt fill my hard drive while running in debug=2 mode Let me know if you need more clarification or any other details regarding this issue Thanks Rahul Gupta --- Reply to this email directly or view it on GitHub: https://github.com/OpenSIPS/opensips/issues/591#issuecomment-128507656___ Devel mailing list Devel@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
[OpenSIPS-Devel] [OpenSIPS/opensips] 21fb5c: Just fixing indentation.
Branch: refs/heads/usrlocCID Home: https://github.com/OpenSIPS/opensips Commit: 21fb5cd6320099d1b7c21bc9131df06006015d66 https://github.com/OpenSIPS/opensips/commit/21fb5cd6320099d1b7c21bc9131df06006015d66 Author: Bogdan-Andrei Iancu bog...@opensips.org Date: 2015-08-06 (Thu, 06 Aug 2015) Changed paths: M modules/registrar/save.c M modules/usrloc/ucontact.c M modules/usrloc/udomain.c M modules/usrloc/urecord.c Log Message: --- Just fixing indentation. ___ Devel mailing list Devel@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
[OpenSIPS-Devel] [OpenSIPS/opensips] 7d20a3: fix contact id in DB_ONLY mode issue
Branch: refs/heads/usrlocCID Home: https://github.com/OpenSIPS/opensips Commit: 7d20a3605b9d6be575193f28952314954730654b https://github.com/OpenSIPS/opensips/commit/7d20a3605b9d6be575193f28952314954730654b Author: ionutrazvanionita ionution...@opensips.org Date: 2015-08-06 (Thu, 06 Aug 2015) Changed paths: M modules/usrloc/ucontact.c M modules/usrloc/urecord.c Log Message: --- fix contact id in DB_ONLY mode issue ___ Devel mailing list Devel@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
[OpenSIPS-Devel] [OpenSIPS/opensips] ad08a0: Fix missing typedef.
Branch: refs/heads/2.1 Home: https://github.com/OpenSIPS/opensips Commit: ad08a0d4c139d6fb5dc8787ad9997c7959b3f95c https://github.com/OpenSIPS/opensips/commit/ad08a0d4c139d6fb5dc8787ad9997c7959b3f95c Author: Bogdan-Andrei Iancu bog...@opensips.org Date: 2015-08-06 (Thu, 06 Aug 2015) Changed paths: M sr_module_deps.h Log Message: --- Fix missing typedef. It seems the code is a bit different from 2.2 , so fixing a8a487 ___ Devel mailing list Devel@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
[OpenSIPS-Devel] [OpenSIPS/opensips] 164dbe: Fix async sleep() in sync mode.
Branch: refs/heads/master Home: https://github.com/OpenSIPS/opensips Commit: 164dbe53752783f7dd1c96c9a6a1874385241337 https://github.com/OpenSIPS/opensips/commit/164dbe53752783f7dd1c96c9a6a1874385241337 Author: Bogdan-Andrei Iancu bog...@opensips.org Date: 2015-08-06 (Thu, 06 Aug 2015) Changed paths: M modules/cfgutils/cfgutils.c Log Message: --- Fix async sleep() in sync mode. If the timer FD cannot be placed in reactor, the resume will work (in sync way), but without any actual sleep. We fix this by remembering the time when the sleep should finish and checking it again in resume function. ___ Devel mailing list Devel@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
[OpenSIPS-Devel] [OpenSIPS/opensips]
Branch: refs/heads/usrlocCID Home: https://github.com/OpenSIPS/opensips ___ Devel mailing list Devel@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
[OpenSIPS-Devel] [OpenSIPS/opensips] 04c9ad: introduced contactid as composed of aorhash|recrod...
Branch: refs/heads/master Home: https://github.com/OpenSIPS/opensips Commit: 04c9add72d463112161eba02c038bc790ceaedb5 https://github.com/OpenSIPS/opensips/commit/04c9add72d463112161eba02c038bc790ceaedb5 Author: ionutrazvanionita ionution...@opensips.org Date: 2015-07-24 (Fri, 24 Jul 2015) Changed paths: M modules/usrloc/dlist.c M modules/usrloc/dlist.h M modules/usrloc/hslot.c M modules/usrloc/ucontact.c M modules/usrloc/ucontact.h M modules/usrloc/udomain.c M modules/usrloc/udomain.h M modules/usrloc/ul_mi.c M modules/usrloc/ul_mod.c M modules/usrloc/ul_mod.h M modules/usrloc/urecord.c M modules/usrloc/urecord.h Log Message: --- introduced contactid as composed of aorhash|recrod label|contact label Commit: f1578b14b07bc245317243bf2616e2dd0d1fbe82 https://github.com/OpenSIPS/opensips/commit/f1578b14b07bc245317243bf2616e2dd0d1fbe82 Author: ionutrazvanionita ionution...@opensips.org Date: 2015-07-27 (Mon, 27 Jul 2015) Changed paths: M db/schema/location.xml M doc/dbschema/xsl/sqlite.xsl A scripts/db_berkeley/opensips/emergency_service_provider M scripts/db_berkeley/opensips/location M scripts/db_berkeley/opensips/version A scripts/dbtext/opensips/emergency_service_provider M scripts/dbtext/opensips/location M scripts/dbtext/opensips/version M scripts/mysql/emergency-create.sql M scripts/mysql/usrloc-create.sql M scripts/oracle/emergency-create.sql M scripts/oracle/usrloc-create.sql M scripts/pi_http/emergency-mod M scripts/pi_http/emergency-table M scripts/pi_http/pi_framework.xml M scripts/pi_http/usrloc-mod M scripts/pi_http/usrloc-table M scripts/postgres/emergency-create.sql M scripts/postgres/usrloc-create.sql M scripts/sqlite/emergency-create.sql M scripts/sqlite/usrloc-create.sql Log Message: --- modified id column to contact_id bigint primary key autoincrement in al dbs; regenerated scripts Commit: 4d7c67f1b6b9d8c42c1e97b76013881461b9d1f7 https://github.com/OpenSIPS/opensips/commit/4d7c67f1b6b9d8c42c1e97b76013881461b9d1f7 Author: ionutrazvanionita ionution...@opensips.org Date: 2015-07-27 (Mon, 27 Jul 2015) Changed paths: M modules/usrloc/ucontact.c M modules/usrloc/udomain.c M modules/usrloc/urecord.c Log Message: --- on DB_ONLY insert contact id value is null(autoincrement is used) Commit: 9404a7654bcd226091b6e065dd6c82e7eb5a0f46 https://github.com/OpenSIPS/opensips/commit/9404a7654bcd226091b6e065dd6c82e7eb5a0f46 Author: ionutrazvanionita ionution...@opensips.org Date: 2015-07-27 (Mon, 27 Jul 2015) Changed paths: M modules/usrloc/ucontact.c Log Message: --- update and delete based on contact_id Commit: bbccb7a310b1ea282cb3100e0cec0cb052c8724e https://github.com/OpenSIPS/opensips/commit/bbccb7a310b1ea282cb3100e0cec0cb052c8724e Author: ionutrazvanionita ionution...@opensips.org Date: 2015-07-28 (Tue, 28 Jul 2015) Changed paths: M scripts/db_berkeley/opensips/version M scripts/dbtext/opensips/version M scripts/mysql/usrloc-create.sql M scripts/oracle/usrloc-create.sql M scripts/postgres/usrloc-create.sql M scripts/sqlite/usrloc-create.sql Log Message: --- updated location version Commit: 603a66072de2ae3f9bb2cd8b1551ab0f85fa3740 https://github.com/OpenSIPS/opensips/commit/603a66072de2ae3f9bb2cd8b1551ab0f85fa3740 Author: ionutrazvanionita ionution...@opensips.org Date: 2015-07-28 (Tue, 28 Jul 2015) Changed paths: M modules/db_sqlite/res.c M modules/db_sqlite/row.c Log Message: --- for PRIMARY KEY AUTOINCREMENT in sqlite: value is stored as BIGINT, type is INT, allowing in code usage to see the key both int and bigint(default int) Commit: b46429e2eb28e52d86f37b28913d656b207d1fc5 https://github.com/OpenSIPS/opensips/commit/b46429e2eb28e52d86f37b28913d656b207d1fc5 Author: ionutrazvanionita ionution...@opensips.org Date: 2015-07-28 (Tue, 28 Jul 2015) Changed paths: M modules/usrloc/ucontact.c M modules/usrloc/udomain.c M modules/usrloc/ul_mod.c Log Message: --- fix db_insert_ucontact: no value for contactid when AUTOINCREMENT wanted Commit: 91828e31b7a73d22487bb5da629af6821f64791b https://github.com/OpenSIPS/opensips/commit/91828e31b7a73d22487bb5da629af6821f64791b Author: ionutrazvanionita ionution...@opensips.org Date: 2015-07-29 (Wed, 29 Jul 2015) Changed paths: M modules/usrloc/dlist.c M modules/usrloc/dlist.h M modules/usrloc/usrloc.c M modules/usrloc/usrloc.h Log Message: --- possibility to delete a contact only with the id Commit: ece27741725d7bbb26a2594803280d376df874d0 https://github.com/OpenSIPS/opensips/commit/ece27741725d7bbb26a2594803280d376df874d0 Author: ionutrazvanionita
Re: [OpenSIPS-Devel] [opensips] crash issue in tcpconn_add_alias (2.1) (#589)
@dbeskoek , could you do a list in gdb, to see what is your line 766 ? In my case it is a LM_WARN : LM_WARN(alias already present and points to another connection (%d : %d and %d : %d)\n, a-parent-id, port, c-id, port); Is this correct ? --- Reply to this email directly or view it on GitHub: https://github.com/OpenSIPS/opensips/issues/589#issuecomment-128388030___ Devel mailing list Devel@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
Re: [OpenSIPS-Devel] [opensips] crash issue in tcpconn_add_alias (2.1) (#589)
thats correct: ``` 761 LM_ERR(no connection found for id %d\n,id); 762 return -1; 763 error_sec: 764 TCPCONN_UNLOCK(id); 765 LM_WARN(possible port hijack attempt\n); 766 LM_WARN(alias already present and points to another connection 767 (%d : %d and %d : %d)\n, a-parent-id, port, c-id, port); 768 return -1; 769 } 770 ``` --- Reply to this email directly or view it on GitHub: https://github.com/OpenSIPS/opensips/issues/589#issuecomment-128392028___ Devel mailing list Devel@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
Re: [OpenSIPS-Devel] [opensips] crash issue in tcpconn_add_alias (2.1) (#589)
Is there any chance to get direct access to the corefile (ssh?) the debugging gets a bit more complex at this stage. --- Reply to this email directly or view it on GitHub: https://github.com/OpenSIPS/opensips/issues/589#issuecomment-128424797___ Devel mailing list Devel@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
[OpenSIPS-Devel] [OpenSIPS/opensips] a8a487: Remove double define.
Branch: refs/heads/2.1 Home: https://github.com/OpenSIPS/opensips Commit: a8a4875304ca6aa53e39bc566e429ac63a3a300e https://github.com/OpenSIPS/opensips/commit/a8a4875304ca6aa53e39bc566e429ac63a3a300e Author: Bogdan-Andrei Iancu bog...@opensips.org Date: 2015-08-06 (Thu, 06 Aug 2015) Changed paths: M sr_module.h Log Message: --- Remove double define. param_export_t is already defined by sr_module_deps.h which is included from here. (cherry picked from commit 0ff15f63e7002e58385228a16bb4e43dbf75a400) ___ Devel mailing list Devel@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
[OpenSIPS-Devel] [OpenSIPS/opensips] 0ff15f: Remove double define.
Branch: refs/heads/master Home: https://github.com/OpenSIPS/opensips Commit: 0ff15f63e7002e58385228a16bb4e43dbf75a400 https://github.com/OpenSIPS/opensips/commit/0ff15f63e7002e58385228a16bb4e43dbf75a400 Author: Bogdan-Andrei Iancu bog...@opensips.org Date: 2015-08-06 (Thu, 06 Aug 2015) Changed paths: M sr_module.h Log Message: --- Remove double define. param_export_t is already defined by sr_module_deps.h which is included from here. ___ Devel mailing list Devel@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
[OpenSIPS-Devel] [OpenSIPS/opensips] ad6a92: Fix usage of timerfd on old systems.
Branch: refs/heads/master Home: https://github.com/OpenSIPS/opensips Commit: ad6a923a4c508f683ef6a9a364f67f738b6b650d https://github.com/OpenSIPS/opensips/commit/ad6a923a4c508f683ef6a9a364f67f738b6b650d Author: Bogdan-Andrei Iancu bog...@opensips.org Date: 2015-08-06 (Thu, 06 Aug 2015) Changed paths: M modules/cfgutils/cfgutils.c Log Message: --- Fix usage of timerfd on old systems. timerfd functions were introduced starting with glib 2.8, so disable them if older glib versions are detected. ___ Devel mailing list Devel@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
[OpenSIPS-Devel] [OpenSIPS/opensips] 2d1506: fix debian packaging
Branch: refs/heads/master Home: https://github.com/OpenSIPS/opensips Commit: 2d15060e48333ab3cad2e9cf3cafb7a006324185 https://github.com/OpenSIPS/opensips/commit/2d15060e48333ab3cad2e9cf3cafb7a006324185 Author: Răzvan Crainea raz...@opensips.org Date: 2015-08-06 (Thu, 06 Aug 2015) Changed paths: M packaging/debian/rules Log Message: --- fix debian packaging ___ Devel mailing list Devel@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
[OpenSIPS-Devel] [OpenSIPS/opensips] b7524e: add rest_client specs for debian, fedora and rpms
Branch: refs/heads/2.1 Home: https://github.com/OpenSIPS/opensips Commit: b7524e2f5b84549476b4f934d56d682264ce227a https://github.com/OpenSIPS/opensips/commit/b7524e2f5b84549476b4f934d56d682264ce227a Author: Razvan Crainea raz...@opensips.org Date: 2015-08-06 (Thu, 06 Aug 2015) Changed paths: M packaging/debian/control M packaging/fedora/opensips.spec M packaging/rpm/opensips.spec.CentOS Log Message: --- add rest_client specs for debian, fedora and rpms Closes #587 (cherry picked from commit 3dfc83cb254f150d2e6dbbb49f48ec9927be3a3c) Commit: 2671738aa7220fd73801c00354395c254f720186 https://github.com/OpenSIPS/opensips/commit/2671738aa7220fd73801c00354395c254f720186 Author: Răzvan Crainea raz...@opensips.org Date: 2015-08-06 (Thu, 06 Aug 2015) Changed paths: M packaging/debian/rules Log Message: --- fix debian packaging (cherry picked from commit 2d15060e48333ab3cad2e9cf3cafb7a006324185) Compare: https://github.com/OpenSIPS/opensips/compare/a8a4875304ca...2671738aa722___ Devel mailing list Devel@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
[OpenSIPS-Devel] [OpenSIPS/opensips] 2bdd10: add rest_client specs for debian, fedora and rpms
Branch: refs/heads/1.11 Home: https://github.com/OpenSIPS/opensips Commit: 2bdd10f3eea77ad578ff5db2aa3869a4f7fda900 https://github.com/OpenSIPS/opensips/commit/2bdd10f3eea77ad578ff5db2aa3869a4f7fda900 Author: Razvan Crainea raz...@opensips.org Date: 2015-08-06 (Thu, 06 Aug 2015) Changed paths: M packaging/debian/control M packaging/fedora/opensips.spec M packaging/rpm/opensips.spec.CentOS Log Message: --- add rest_client specs for debian, fedora and rpms Closes #587 (cherry picked from commit 3dfc83cb254f150d2e6dbbb49f48ec9927be3a3c) Conflicts: packaging/debian/control packaging/fedora/opensips.spec Commit: 235e3e46ab1f376386c618d770b95fbea7955d1d https://github.com/OpenSIPS/opensips/commit/235e3e46ab1f376386c618d770b95fbea7955d1d Author: Răzvan Crainea raz...@opensips.org Date: 2015-08-06 (Thu, 06 Aug 2015) Changed paths: M packaging/debian/rules Log Message: --- fix debian packaging (cherry picked from commit 2d15060e48333ab3cad2e9cf3cafb7a006324185) Conflicts: packaging/debian/rules Compare: https://github.com/OpenSIPS/opensips/compare/1b5c86e01636...235e3e46ab1f___ Devel mailing list Devel@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
Re: [OpenSIPS-Devel] [opensips] crash issue in tcpconn_add_alias (2.1) (#589)
ok, try printing in gdb the vars: a, a-parent, c --- Reply to this email directly or view it on GitHub: https://github.com/OpenSIPS/opensips/issues/589#issuecomment-128418538___ Devel mailing list Devel@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
Re: [OpenSIPS-Devel] [opensips] crash issue in tcpconn_add_alias (2.1) (#589)
``` (gdb) p a $1 = (struct tcp_conn_alias *) 0x7f31115f35d8 (gdb) p a-parent $2 = (struct tcp_connection *) 0x0 (gdb) p c $3 = (struct tcp_connection *) 0x7f311a308470 ``` --- Reply to this email directly or view it on GitHub: https://github.com/OpenSIPS/opensips/issues/589#issuecomment-128420950___ Devel mailing list Devel@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/devel
