Re: [vpp-dev] VPP API sometimes hangs on ip_route_details response #vppcapi #vpp_stability #vpp #binapi
Hello, Thanks Vratko for the reply. I’m happy to provide access to an environment where we quite easily reproduce the issue, is this might help. Sylvain De : "Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco)" Date : mercredi 4 septembre 2019 à 18:28 À : Sylvain CADILHAC , "vpp-dev@lists.fd.io" Objet : Re: [vpp-dev] VPP API sometimes hangs on ip_route_details response #vppcapi #vpp_stability #vpp #binapi Looks like VPP-1753<https://jira.fd.io/browse/VPP-1753> to me. I do not have much to add, heisenbugs are hard to figure out. Vratko. From: vpp-dev@lists.fd.io on behalf of sylvain.cadil...@jaguar-network.com Sent: Wednesday, September 4, 2019 01:36 To: vpp-dev@lists.fd.io Subject: [vpp-dev] VPP API sometimes hangs on ip_route_details response #vppcapi #vpp_stability #vpp #binapi Hi vpp-dev, I'm running a C program which adds/removes interfaces, IPs, routes, etc. to VPP via the C API, along with a Python test suite which checks the results of the first program multiple times by querying VPP using the Python bindings. I'm seeing a fairly reproductible issue (when repeating the test suite a number of times), where the VPP API thread remains blocked, always after the clib_mem_free of a ip_route_details reply message (response to Python). Traces: * This is VPP 19.08 with slight plugin changes; I can provide the packages if needed. * The backtrace is here: https://gist.github.com/SCadilhac/bb9ca9600d757b13726e05ae34923c1a#file-backtrace * The startup config file is here: https://gist.github.com/SCadilhac/bb9ca9600d757b13726e05ae34923c1a#file-startup-conf * I don't know how to extract the API dump, as the CLI gets unresponsive, and the process /tmp/api_post_mortem file remains unwritten. * The gzipped core dump is here: http://www.netfishers.onl/downloads/core.24542.gz Any help is welcome :-). Is there anything I'm obviously doing wrong? I'm happy to open a Jira issue if this can help. Thanks, Sylvain Ce message et toutes les pièces jointes (ci-après le "message") sont établis à l'intention exclusive de ses destinataires. Ce message est strictement confidentiel et peut comporter des données à caractère personnel ; si vous recevez ce message par erreur, merci de le détruire et d'en avertir immédiatement l'expéditeur par retour d'e-mail. Toute utilisation de ce message non conforme à sa destination, toute diffusion ou toute publication, totale ou partielle, est interdite, sauf autorisation expresse de Jaguar Network. La sécurité des communications sur Internet ne peut être garantie et Jaguar Network informe en conséquence qu'elle ne peut accepter aucune responsabilité quant au contenu de ce message. This message and all attachments (hereinafter the "message") are intended solely for the intended recipients. This message is strictly confidential and may include personal data; if you receive this message by mistake, please destroy it and notify the sender immediately by return e-mail. Any use of this message that does not comply with its intended purpose, any broadcast or any publication, total or partial, is prohibited, unless expressly authorized by Jaguar Network. Internet communications security cannot be guaranteed and Jaguar Network therefore informs that it cannot accept any responsibility for the content of this message. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#13910): https://lists.fd.io/g/vpp-dev/message/13910 Mute This Topic: https://lists.fd.io/mt/33132760/21656 Mute #vpp: https://lists.fd.io/mk?hashtag=vpp=1480452 Mute #binapi: https://lists.fd.io/mk?hashtag=binapi=1480452 Mute #vpp_stability: https://lists.fd.io/mk?hashtag=vpp_stability=1480452 Mute #vppcapi: https://lists.fd.io/mk?hashtag=vppcapi=1480452 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] VPP API sometimes hangs on ip_route_details response #vppcapi #vpp_stability #vpp #binapi
Looks like VPP-1753<https://jira.fd.io/browse/VPP-1753> to me. I do not have much to add, heisenbugs are hard to figure out. Vratko. From: vpp-dev@lists.fd.io on behalf of sylvain.cadil...@jaguar-network.com Sent: Wednesday, September 4, 2019 01:36 To: vpp-dev@lists.fd.io Subject: [vpp-dev] VPP API sometimes hangs on ip_route_details response #vppcapi #vpp_stability #vpp #binapi Hi vpp-dev, I'm running a C program which adds/removes interfaces, IPs, routes, etc. to VPP via the C API, along with a Python test suite which checks the results of the first program multiple times by querying VPP using the Python bindings. I'm seeing a fairly reproductible issue (when repeating the test suite a number of times), where the VPP API thread remains blocked, always after the clib_mem_free of a ip_route_details reply message (response to Python). Traces: * This is VPP 19.08 with slight plugin changes; I can provide the packages if needed. * The backtrace is here: https://gist.github.com/SCadilhac/bb9ca9600d757b13726e05ae34923c1a#file-backtrace * The startup config file is here: https://gist.github.com/SCadilhac/bb9ca9600d757b13726e05ae34923c1a#file-startup-conf * I don't know how to extract the API dump, as the CLI gets unresponsive, and the process /tmp/api_post_mortem file remains unwritten. * The gzipped core dump is here: http://www.netfishers.onl/downloads/core.24542.gz Any help is welcome :-). Is there anything I'm obviously doing wrong? I'm happy to open a Jira issue if this can help. Thanks, Sylvain -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#13905): https://lists.fd.io/g/vpp-dev/message/13905 Mute This Topic: https://lists.fd.io/mt/33132760/21656 Mute #vpp: https://lists.fd.io/mk?hashtag=vpp=1480452 Mute #binapi: https://lists.fd.io/mk?hashtag=binapi=1480452 Mute #vpp_stability: https://lists.fd.io/mk?hashtag=vpp_stability=1480452 Mute #vppcapi: https://lists.fd.io/mk?hashtag=vppcapi=1480452 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] VPP API sometimes hangs on ip_route_details response #vppcapi #vpp_stability #vpp #binapi
Hi vpp-dev, I'm running a C program which adds/removes interfaces, IPs, routes, etc. to VPP via the C API, along with a Python test suite which checks the results of the first program multiple times by querying VPP using the Python bindings. I'm seeing a fairly reproductible issue (when repeating the test suite a number of times), where the VPP API thread remains blocked, always after the clib_mem_free of a ip_route_details reply message (response to Python). Traces: * This is VPP 19.08 with slight plugin changes; I can provide the packages if needed. * The backtrace is here: https://gist.github.com/SCadilhac/bb9ca9600d757b13726e05ae34923c1a#file-backtrace * The startup config file is here: https://gist.github.com/SCadilhac/bb9ca9600d757b13726e05ae34923c1a#file-startup-conf * I don't know how to extract the API dump, as the CLI gets unresponsive, and the process /tmp/api_post_mortem file remains unwritten. * The gzipped core dump is here: http://www.netfishers.onl/downloads/core.24542.gz Any help is welcome :-). Is there anything I'm obviously doing wrong? I'm happy to open a Jira issue if this can help. Thanks, Sylvain -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#13891): https://lists.fd.io/g/vpp-dev/message/13891 Mute This Topic: https://lists.fd.io/mt/33132760/21656 Mute #vppcapi: https://lists.fd.io/mk?hashtag=vppcapi=1480452 Mute #vpp_stability: https://lists.fd.io/mk?hashtag=vpp_stability=1480452 Mute #vpp: https://lists.fd.io/mk?hashtag=vpp=1480452 Mute #binapi: https://lists.fd.io/mk?hashtag=binapi=1480452 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
