Re: [Sip-implementors] Reinvite/ACK race
Alex Balashov writes: > On Tue, Oct 31, 2017 at 10:38:00PM -0400, Dale R. Worley wrote: >> As others have said, the problem must be solved by the UAs, not the >> proxy. Indeed, what with possible network delays and re-sending of >> lost packets, there's no way to guarantee end-to-end sequencing of >> messages. Even with TCP service, there's no guarantee that none of >> the intermediate proxies aren't using multiple TCP connections to >> carry messages between themselves. > > That's certainly true. But in this case I do control the entire setup > topology end-to-end, and implicit in my thesis was the desire to assert > control over that over which control can be asserted. :-) Well, I'd answer "the difference between theory and practice is larger in practice than in theory". If you can get what you want without exerting special control over all of the parts of the system -- a control which SIP is not designed to ensure you can have -- you're a lot more likely to succeed. > The scenario described in the foregoing is for a reinvite occuring after > an *initial* INVITE cycle but arriving before the ACK for that initial > INVITE. > > The scenario I've got going on is a reinvite following another reinvite. > > As you suggest, there are few reasons to think that the reasoning > should be different, or at any rate substantially different, for a > consecutive reinvite vs. a reinvite following an initial > INVITE. Nevertheless, that was what motivated the perception that > [section] 3.1.4 may not speak to the fine letter of the exact scenario > in play. True... But I wouldn't want to have to build a system that distinguished the two cases. Dale ___ Sip-implementors mailing list Sip-implementors@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/sip-implementors
Re: [Sip-implementors] Reinvite/ACK race
On Tue, Oct 31, 2017 at 10:38:00PM -0400, Dale R. Worley wrote: > As others have said, the problem must be solved by the UAs, not the > proxy. Indeed, what with possible network delays and re-sending of > lost packets, there's no way to guarantee end-to-end sequencing of > messages. Even with TCP service, there's no guarantee that none of > the intermediate proxies aren't using multiple TCP connections to > carry messages between themselves. That's certainly true. But in this case I do control the entire setup topology end-to-end, and implicit in my thesis was the desire to assert control over that over which control can be asserted. :-) > You're correct to look to 5407 (which is important enough to be known as > just "Hasebe") for guidance. And it does look like section 3.1.4 is the > relevant case, or very close to it. > >This scenario illustrates the race condition that occurs when a UAS >in the Moratorium state [having sent 2xx but not received ACK] >receives a re-INVITE sent by a UAC in the Established state. > > And as the text goes on, sending 2xx is recommended, rather than 491, > because all of the state changes within the UA associated with the first > re-INVITE have already been completed. But a 491 response is also > permissible. The reason that I did not think § 3.1.4 was exactly pertinent was due to this formulation: The UAS receives a re-INVITE (with offer2) before receiving an ACK for the ini-INVITE (with offer1). The UAS sends a 200 OK (with answer2) to the re-INVITE (F8) because it has sent a 200 OK (with answer1) to the ini-INVITE (F3, F5) and the dialog has already been established. (Because F5 is a retransmission of F3, SDP negotiation is not performed here.) The scenario described in the foregoing is for a reinvite occuring after an *initial* INVITE cycle but arriving before the ACK for that initial INVITE. The scenario I've got going on is a reinvite following another reinvite. As you suggest, there are few reasons to think that the reasoning should be different, or at any rate substantially different, for a consecutive reinvite vs. a reinvite following an initial INVITE. Nevertheless, that was what motivated the perception that § 3.1.4 may not speak to the fine letter of the exact scenario in play. -- Alex -- Alex Balashov | Principal | Evariste Systems LLC Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free) Web: http://www.evaristesys.com/, http://www.csrpswitch.com/ ___ Sip-implementors mailing list Sip-implementors@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/sip-implementors
Re: [Sip-implementors] Reinvite/ACK race
Alex Balashov writes: > 5. UA B sends end-to-end ACK for reinvite #1 and almost >simultaneously sends reinvite #2. The temporal delta is >between reinvite #2 and ACK for reinvite #1 on the wire >is 3 ms. > > The issue is that the concurrency characteristics of proxy P are such > that its worker threads are very loosely coupled, and there's no > synchronisation among them for message ordering. Transport is UDP, > naturally. As others have said, the problem must be solved by the UAs, not the proxy. Indeed, what with possible network delays and re-sending of lost packets, there's no way to guarantee end-to-end sequencing of messages. Even with TCP service, there's no guarantee that none of the intermediate proxies aren't using multiple TCP connections to carry messages between themselves. > So, the result -- for all kinds of stochastic processing and userspace > scheduling type reasons -- is that the reinvite is forwarded first, > before the ACK. That leads to a 500 / 491 scenario UA A. > > Is there any general guidance on what to do with these scenarios? I > looked at RFC 5407 [section] 3.1.4, which appears to describe a > similar, but not identical scenario involving an initial INVITE and > subsequent reinvite. As far as I can tell, the recommendation in that > standard is "space the messaging out more in time". You're correct to look to 5407 (which is important enough to be known as just "Hasebe") for guidance. And it does look like section 3.1.4 is the relevant case, or very close to it. This scenario illustrates the race condition that occurs when a UAS in the Moratorium state [having sent 2xx but not received ACK] receives a re-INVITE sent by a UAC in the Established state. And as the text goes on, sending 2xx is recommended, rather than 491, because all of the state changes within the UA associated with the first re-INVITE have already been completed. But a 491 response is also permissible. Dale ___ Sip-implementors mailing list Sip-implementors@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/sip-implementors
Re: [Sip-implementors] Reinvite/ACK race
Paul, I'm not a SIP standards expert by any stretch, but yours is the interpretation I tend to favour and which sits best with me. Since in-dialog requests are originated by the dialog parties, timing issues belong with them, not with intermediate proxies IMHO. On October 31, 2017 10:50:45 AM EDT, Paul Kyzivat wrote: >On 10/31/17 8:44 AM, Liviu Chircu wrote: >> Hi Alex, >> >> IMO, building logic into the proxy which encourages/mends the proper >> sequencing of UDP messages does nothing more than to hide the >underlying >> problem, i.e. "UDP does not guarantee message sequencing in the first > >> place" *. There is also a subtle point to be made here: your proxy's >> loosely coupled/parallelized way of handling the two transactions is >> effectively breaking the RFC some % of the time, since the proxy's TU >is >> generating a new INVITE while another INVITE transaction is in >progress >> (strictly judging by the network timestamps). > >One can argue that the proxy is doing nothing wrong and perhaps being >helpful by exposing bugs in handling of reordered UDP messages. > > Thanks, > Paul > >> * From this angle, the only sane thing left to do is to have the >proxy >> retry the re-INVITE for UASs who are properly responding with 491 and > >> propagate any other 4xx/5xx error replies back to the UAC (UA B in >our >> case) in the hope that they retry their re-INVITE. >> >> Best regards, >> >> Liviu Chircu >> OpenSIPS Developer >> http://www.opensips-solutions.com >> >> On 30.10.2017 19:11, Alex Balashov wrote: >>> Hi, >>> >>> I've got a scenario like so: >>> >>> UA A -> Proxy P > UA B >>> >>> 1. UA A initiates call through Proxy P; >>> >>> 2. Dialog is established and confirmed, with Record-Route; >>> >>> 3. UA B sends reinvite #1 through P to A; >>> >>> 4. UA B sends 2xx reply; >>> >>> 5. UA B sends end-to-end ACK for reinvite #1 and almost >>> simultaneously sends reinvite #2. The temporal delta is >>> between reinvite #2 and ACK for reinvite #1 on the wire >>> is 3 ms. >>> >>> The issue is that the concurrency characteristics of proxy P are >such >>> that its worker threads are very loosely coupled, and there's no >>> synchronisation among them for message ordering. Transport is UDP, >>> naturally. >>> >>> So, the result — for all kinds of stochastic processing and >userspace >>> scheduling type reasons — is that the reinvite is forwarded first, >>> before the ACK. That leads to a 500 / 491 scenario UA A. >>> >>> Is there any general guidance on what to do with these scenarios? I >>> looked at RFC 5407 § 3.1.4, which appears to describe a similar, but >not >>> identical scenario involving an initial INVITE and subsequent >reinvite. >>> As far as I can tell, the recommendation in that standard is "space >the >>> messaging out more in time". >>> >>> Switching to TCP would presumably help, since any given flow would >>> involve a single connection to a single worker thread and the >transport >>> would guarantee ordering. However, that's not really feasible in >this >>> implementation for a host of reasons. >>> >>> Any other thoughts welcome! >>> >>> Cheers, >>> >>> -- Alex >>> >> >> ___ >> Sip-implementors mailing list >> Sip-implementors@lists.cs.columbia.edu >> https://lists.cs.columbia.edu/mailman/listinfo/sip-implementors > >___ >Sip-implementors mailing list >Sip-implementors@lists.cs.columbia.edu >https://lists.cs.columbia.edu/mailman/listinfo/sip-implementors -- Alex -- Sent via mobile, please forgive typos and brevity. ___ Sip-implementors mailing list Sip-implementors@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/sip-implementors
Re: [Sip-implementors] Reinvite/ACK race
On 10/31/17 8:44 AM, Liviu Chircu wrote: Hi Alex, IMO, building logic into the proxy which encourages/mends the proper sequencing of UDP messages does nothing more than to hide the underlying problem, i.e. "UDP does not guarantee message sequencing in the first place" *. There is also a subtle point to be made here: your proxy's loosely coupled/parallelized way of handling the two transactions is effectively breaking the RFC some % of the time, since the proxy's TU is generating a new INVITE while another INVITE transaction is in progress (strictly judging by the network timestamps). One can argue that the proxy is doing nothing wrong and perhaps being helpful by exposing bugs in handling of reordered UDP messages. Thanks, Paul * From this angle, the only sane thing left to do is to have the proxy retry the re-INVITE for UASs who are properly responding with 491 and propagate any other 4xx/5xx error replies back to the UAC (UA B in our case) in the hope that they retry their re-INVITE. Best regards, Liviu Chircu OpenSIPS Developer http://www.opensips-solutions.com On 30.10.2017 19:11, Alex Balashov wrote: Hi, I've got a scenario like so: UA A -> Proxy P > UA B 1. UA A initiates call through Proxy P; 2. Dialog is established and confirmed, with Record-Route; 3. UA B sends reinvite #1 through P to A; 4. UA B sends 2xx reply; 5. UA B sends end-to-end ACK for reinvite #1 and almost simultaneously sends reinvite #2. The temporal delta is between reinvite #2 and ACK for reinvite #1 on the wire is 3 ms. The issue is that the concurrency characteristics of proxy P are such that its worker threads are very loosely coupled, and there's no synchronisation among them for message ordering. Transport is UDP, naturally. So, the result — for all kinds of stochastic processing and userspace scheduling type reasons — is that the reinvite is forwarded first, before the ACK. That leads to a 500 / 491 scenario UA A. Is there any general guidance on what to do with these scenarios? I looked at RFC 5407 § 3.1.4, which appears to describe a similar, but not identical scenario involving an initial INVITE and subsequent reinvite. As far as I can tell, the recommendation in that standard is "space the messaging out more in time". Switching to TCP would presumably help, since any given flow would involve a single connection to a single worker thread and the transport would guarantee ordering. However, that's not really feasible in this implementation for a host of reasons. Any other thoughts welcome! Cheers, -- Alex ___ Sip-implementors mailing list Sip-implementors@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/sip-implementors ___ Sip-implementors mailing list Sip-implementors@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/sip-implementors
Re: [Sip-implementors] Reinvite/ACK race
Hi Alex, IMO, building logic into the proxy which encourages/mends the proper sequencing of UDP messages does nothing more than to hide the underlying problem, i.e. "UDP does not guarantee message sequencing in the first place" *. There is also a subtle point to be made here: your proxy's loosely coupled/parallelized way of handling the two transactions is effectively breaking the RFC some % of the time, since the proxy's TU is generating a new INVITE while another INVITE transaction is in progress (strictly judging by the network timestamps). * From this angle, the only sane thing left to do is to have the proxy retry the re-INVITE for UASs who are properly responding with 491 and propagate any other 4xx/5xx error replies back to the UAC (UA B in our case) in the hope that they retry their re-INVITE. Best regards, Liviu Chircu OpenSIPS Developer http://www.opensips-solutions.com On 30.10.2017 19:11, Alex Balashov wrote: Hi, I've got a scenario like so: UA A -> Proxy P > UA B 1. UA A initiates call through Proxy P; 2. Dialog is established and confirmed, with Record-Route; 3. UA B sends reinvite #1 through P to A; 4. UA B sends 2xx reply; 5. UA B sends end-to-end ACK for reinvite #1 and almost simultaneously sends reinvite #2. The temporal delta is between reinvite #2 and ACK for reinvite #1 on the wire is 3 ms. The issue is that the concurrency characteristics of proxy P are such that its worker threads are very loosely coupled, and there's no synchronisation among them for message ordering. Transport is UDP, naturally. So, the result — for all kinds of stochastic processing and userspace scheduling type reasons — is that the reinvite is forwarded first, before the ACK. That leads to a 500 / 491 scenario UA A. Is there any general guidance on what to do with these scenarios? I looked at RFC 5407 § 3.1.4, which appears to describe a similar, but not identical scenario involving an initial INVITE and subsequent reinvite. As far as I can tell, the recommendation in that standard is "space the messaging out more in time". Switching to TCP would presumably help, since any given flow would involve a single connection to a single worker thread and the transport would guarantee ordering. However, that's not really feasible in this implementation for a host of reasons. Any other thoughts welcome! Cheers, -- Alex ___ Sip-implementors mailing list Sip-implementors@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/sip-implementors
Re: [Sip-implementors] Reinvite/ACK race
Piece of errata here (always when I'm typing): On Mon, Oct 30, 2017 at 01:11:50PM -0400, Alex Balashov wrote: > Hi, > > I've got a scenario like so: > >UA A -> Proxy P > UA B > > 1. UA A initiates call through Proxy P; > > 2. Dialog is established and confirmed, with Record-Route; > > 3. UA B sends reinvite #1 through P to A; > > 4. UA B sends 2xx reply; In #4, UA A is definitely sending the 2xx reply. -- Alex Balashov | Principal | Evariste Systems LLC Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free) Web: http://www.evaristesys.com/, http://www.csrpswitch.com/ ___ Sip-implementors mailing list Sip-implementors@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/sip-implementors