Re: [RADIATOR] AuthBy DUO issue

[Heikki Vatiainen]

On 16.8.2021 19.23, alexander.hartma...@t-systems.com wrote:

A simpler might be to mark Duo dead for a configurable number of seconds 
after which it's marked as alive again without a check. The next 
authentication would then either work or again trigger marking it as dead.


This solution is now in 4.25 patches and available with patch packages.

What now happens is that when CheckTimerInterval is set to 0, API 
failure sets the API status to failed as usually. When 
FailureBackoffTime, a new parameter, seconds have elapsed since the 
failure was detected, the API is considered to be working again.


In other words, this matches the behaviour quoted above with 
FailureBackoffTime being the configurable parameter for the number of 
seconds.


The default value of FailureBackoffTime is 60. If set to 0, then the API 
never remains in the failed state. Longer values avoid using the API, 
for example, when there's a need to access many different devices to 
troubleshoot timeout causes.


As always, feedback is welcome.

Thanks,
Heikki

--
Heikki Vatiainen
OSC, makers of Radiator
Visit radiatorsoftware.com for Radiator AAA server software
___
radiator mailing list
radiator@lists.open.com.au
https://lists.open.com.au/mailman/listinfo/radiator

Re: [RADIATOR] AuthBy DUO issue

[Alfred Reibenschuh]

hello
 
that really depends on your topology and the high-availability setup of the DUO api
 
in my case i ip hardcoded all upstream servers and a dynamic dns based upstream into nginx
 
and as you already mentioned that would ensure always an answer unless the whole upstream infrastructure went offline.
 
since nginx can do raw tcp, the ldap and tacacs upstreams enjoy a similar config.
 
you can get similar results with ha-proxy instead of nginx if that is more to your liking
Yours sincerelyAlfred Reibenschuh
Network Engineer 
(Management & Monitoring Architect)
Unified Communication Services
Network & Telecommunication AT
 
Value Transformation Services GmbHAn IBM CompanyObere Donaustrasse 951020 Wien
Phone: +43-1-2056320-143Mobile: +43-664-3523820
mail: alfred.reibenschuh_v-tservi...@at.ibm.comwebex: https://ibm.webex.com/meet/alfred.reibenschuh_v-tservicesPlease consider the environment before printing this e-mail.This e-mail is confidential and may also contain privileged information. If you are not the intended recipient you are not authorized to read, print, save, process or disclose this message. If you have received this message by mistake, please inform the sender immediately and delete this e-mail, its attachments and any copies.Any use, distribution, reproduction or disclosure by any person other than the intended recipient is strictly prohibited and the person responsible may incur penalties. 
Thank you!
 
 
- Original message -From: To: , Cc:Subject: [EXTERNAL] AW: [RADIATOR] AuthBy DUO issueDate: Thu, Aug 19, 2021 12:27   
Hello Alfred,
how would the reverse proxy help? Just by ensuring that there is always (as long as the reverse proxy works ) a response to the https request?
 
Thanks, Alex
 
 
T-SYSTEMS AUSTRIA GESMBHPU Cyber SecurityNetwork ArchitectureOperation Manager AuthenticationRennweg 97-99, A-1030 Vienna+43 57057 4320 (phone)+43 676 8642 4320 (mobile)E-mail: alexander.hartma...@t-systems.comInternet: www.t-systems.atBlog: blog.t-systems.atSocial Media: Facebook, Linkedin, TwitterBIG CHANGES START SMALL – CONSERVE RESOURCES BY NOT PRINTING EVERY E-MAIL.T-Systems Austria GesmbH, Rennweg 97-99, A-1030 ViennaCommercial Court Vienna, FN 79340bNotice: This transmittal and/or attachments may be privileged or confidential. It isintended solely for the addressee named above. If you received this transmittal in error,please notify us immediately by reply and delete this message and all its attachments.Thank you.
 
Von: radiator  im Auftrag von Alfred Reibenschuh Gesendet: Dienstag, 17. August 2021 11:26An: radiator@lists.open.com.au Betreff: Re: [RADIATOR] AuthBy DUO issue
 
hello
 
seams like you are facing the same "uncooperative" upstream system issue i have had in the past.
 
i have had similar problems with radius and other protocols, that radiator
would mark all upstream servers offline and never recovering.
 
i did not follow your complete conversation, but iirc DUO is http-based,
so if your issue is ha-related you could get away with setting  CheckTimerInterval to 0
and putting a reverse-proxy between radiator and DUO like NGINX
(the community edition of nginx would be sufficient)
 
Yours sincerelyAlfred Reibenschuh
Network Engineer 
(Management & Monitoring Architect)
Unified Communication Services
Network & Telecommunication AT
 
Value Transformation Services GmbHAn IBM CompanyObere Donaustrasse 951020 Wien
Phone: +43-1-2056320-143Mobile: +43-664-3523820
mail: alfred.reibenschuh_v-tservi...@at.ibm.comwebex: https://ibm.webex.com/meet/alfred.reibenschuh_v-tservicesPlease consider the environment before printing this e-mail.This e-mail is confidential and may also contain privileged information. If you are not the intended recipient you are not authorized to read, print, save, process or disclose this message. If you have received this message by mistake, please inform the sender immediately and delete this e-mail, its attachments and any copies.Any use, distribution, reproduction or disclosure by any person other than the intended recipient is strictly prohibited and the person responsible may incur penalties. 
Thank you!
 
 
Message: 1
Date: Mon, 16 Aug 2021 16:23:45 +From: To: , Subject: Re: [RADIATOR] AuthBy DUO issueMessage-ID:Content-Type: text/plain; charset="windows-1252"Hi,that sounds like a sane solution.A simpler might be to mark Duo dead for a configurable number of seconds after which it's marked as alive again without a check. The next authentication would then either work or again trigger marking it as dead.Thanks, AlexT-SYSTEMS AUSTRIA GESMBHPU Cyber SecurityNetwork ArchitectureOperation

Re: [RADIATOR] AuthBy DUO issue

[Alexander.Hartmaier]

Hello Alfred,
how would the reverse proxy help? Just by ensuring that there is always (as 
long as the reverse proxy works ) a response to the https request?

Thanks, Alex

T-SYSTEMS AUSTRIA GESMBH
PU Cyber Security
Network Architecture
Operation Manager Authentication
Rennweg 97-99, A-1030 Vienna
+43 57057 4320 (phone)
+43 676 8642 4320 (mobile)
E-mail: alexander.hartma...@t-systems.com
Internet: www.t-systems.at
Blog: blog.t-systems.at
Social Media: Facebook, Linkedin, Twitter

BIG CHANGES START SMALL – CONSERVE RESOURCES BY NOT PRINTING EVERY E-MAIL.


T-Systems Austria GesmbH, Rennweg 97-99, A-1030 Vienna
Commercial Court Vienna, FN 79340b

Notice: This transmittal and/or attachments may be privileged or confidential. 
It is
intended solely for the addressee named above. If you received this transmittal 
in error,
please notify us immediately by reply and delete this message and all its 
attachments.
Thank you.


Von: radiator  im Auftrag von Alfred 
Reibenschuh 
Gesendet: Dienstag, 17. August 2021 11:26
An: radiator@lists.open.com.au 
Betreff: Re: [RADIATOR] AuthBy DUO issue

hello

seams like you are facing the same "uncooperative" upstream system issue i have 
had in the past.

i have had similar problems with radius and other protocols, that radiator
would mark all upstream servers offline and never recovering.

i did not follow your complete conversation, but iirc DUO is http-based,
so if your issue is ha-related you could get away with setting 
CheckTimerInterval to 0
and putting a reverse-proxy between radiator and DUO like NGINX
(the community edition of nginx would be sufficient)


Yours sincerely

Alfred Reibenschuh

Network Engineer
(Management & Monitoring Architect)

Unified Communication Services
Network & Telecommunication AT

Value Transformation Services GmbH
An IBM Company
Obere Donaustrasse 95
1020 Wien

Phone: +43-1-2056320-143
Mobile: +43-664-3523820
mail: 
alfred.reibenschuh_v-tservi...@at.ibm.com<mailto:alfred.reibenschuh_v-tservi...@at.ibm.com>
webex: https://ibm.webex.com/meet/alfred.reibenschuh_v-tservices

Please consider the environment before printing this e-mail.

This e-mail is confidential and may also contain privileged information. If you 
are not the intended recipient you are not authorized to read, print, save, 
process or disclose this message. If you have received this message by mistake, 
please inform the sender immediately and delete this e-mail, its attachments 
and any copies.
Any use, distribution, reproduction or disclosure by any person other than the 
intended recipient is strictly prohibited and the person responsible may incur 
penalties.

Thank you!


Message: 1
Date: Mon, 16 Aug 2021 16:23:45 +
From: 
To: , 
Subject: Re: [RADIATOR] AuthBy DUO issue
Message-ID:


Content-Type: text/plain; charset="windows-1252"

Hi,
that sounds like a sane solution.

A simpler might be to mark Duo dead for a configurable number of seconds after 
which it's marked as alive again without a check. The next authentication would 
then either work or again trigger marking it as dead.

Thanks, Alex

T-SYSTEMS AUSTRIA GESMBH
PU Cyber Security
Network Architecture
Operation Manager Authentication
Rennweg 97-99, A-1030 Vienna
+43 57057 4320 (phone)
+43 676 8642 4320 (mobile)
E-mail: alexander.hartma...@t-systems.com
Internet: www.t-systems.at
Blog: blog.t-systems.at
Social Media: Facebook, Linkedin, Twitter

BIG CHANGES START SMALL ? CONSERVE RESOURCES BY NOT PRINTING EVERY E-MAIL.


T-Systems Austria GesmbH, Rennweg 97-99, A-1030 Vienna
Commercial Court Vienna, FN 79340b

Notice: This transmittal and/or attachments may be privileged or confidential. 
It is
intended solely for the addressee named above. If you received this transmittal 
in error,
please notify us immediately by reply and delete this message and all its 
attachments.
Thank you.


Von: radiator  im Auftrag von Heikki 
Vatiainen 
Gesendet: Mittwoch, 14. Juli 2021 20:26
An: radiator@lists.open.com.au 
Betreff: Re: [RADIATOR] AuthBy DUO issue

On 13.7.2021 18.05, alexander.hartma...@t-systems.com wrote:

> We've encountered another issue today: when CheckTimerInterval is
> configured to 0, to disable the periodic DUO API check which fills our

Re: [RADIATOR] AuthBy DUO issue

[Alfred Reibenschuh]

hello
 
seams like you are facing the same "uncooperative" upstream system issue i have had in the past.
 
i have had similar problems with radius and other protocols, that radiator
would mark all upstream servers offline and never recovering.
 
i did not follow your complete conversation, but iirc DUO is http-based,
so if your issue is ha-related you could get away with setting CheckTimerInterval to 0
and putting a reverse-proxy between radiator and DUO like NGINX
(the community edition of nginx would be sufficient)
 
Yours sincerelyAlfred Reibenschuh
Network Engineer 
(Management & Monitoring Architect)
Unified Communication Services
Network & Telecommunication AT
 
Value Transformation Services GmbHAn IBM CompanyObere Donaustrasse 951020 Wien
Phone: +43-1-2056320-143Mobile: +43-664-3523820
mail: alfred.reibenschuh_v-tservi...@at.ibm.comwebex: https://ibm.webex.com/meet/alfred.reibenschuh_v-tservicesPlease consider the environment before printing this e-mail.This e-mail is confidential and may also contain privileged information. If you are not the intended recipient you are not authorized to read, print, save, process or disclose this message. If you have received this message by mistake, please inform the sender immediately and delete this e-mail, its attachments and any copies.Any use, distribution, reproduction or disclosure by any person other than the intended recipient is strictly prohibited and the person responsible may incur penalties. 
Thank you!
 
 
Message: 1
Date: Mon, 16 Aug 2021 16:23:45 +From: To: , Subject: Re: [RADIATOR] AuthBy DUO issueMessage-ID:Content-Type: text/plain; charset="windows-1252"Hi,that sounds like a sane solution.A simpler might be to mark Duo dead for a configurable number of seconds after which it's marked as alive again without a check. The next authentication would then either work or again trigger marking it as dead.Thanks, AlexT-SYSTEMS AUSTRIA GESMBHPU Cyber SecurityNetwork ArchitectureOperation Manager AuthenticationRennweg 97-99, A-1030 Vienna+43 57057 4320 (phone)+43 676 8642 4320 (mobile)E-mail: alexander.hartma...@t-systems.comInternet: www.t-systems.atBlog: blog.t-systems.atSocial Media: Facebook, Linkedin, TwitterBIG CHANGES START SMALL ? CONSERVE RESOURCES BY NOT PRINTING EVERY E-MAIL.T-Systems Austria GesmbH, Rennweg 97-99, A-1030 ViennaCommercial Court Vienna, FN 79340bNotice: This transmittal and/or attachments may be privileged or confidential. It isintended solely for the addressee named above. If you received this transmittal in error,please notify us immediately by reply and delete this message and all its attachments.Thank you.Von: radiator  im Auftrag von Heikki Vatiainen Gesendet: Mittwoch, 14. Juli 2021 20:26An: radiator@lists.open.com.au Betreff: Re: [RADIATOR] AuthBy DUO issueOn 13.7.2021 18.05, alexander.hartma...@t-systems.com wrote:> We've encountered another issue today: when CheckTimerInterval is> configured to 0, to disable the periodic DUO API check which fills our> log and generated unnecessary traffic and load, the API never recovers> when marked as dead.That seems to be correct, but likely not expected.> Do you have a suggestion how to solve this besides configuring> CheckTimerInterval for something else?Currently there is nothing to solve this. A strategy, such as startingthe poll timer when the API is down and letting it poll until it's up,would be needed.If you have a preferred idea, please let us know.Thanks,Heikki--Heikki Vatiainen Radiator: the most portable, flexible and configurable RADIUS serveranywhere. SQL, proxy, DBM, files, LDAP, TACACS+, PAM, Active Directory,EAP, TLS, TTLS, PEAP, WiMAX, RSA, Vasco, Yubikey, HOTP, TOTP,DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, etc.___

___
radiator mailing list
radiator@lists.open.com.au
https://lists.open.com.au/mailman/listinfo/radiator

Re: [RADIATOR] AuthBy DUO issue

[Alexander.Hartmaier]

Hi,
that sounds like a sane solution.

A simpler might be to mark Duo dead for a configurable number of seconds after 
which it's marked as alive again without a check. The next authentication would 
then either work or again trigger marking it as dead.

Thanks, Alex

T-SYSTEMS AUSTRIA GESMBH
PU Cyber Security
Network Architecture
Operation Manager Authentication
Rennweg 97-99, A-1030 Vienna
+43 57057 4320 (phone)
+43 676 8642 4320 (mobile)
E-mail: alexander.hartma...@t-systems.com
Internet: www.t-systems.at
Blog: blog.t-systems.at
Social Media: Facebook, Linkedin, Twitter

BIG CHANGES START SMALL – CONSERVE RESOURCES BY NOT PRINTING EVERY E-MAIL.


T-Systems Austria GesmbH, Rennweg 97-99, A-1030 Vienna
Commercial Court Vienna, FN 79340b

Notice: This transmittal and/or attachments may be privileged or confidential. 
It is
intended solely for the addressee named above. If you received this transmittal 
in error,
please notify us immediately by reply and delete this message and all its 
attachments.
Thank you.


Von: radiator  im Auftrag von Heikki 
Vatiainen 
Gesendet: Mittwoch, 14. Juli 2021 20:26
An: radiator@lists.open.com.au 
Betreff: Re: [RADIATOR] AuthBy DUO issue

On 13.7.2021 18.05, alexander.hartma...@t-systems.com wrote:

> We've encountered another issue today: when CheckTimerInterval is
> configured to 0, to disable the periodic DUO API check which fills our
> log and generated unnecessary traffic and load, the API never recovers
> when marked as dead.

That seems to be correct, but likely not expected.

> Do you have a suggestion how to solve this besides configuring
> CheckTimerInterval for something else?

Currently there is nothing to solve this. A strategy, such as starting
the poll timer when the API is down and letting it poll until it's up,
would be needed.

If you have a preferred idea, please let us know.

Thanks,
Heikki

--
Heikki Vatiainen 

Radiator: the most portable, flexible and configurable RADIUS server
anywhere. SQL, proxy, DBM, files, LDAP, TACACS+, PAM, Active Directory,
EAP, TLS, TTLS, PEAP, WiMAX, RSA, Vasco, Yubikey, HOTP, TOTP,
DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, etc.
___
radiator mailing list
radiator@lists.open.com.au
https://lists.open.com.au/mailman/listinfo/radiator
___
radiator mailing list
radiator@lists.open.com.au
https://lists.open.com.au/mailman/listinfo/radiator

Re: [RADIATOR] AuthBy DUO issue

[Heikki Vatiainen]

On 13.7.2021 18.05, alexander.hartma...@t-systems.com wrote:

We've encountered another issue today: when CheckTimerInterval is 
configured to 0, to disable the periodic DUO API check which fills our 
log and generated unnecessary traffic and load, the API never recovers 
when marked as dead.


That seems to be correct, but likely not expected.

Do you have a suggestion how to solve this besides configuring 
CheckTimerInterval for something else?


Currently there is nothing to solve this. A strategy, such as starting 
the poll timer when the API is down and letting it poll until it's up, 
would be needed.


If you have a preferred idea, please let us know.

Thanks,
Heikki

--
Heikki Vatiainen 

Radiator: the most portable, flexible and configurable RADIUS server
anywhere. SQL, proxy, DBM, files, LDAP, TACACS+, PAM, Active Directory,
EAP, TLS, TTLS, PEAP, WiMAX, RSA, Vasco, Yubikey, HOTP, TOTP,
DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, etc.
___
radiator mailing list
radiator@lists.open.com.au
https://lists.open.com.au/mailman/listinfo/radiator

Re: [RADIATOR] AuthBy DUO issue

[Alexander.Hartmaier]

Hi Heikki,
thanks for the new version, we'll look into deploying it.

We've encountered another issue today: when CheckTimerInterval is configured to 
0, to disable the periodic DUO API check which fills our log and generated 
unnecessary traffic and load, the API never recovers when marked as dead.

Do you have a suggestion how to solve this besides configuring 
CheckTimerInterval for something else?

Thanks, Alex

T-SYSTEMS AUSTRIA GESMBH
PU Cyber Security
Network Architecture
Operation Manager Authentication
Rennweg 97-99, A-1030 Vienna
+43 57057 4320 (phone)
+43 676 8642 4320 (mobile)
E-mail: alexander.hartma...@t-systems.com
Internet: www.t-systems.at
Blog: blog.t-systems.at
Social Media: Facebook, Linkedin, Twitter

BIG CHANGES START SMALL – CONSERVE RESOURCES BY NOT PRINTING EVERY E-MAIL.


T-Systems Austria GesmbH, Rennweg 97-99, A-1030 Vienna
Commercial Court Vienna, FN 79340b

Notice: This transmittal and/or attachments may be privileged or confidential. 
It is
intended solely for the addressee named above. If you received this transmittal 
in error,
please notify us immediately by reply and delete this message and all its 
attachments.
Thank you.


Von: radiator  im Auftrag von Heikki 
Vatiainen 
Gesendet: Mittwoch, 30. Juni 2021 18:48
An: radiator@lists.open.com.au 
Betreff: Re: [RADIATOR] AuthBy DUO issue

On 11.6.2021 14.42, Heikki Vatiainen wrote:
> On 8.6.2021 15.06, alexander.hartma...@t-systems.com wrote:

>> What is your plan to fix this issue?
>
> One option is to select only TLSv1.2 by default and make it
> configurable. If the problem is with Net::HTTPS::NB or HTTP::Async,
> allow by default TLSv1.3 when a working version of this/those is detected.
AuthBy DUO now disables TLSv1.3 until a better fix is available. DUO
seems to support TLS versions older than TLSv1.2 so for this reason
TLSv1.2 is not forced by Radiator. Those who still require older
versions can still continue to use them. TLS library continues to
negotiate the highest version that's available.

Updated packages are available from Radiator patch package downloads.

Thanks,
Heikki

--
Heikki Vatiainen 

Radiator: the most portable, flexible and configurable RADIUS server
anywhere. SQL, proxy, DBM, files, LDAP, TACACS+, PAM, Active Directory,
EAP, TLS, TTLS, PEAP, WiMAX, RSA, Vasco, Yubikey, HOTP, TOTP,
DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, etc.
___
radiator mailing list
radiator@lists.open.com.au
https://lists.open.com.au/mailman/listinfo/radiator
___
radiator mailing list
radiator@lists.open.com.au
https://lists.open.com.au/mailman/listinfo/radiator

Re: [RADIATOR] AuthBy DUO issue

[Heikki Vatiainen]

On 11.6.2021 14.42, Heikki Vatiainen wrote:

On 8.6.2021 15.06, alexander.hartma...@t-systems.com wrote:



What is your plan to fix this issue?


One option is to select only TLSv1.2 by default and make it 
configurable. If the problem is with Net::HTTPS::NB or HTTP::Async, 
allow by default TLSv1.3 when a working version of this/those is detected.
AuthBy DUO now disables TLSv1.3 until a better fix is available. DUO 
seems to support TLS versions older than TLSv1.2 so for this reason 
TLSv1.2 is not forced by Radiator. Those who still require older 
versions can still continue to use them. TLS library continues to 
negotiate the highest version that's available.


Updated packages are available from Radiator patch package downloads.

Thanks,
Heikki

--
Heikki Vatiainen 

Radiator: the most portable, flexible and configurable RADIUS server
anywhere. SQL, proxy, DBM, files, LDAP, TACACS+, PAM, Active Directory,
EAP, TLS, TTLS, PEAP, WiMAX, RSA, Vasco, Yubikey, HOTP, TOTP,
DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, etc.
___
radiator mailing list
radiator@lists.open.com.au
https://lists.open.com.au/mailman/listinfo/radiator

Re: [RADIATOR] AuthBy DUO issue

[Heikki Vatiainen]

On 8.6.2021 15.06, alexander.hartma...@t-systems.com wrote:

I think this is a good explanation what I think might be happening with
[1] below.

https://wiki.openssl.org/index.php/TLS1.3#Non-application_data_records 



That makes sense!
But when OpenSSL receives and reads that data, shouldn't the socket stop 
reporting available data?


I'd say what happens is that when the module reads the socket and 
OpenSSL machinery is rotated consuming all input from the socket, there 
is no user data left and read for user data does not return until some 
is left. That would be when a response comes from DUO. Or something 
similar is happening. I think OpenSSL does not support returning zero 
length data.


It's not doing a busy loop trying to read the socket, so it seems to 
block somehow.


It might be that the assumption in the module is that when a socket is 
readable after TLS handshake, it means that there's data or the 
connection was closed. It may not be prepared for nothing but a 
handshake message.


Caution: I haven't yet looked into this in detail.


What is your plan to fix this issue?


One option is to select only TLSv1.2 by default and make it 
configurable. If the problem is with Net::HTTPS::NB or HTTP::Async, 
allow by default TLSv1.3 when a working version of this/those is detected.


Will you provide a patch for HTTP::Async or migrate AuthDUO.pm to for 
example AuthREST.pm?


HTTP::Async and/or Net::HTTPS::NB would need a fix for current 
installations. The AuthREST.pm, actually DUO AuthBy built on top of 
HTTPClient.pm is something we have considered too. We now have HTTP 
client support for exactly these kinds of things.


Thanks again for following up on how it goes now.


[1] 
https://metacpan.org/release/HTTP-Async/source/lib/HTTP/Async.pm#L551 




--
Heikki Vatiainen 

Radiator: the most portable, flexible and configurable RADIUS server
anywhere. SQL, proxy, DBM, files, LDAP, TACACS+, PAM, Active Directory,
EAP, TLS, TTLS, PEAP, WiMAX, RSA, Vasco, Yubikey, HOTP, TOTP,
DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, etc.
___
radiator mailing list
radiator@lists.open.com.au
https://lists.open.com.au/mailman/listinfo/radiator

Re: [RADIATOR] AuthBy DUO issue

[Heikki Vatiainen]

On 1.6.2021 19.23, alexander.hartma...@t-systems.com wrote:


today at noon I had the change to reenable Cisco Duo and set TLS to v1.2.
So far it looks good, I saw other authentication requests getting 
processed while AuthBy DUO waited for a user response.


Thanks for the update. Please keep us posted if the situation changes.


I haven't grapsed so far how TLSv1.3 could cause this bug?
If I understood it correctly, the underlying socket reports available 
data via select although none has been received so far [1].

Is that correct?


That's my hunch too. That is, I did not verify it throughly but it 
looked familiar.



How does the TLS version influence that?


TLSv1.3 is much different from SSL3.0 - TLSv1.2. One major difference is 
the way session resumption works with TLSv1.3. With the other versions, 
information needed for resumption was communicated between the peers 
during the TLS handshake. With TLSv1.3 the handshake complets first and 
then the optional resumption information (Pre-Shared Keys) are sent by 
the server. The main thing is that it's now normal to receive non-data 
records after handshake has finished.


With TLSv1.3, when the the socket indicates it's readable, there's no 
userdata and the read blocks. It does update the session's internal 
state and ability to resume, though.


I think this is a good explanation what I think might be happening with 
[1] below.


https://wiki.openssl.org/index.php/TLS1.3#Non-application_data_records


[1] 
https://metacpan.org/release/HTTP-Async/source/lib/HTTP/Async.pm#L551 


Thanks,
Heikki

--
Heikki Vatiainen 

Radiator: the most portable, flexible and configurable RADIUS server
anywhere. SQL, proxy, DBM, files, LDAP, TACACS+, PAM, Active Directory,
EAP, TLS, TTLS, PEAP, WiMAX, RSA, Vasco, Yubikey, HOTP, TOTP,
DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, etc.
___
radiator mailing list
radiator@lists.open.com.au
https://lists.open.com.au/mailman/listinfo/radiator

Re: [RADIATOR] AuthBy DUO issue

[Alexander.Hartmaier]

Hi Heikki,
today at noon I had the change to reenable Cisco Duo and set TLS to v1.2.
So far it looks good, I saw other authentication requests getting processed 
while AuthBy DUO waited for a user response.

I haven't grapsed so far how TLSv1.3 could cause this bug?
If I understood it correctly, the underlying socket reports available data via 
select although none has been received so far [1].
Is that correct?
How does the TLS version influence that?

Best regards, Alex

[1] https://metacpan.org/release/HTTP-Async/source/lib/HTTP/Async.pm#L551

T-SYSTEMS AUSTRIA GESMBH
PU Cyber Security
Network Architecture
Operation Manager Authentication
Rennweg 97-99, A-1030 Vienna
+43 57057 4320 (phone)
+43 676 8642 4320 (mobile)
E-mail: alexander.hartma...@t-systems.com
Internet: www.t-systems.at
Blog: blog.t-systems.at
Social Media: Facebook, Linkedin, Twitter

BIG CHANGES START SMALL – CONSERVE RESOURCES BY NOT PRINTING EVERY E-MAIL.


T-Systems Austria GesmbH, Rennweg 97-99, A-1030 Vienna
Commercial Court Vienna, FN 79340b

Notice: This transmittal and/or attachments may be privileged or confidential. 
It is
intended solely for the addressee named above. If you received this transmittal 
in error,
please notify us immediately by reply and delete this message and all its 
attachments.
Thank you.


Von: radiator  im Auftrag von Hartmaier, 
Alexander 
Gesendet: Freitag, 28. Mai 2021 09:25
An: h...@open.com.au ; radiator@lists.open.com.au 

Betreff: Re: [RADIATOR] AuthBy DUO issue

Good morning Heikki,
awesome support from you as always, thank you!!!

I saw that the connection to Duo is TLS 1.3 in the packet captures I've taken.
Will try your suggestion and report back.

Best regards, Alex

T-SYSTEMS AUSTRIA GESMBH
PU Cyber Security
Network Architecture
Operation Manager Authentication
Rennweg 97-99, A-1030 Vienna
+43 57057 4320 (phone)
+43 676 8642 4320 (mobile)
E-mail: alexander.hartma...@t-systems.com
Internet: www.t-systems.at
Blog: blog.t-systems.at
Social Media: Facebook, Linkedin, Twitter

BIG CHANGES START SMALL – CONSERVE RESOURCES BY NOT PRINTING EVERY E-MAIL.


T-Systems Austria GesmbH, Rennweg 97-99, A-1030 Vienna
Commercial Court Vienna, FN 79340b

Notice: This transmittal and/or attachments may be privileged or confidential. 
It is
intended solely for the addressee named above. If you received this transmittal 
in error,
please notify us immediately by reply and delete this message and all its 
attachments.
Thank you.


Von: radiator  im Auftrag von Heikki 
Vatiainen 
Gesendet: Donnerstag, 27. Mai 2021 18:57
An: radiator@lists.open.com.au 
Betreff: Re: [RADIATOR] AuthBy DUO issue

On 27.5.2021 19.36, Heikki Vatiainen wrote:
> On 27.5.2021 14.58, alexander.hartma...@t-systems.com wrote:

>> Is this a known issue?

> As mentioned above, it's not. From what I know it's been used
> successfully on RHEL/CentOS systems and it works for me on Mac.

The problem might be TLS version related. The above don't do TLSv1.3.

> I'd say this is something specific for Debian 10 because the problem is
> not that hard to reproduce. This needs further investigation.

If possible, can you update AuthDUO.pm sub get_ssl_opts() with the
following:

   $ssl_opts{SSL_version} = 'TLSv1_2';

This kind of behaviour where TLS socket indicates read but there's no
user data available reminded me about TLS 1.3 and how it can send keys
for session resumption after TLS handshake has been done.

A look at HTTPS traffic shows that there's both TLS 1.2 and 1.3 by
default. Restricting TLS to 1.2 seems to make the problem go away.

If you could also check this, please let me know if it changes anything.

Thanks,
Heikki


--
Heikki Vatiainen 

Radiator: the most portable, flexible and configurable RADIUS server
anywhere. SQL, proxy, DBM, files, LDAP, TACACS+, PAM, Active Directory,
EAP, TLS, TTLS, PEAP, WiMAX, RSA, Vasco, Yubikey, HOTP, TOTP,
DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, etc.
___
radiator mailing list
radiator@lists.open.com.au
https://lists.open.com.au/mailman/listinfo/radiator
___
radiator mailing list
radiator@lists.open.com.au
https://lists.open.com.au/mailman/listinfo/radiator

Re: [RADIATOR] AuthBy DUO issue

[Alexander.Hartmaier]

Good morning Heikki,
awesome support from you as always, thank you!!!

I saw that the connection to Duo is TLS 1.3 in the packet captures I've taken.
Will try your suggestion and report back.

Best regards, Alex

T-SYSTEMS AUSTRIA GESMBH
PU Cyber Security
Network Architecture
Operation Manager Authentication
Rennweg 97-99, A-1030 Vienna
+43 57057 4320 (phone)
+43 676 8642 4320 (mobile)
E-mail: alexander.hartma...@t-systems.com
Internet: www.t-systems.at
Blog: blog.t-systems.at
Social Media: Facebook, Linkedin, Twitter

BIG CHANGES START SMALL – CONSERVE RESOURCES BY NOT PRINTING EVERY E-MAIL.


T-Systems Austria GesmbH, Rennweg 97-99, A-1030 Vienna
Commercial Court Vienna, FN 79340b

Notice: This transmittal and/or attachments may be privileged or confidential. 
It is
intended solely for the addressee named above. If you received this transmittal 
in error,
please notify us immediately by reply and delete this message and all its 
attachments.
Thank you.


Von: radiator  im Auftrag von Heikki 
Vatiainen 
Gesendet: Donnerstag, 27. Mai 2021 18:57
An: radiator@lists.open.com.au 
Betreff: Re: [RADIATOR] AuthBy DUO issue

On 27.5.2021 19.36, Heikki Vatiainen wrote:
> On 27.5.2021 14.58, alexander.hartma...@t-systems.com wrote:

>> Is this a known issue?

> As mentioned above, it's not. From what I know it's been used
> successfully on RHEL/CentOS systems and it works for me on Mac.

The problem might be TLS version related. The above don't do TLSv1.3.

> I'd say this is something specific for Debian 10 because the problem is
> not that hard to reproduce. This needs further investigation.

If possible, can you update AuthDUO.pm sub get_ssl_opts() with the
following:

   $ssl_opts{SSL_version} = 'TLSv1_2';

This kind of behaviour where TLS socket indicates read but there's no
user data available reminded me about TLS 1.3 and how it can send keys
for session resumption after TLS handshake has been done.

A look at HTTPS traffic shows that there's both TLS 1.2 and 1.3 by
default. Restricting TLS to 1.2 seems to make the problem go away.

If you could also check this, please let me know if it changes anything.

Thanks,
Heikki


--
Heikki Vatiainen 

Radiator: the most portable, flexible and configurable RADIUS server
anywhere. SQL, proxy, DBM, files, LDAP, TACACS+, PAM, Active Directory,
EAP, TLS, TTLS, PEAP, WiMAX, RSA, Vasco, Yubikey, HOTP, TOTP,
DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, etc.
___
radiator mailing list
radiator@lists.open.com.au
https://lists.open.com.au/mailman/listinfo/radiator
___
radiator mailing list
radiator@lists.open.com.au
https://lists.open.com.au/mailman/listinfo/radiator

Re: [RADIATOR] AuthBy DUO issue

[Heikki Vatiainen]

On 27.5.2021 19.36, Heikki Vatiainen wrote:

On 27.5.2021 14.58, alexander.hartma...@t-systems.com wrote:



Is this a known issue?


As mentioned above, it's not. From what I know it's been used 
successfully on RHEL/CentOS systems and it works for me on Mac.


The problem might be TLS version related. The above don't do TLSv1.3.

I'd say this is something specific for Debian 10 because the problem is 
not that hard to reproduce. This needs further investigation.


If possible, can you update AuthDUO.pm sub get_ssl_opts() with the 
following:


  $ssl_opts{SSL_version} = 'TLSv1_2';

This kind of behaviour where TLS socket indicates read but there's no 
user data available reminded me about TLS 1.3 and how it can send keys 
for session resumption after TLS handshake has been done.


A look at HTTPS traffic shows that there's both TLS 1.2 and 1.3 by 
default. Restricting TLS to 1.2 seems to make the problem go away.


If you could also check this, please let me know if it changes anything.

Thanks,
Heikki


--
Heikki Vatiainen 

Radiator: the most portable, flexible and configurable RADIUS server
anywhere. SQL, proxy, DBM, files, LDAP, TACACS+, PAM, Active Directory,
EAP, TLS, TTLS, PEAP, WiMAX, RSA, Vasco, Yubikey, HOTP, TOTP,
DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, etc.
___
radiator mailing list
radiator@lists.open.com.au
https://lists.open.com.au/mailman/listinfo/radiator

Re: [RADIATOR] AuthBy DUO issue

[Heikki Vatiainen]

On 27.5.2021 14.58, alexander.hartma...@t-systems.com wrote:

I've tracked down the issue to the poke call at the beginning of 
checkForResponses which doesn't return for half a minute sometimes::q


Thanks for the extensive debugging. I don't think this has been reported 
before. I started a new Debian 10, installed the same Radiator packages 
as you did. For AuthBy DUO specific dependencies, I used apt to install 
these: libhttp-async-perl and libnet-https-nb-perl


Attempting the authentication multiple times, I was able to reproduce 
the problem. It blocks until there's an answer from DUO API.


Now that the location has been narrowed down to _process_in_progress, a 
couple of questions and comments related to your previous message:


1. Does it stop right after it pushes the request to API? That is, not 
after a couple of polls that check for user action. That is how it seems 
to me. It does preauth and then the actual auth request blocks until 
there's some kind of response.


Does it fail constantly for you? I need to do multiple attempts to get 
it fail, but eventually it does. In most cases it works as expected.


2. Trace ID values with all zeroes are logged when the logging does not 
have a current request available. For example, periodic polling uses all 
zeroes.



 Thu May 27 11:24:19 2021: DEBUG: before poke
 Thu May 27 11:24:19 2021: DEBUG: after poke
 Thu May 27 11:24:19 2021: DEBUG: after while loop: 0
 Thu May 27 11:24:19 2021: DEBUG: before poke
 Thu May 27 11:24:52 2021: DEBUG: after poke
 Thu May 27 11:24:52 2021: DEBUG: 200 OK

Digging deeper revealed _process_in_progress is the function called by 
poke which doesn´t return in a timely manner.


Is this a known issue?
As mentioned above, it's not. From what I know it's been used 
successfully on RHEL/CentOS systems and it works for me on Mac.



Is forking recommended for AuthBy DUO? So far we don´t have Fork in use.


Fork appears not to work with this AuthBy. The forked instance does not 
stays around to do polling.


I'd say this is something specific for Debian 10 because the problem is 
not that hard to reproduce. This needs further investigation.


Thanks,
Heikki

*Von:* radiator  im Auftrag von 
Hartmaier, Alexander 

*Gesendet:* Donnerstag, 27. Mai 2021 12:52
*An:* radiator@lists.open.com.au 
*Betreff:* [RADIATOR] AuthBy DUO issue
Hi,
today we experienced an issue where two handlers using AuthBy DUO 
blocked a whole radiator instance.
It seems to be triggerend when a user doesn't response to the push 
notification.

As Radiator is using HTTP::Async this shouldn't happen.
A packet capture of the Duo https api calls and level 5 Radiator trace 
shows that the response to the POST takes 60 seconds and contains the 
status_msg: "Login timed out.".

During those 60 seconds no other radius requests are handled.

This instance is running Debian 10 with the radiator_4.25-5_all.deb and 
radiator-radius-utilxs_2.3-1.buster_amd64.deb packages.


Any ideas what's causing this? I'm out of ideas after reading lots of 
HTTP::Async and Radiator source code.


What I noticed it that the level 5 LOG_EXTRA_DEBUG messages are missing 
the LogTraceId value.


Thanks, Alex


--
Heikki Vatiainen 

Radiator: the most portable, flexible and configurable RADIUS server
anywhere. SQL, proxy, DBM, files, LDAP, TACACS+, PAM, Active Directory,
EAP, TLS, TTLS, PEAP, WiMAX, RSA, Vasco, Yubikey, HOTP, TOTP,
DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, etc.
___
radiator mailing list
radiator@lists.open.com.au
https://lists.open.com.au/mailman/listinfo/radiator

Re: [RADIATOR] AuthBy DUO issue

[Alexander.Hartmaier]

I've tracked down the issue to the poke call at the beginning of 
checkForResponses which doesn't return for half a minute sometimes::q

 Thu May 27 11:24:19 2021: DEBUG: before poke
 Thu May 27 11:24:19 2021: DEBUG: after poke
 Thu May 27 11:24:19 2021: DEBUG: after while loop: 0
 Thu May 27 11:24:19 2021: DEBUG: before poke
 Thu May 27 11:24:52 2021: DEBUG: after poke
 Thu May 27 11:24:52 2021: DEBUG: 200 OK

Digging deeper revealed _process_in_progress is the function called by poke 
which doesn´t return in a timely manner.

Is this a known issue?
Is forking recommended for AuthBy DUO? So far we don´t have Fork in use.

Thanks, Alex

T-SYSTEMS AUSTRIA GESMBH
PU Cyber Security
Network Architecture
Operation Manager Authentication
Rennweg 97-99, A-1030 Vienna
+43 57057 4320 (phone)
+43 676 8642 4320 (mobile)
E-mail: alexander.hartma...@t-systems.com
Internet: www.t-systems.at
Blog: blog.t-systems.at
Social Media: Facebook, Linkedin, Twitter

BIG CHANGES START SMALL – CONSERVE RESOURCES BY NOT PRINTING EVERY E-MAIL.


T-Systems Austria GesmbH, Rennweg 97-99, A-1030 Vienna
Commercial Court Vienna, FN 79340b

Notice: This transmittal and/or attachments may be privileged or confidential. 
It is
intended solely for the addressee named above. If you received this transmittal 
in error,
please notify us immediately by reply and delete this message and all its 
attachments.
Thank you.


Von: radiator  im Auftrag von Hartmaier, 
Alexander 
Gesendet: Donnerstag, 27. Mai 2021 12:52
An: radiator@lists.open.com.au 
Betreff: [RADIATOR] AuthBy DUO issue

Hi,
today we experienced an issue where two handlers using AuthBy DUO blocked a 
whole radiator instance.
It seems to be triggerend when a user doesn't response to the push notification.
As Radiator is using HTTP::Async this shouldn't happen.
A packet capture of the Duo https api calls and level 5 Radiator trace shows 
that the response to the POST takes 60 seconds and contains the status_msg: 
"Login timed out.".
During those 60 seconds no other radius requests are handled.

This instance is running Debian 10 with the radiator_4.25-5_all.deb and 
radiator-radius-utilxs_2.3-1.buster_amd64.deb packages.

Any ideas what's causing this? I'm out of ideas after reading lots of 
HTTP::Async and Radiator source code.

What I noticed it that the level 5 LOG_EXTRA_DEBUG messages are missing the 
LogTraceId value.

Thanks, Alex

T-SYSTEMS AUSTRIA GESMBH
PU Cyber Security
Network Architecture
Operation Manager Authentication
Rennweg 97-99, A-1030 Vienna
+43 57057 4320 (phone)
+43 676 8642 4320 (mobile)
E-mail: alexander.hartma...@t-systems.com
Internet: www.t-systems.at
Blog: blog.t-systems.at
Social Media: Facebook, Linkedin, Twitter

BIG CHANGES START SMALL – CONSERVE RESOURCES BY NOT PRINTING EVERY E-MAIL.


T-Systems Austria GesmbH, Rennweg 97-99, A-1030 Vienna
Commercial Court Vienna, FN 79340b

Notice: This transmittal and/or attachments may be privileged or confidential. 
It is
intended solely for the addressee named above. If you received this transmittal 
in error,
please notify us immediately by reply and delete this message and all its 
attachments.
Thank you.

___
radiator mailing list
radiator@lists.open.com.au
https://lists.open.com.au/mailman/listinfo/radiator