[jira] [Commented] (SSHD-969) compatibility problem with cisco-2.0 ssh on ios-xr
[
https://issues.apache.org/jira/browse/SSHD-969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17050491#comment-17050491
]
Yuefeng commented on SSHD-969:
--
Found the problem -
[https://www.cisco.com/en/US/docs/routers/xr12000/software/xr12k_r4.0/security/command/reference/sr40xr12kbook_chapter9.html#wp368875508]
ios-xr implicitly rate limits ssh connections to 60/min (implemented as 1
connection per second). Connections attempts above the rate gets "reset".
> compatibility problem with cisco-2.0 ssh on ios-xr
> ---
>
> Key: SSHD-969
> URL: https://issues.apache.org/jira/browse/SSHD-969
> Project: MINA SSHD
> Issue Type: Bug
>Reporter: Yuefeng
>Priority: Major
>
> When mina ssh library is used to connect with a Cisco IOS-XR device, the
> connection is consistently reset by IOS-XR, here's mina log
>
>
> {code:java}
> 2020-02-25 20:12:15.531Z [sshd-SshClient[468ca10d]-nio2-thread-1] DEBUG
> o.a.s.c.session.ClientSessionImpl:271 -
> initializeProxyConnector(ClientSessionImpl[null@/10.90.10.193:22]) no proxy
> to initialize
> 2020-02-25 20:12:15.531Z [pool-171-thread-1] INFO
> com.forwardnetworks.client.a.f.a.d:20 - Connected to 'xrv9k'
> 2020-02-25 20:12:15.531Z [pool-171-thread-1] DEBUG
> o.a.s.c.session.ClientSessionImpl:204 -
> addPasswordIdentity(ClientSessionImpl[*@/10.90.10.193:22]) SHA256:**
> 2020-02-25 20:12:15.532Z [pool-171-thread-1] INFO
> com.forwardnetworks.client.a.f.a.d:28 - Authenticating client session to
> xrv9k for up to 1 seconds.
> 2020-02-25 20:12:15.532Z [pool-171-thread-1] DEBUG
> o.a.s.c.s.ClientUserAuthService:162 -
> auth(ClientSessionImpl[cisco@/10.90.10.193:22])[ssh-connection] send
> SSH_MSG_USERAUTH_REQUEST for 'none'
> 2020-02-25 20:12:15.532Z [pool-171-thread-1] DEBUG
> o.a.s.c.session.ClientSessionImpl:808 -
> writePacket(ClientSessionImpl[cisco@/10.90.10.193:22])[SSH_MSG_USERAUTH_REQUEST]
> Start flagging packets as pending until key exchange is done
> 2020-02-25 20:12:15.533Z [sshd-SshClient[468ca10d]-nio2-thread-5] DEBUG
> o.a.s.c.session.ClientSessionImpl:167 -
> signalAuthFailure(ClientSessionImpl[cisco@/10.90.10.193:22])
> type=IOException, signalled=true: Connection reset by peer
> {code}
>
> Ssh (openssh) successfully connects to the save device
> {code:java}
> fwd-collector@collector:/usr/local/fwd/logs$ ssh -vvv @10.90.10.193
> OpenSSH_7.2p2 Ubuntu-4ubuntu2.7, OpenSSL 1.0.2g 1 Mar 2016
> debug1: Reading configuration data /etc/ssh/ssh_config
> debug1: /etc/ssh/ssh_config line 19: Applying options for *
> debug2: resolving "10.90.10.193" port 22
> debug2: ssh_connect_direct: needpriv 0
> debug1: Connecting to 10.90.10.193 [10.90.10.193] port 22.
> debug1: Connection established.
> debug1: key_load_public: No such file or directory
> debug1: identity file /home/fwd-collector/.ssh/id_rsa type -1
> debug1: key_load_public: No such file or directory
> debug1: identity file /home/fwd-collector/.ssh/id_rsa-cert type -1
> debug1: key_load_public: No such file or directory
> debug1: identity file /home/fwd-collector/.ssh/id_dsa type -1
> debug1: key_load_public: No such file or directory
> debug1: identity file /home/fwd-collector/.ssh/id_dsa-cert type -1
> debug1: key_load_public: No such file or directory
> debug1: identity file /home/fwd-collector/.ssh/id_ecdsa type -1
> debug1: key_load_public: No such file or directory
> debug1: identity file /home/fwd-collector/.ssh/id_ecdsa-cert type -1
> debug1: key_load_public: No such file or directory
> debug1: identity file /home/fwd-collector/.ssh/id_ed25519 type -1
> debug1: key_load_public: No such file or directory
> debug1: identity file /home/fwd-collector/.ssh/id_ed25519-cert type -1
> debug1: Enabling compatibility mode for protocol 2.0
> debug1: Local version string SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.7
> debug1: Remote protocol version 2.0, remote software version Cisco-2.0
> debug1: no match: Cisco-2.0
> debug2: fd 3 setting O_NONBLOCK
> debug1: Authenticating to 10.90.10.193:22 as 'admin'
> debug3: hostkeys_foreach: reading file "/home/fwd-collector/.ssh/known_hosts"
> debug3: record_hostkey: found key type RSA in file
> /home/fwd-collector/.ssh/known_hosts:28
> debug3: load_hostkeys: loaded 1 keys from 10.90.10.193
> debug3: order_hostkeyalgs: prefer hostkeyalgs:
> ssh-rsa-cert-...@openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-rsa
> debug3: send packet: type 20
> debug1: SSH2_MSG_KEXINIT sent
> debug3: receive packet: type 20
> debug1: SSH2_MSG_KEXINIT received
> debug2: local client KEXINIT proposal
> debug2: KEX algorithms:
> curve25519-sha...@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,ext-info-c
> debug2: host key algorithms:
>
[jira] [Commented] (SSHD-969) compatibility problem with cisco-2.0 ssh on ios-xr
[
https://issues.apache.org/jira/browse/SSHD-969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047786#comment-17047786
]
Lyor Goldstein commented on SSHD-969:
-
There is one more thing you should verify - the KEX options being sent by the
MINA SSHD client (e.g., supported ciphers, MAC, etc.) - the server might have
problems with one of the options and thus close the connection. While this
behavior is a violation of the SSH protocol, some servers do mis-behave. See
also [SSHD-930|https://issues.apache.org/jira/browse/SSHD-930]
Finally, if there is any way to get a hold of the server side logs and view
them it might also help diagnose the issue.
> compatibility problem with cisco-2.0 ssh on ios-xr
> ---
>
> Key: SSHD-969
> URL: https://issues.apache.org/jira/browse/SSHD-969
> Project: MINA SSHD
> Issue Type: Bug
>Reporter: Yuefeng
>Priority: Major
>
> When mina ssh library is used to connect with a Cisco IOS-XR device, the
> connection is consistently reset by IOS-XR, here's mina log
>
>
> {code:java}
> 2020-02-25 20:12:15.531Z [sshd-SshClient[468ca10d]-nio2-thread-1] DEBUG
> o.a.s.c.session.ClientSessionImpl:271 -
> initializeProxyConnector(ClientSessionImpl[null@/10.90.10.193:22]) no proxy
> to initialize
> 2020-02-25 20:12:15.531Z [pool-171-thread-1] INFO
> com.forwardnetworks.client.a.f.a.d:20 - Connected to 'xrv9k'
> 2020-02-25 20:12:15.531Z [pool-171-thread-1] DEBUG
> o.a.s.c.session.ClientSessionImpl:204 -
> addPasswordIdentity(ClientSessionImpl[*@/10.90.10.193:22]) SHA256:**
> 2020-02-25 20:12:15.532Z [pool-171-thread-1] INFO
> com.forwardnetworks.client.a.f.a.d:28 - Authenticating client session to
> xrv9k for up to 1 seconds.
> 2020-02-25 20:12:15.532Z [pool-171-thread-1] DEBUG
> o.a.s.c.s.ClientUserAuthService:162 -
> auth(ClientSessionImpl[cisco@/10.90.10.193:22])[ssh-connection] send
> SSH_MSG_USERAUTH_REQUEST for 'none'
> 2020-02-25 20:12:15.532Z [pool-171-thread-1] DEBUG
> o.a.s.c.session.ClientSessionImpl:808 -
> writePacket(ClientSessionImpl[cisco@/10.90.10.193:22])[SSH_MSG_USERAUTH_REQUEST]
> Start flagging packets as pending until key exchange is done
> 2020-02-25 20:12:15.533Z [sshd-SshClient[468ca10d]-nio2-thread-5] DEBUG
> o.a.s.c.session.ClientSessionImpl:167 -
> signalAuthFailure(ClientSessionImpl[cisco@/10.90.10.193:22])
> type=IOException, signalled=true: Connection reset by peer
> {code}
>
> Ssh (openssh) successfully connects to the save device
> {code:java}
> fwd-collector@collector:/usr/local/fwd/logs$ ssh -vvv @10.90.10.193
> OpenSSH_7.2p2 Ubuntu-4ubuntu2.7, OpenSSL 1.0.2g 1 Mar 2016
> debug1: Reading configuration data /etc/ssh/ssh_config
> debug1: /etc/ssh/ssh_config line 19: Applying options for *
> debug2: resolving "10.90.10.193" port 22
> debug2: ssh_connect_direct: needpriv 0
> debug1: Connecting to 10.90.10.193 [10.90.10.193] port 22.
> debug1: Connection established.
> debug1: key_load_public: No such file or directory
> debug1: identity file /home/fwd-collector/.ssh/id_rsa type -1
> debug1: key_load_public: No such file or directory
> debug1: identity file /home/fwd-collector/.ssh/id_rsa-cert type -1
> debug1: key_load_public: No such file or directory
> debug1: identity file /home/fwd-collector/.ssh/id_dsa type -1
> debug1: key_load_public: No such file or directory
> debug1: identity file /home/fwd-collector/.ssh/id_dsa-cert type -1
> debug1: key_load_public: No such file or directory
> debug1: identity file /home/fwd-collector/.ssh/id_ecdsa type -1
> debug1: key_load_public: No such file or directory
> debug1: identity file /home/fwd-collector/.ssh/id_ecdsa-cert type -1
> debug1: key_load_public: No such file or directory
> debug1: identity file /home/fwd-collector/.ssh/id_ed25519 type -1
> debug1: key_load_public: No such file or directory
> debug1: identity file /home/fwd-collector/.ssh/id_ed25519-cert type -1
> debug1: Enabling compatibility mode for protocol 2.0
> debug1: Local version string SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.7
> debug1: Remote protocol version 2.0, remote software version Cisco-2.0
> debug1: no match: Cisco-2.0
> debug2: fd 3 setting O_NONBLOCK
> debug1: Authenticating to 10.90.10.193:22 as 'admin'
> debug3: hostkeys_foreach: reading file "/home/fwd-collector/.ssh/known_hosts"
> debug3: record_hostkey: found key type RSA in file
> /home/fwd-collector/.ssh/known_hosts:28
> debug3: load_hostkeys: loaded 1 keys from 10.90.10.193
> debug3: order_hostkeyalgs: prefer hostkeyalgs:
> ssh-rsa-cert-...@openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-rsa
> debug3: send packet: type 20
> debug1: SSH2_MSG_KEXINIT sent
> debug3: receive packet: type 20
> debug1: SSH2_MSG_KEXINIT received
> debug2: local client KEXINIT proposal
> debug2: KEX algorithms:
>
[jira] [Commented] (SSHD-969) compatibility problem with cisco-2.0 ssh on ios-xr
[
https://issues.apache.org/jira/browse/SSHD-969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046686#comment-17046686
]
Lyor Goldstein commented on SSHD-969:
-
{quote}
I noticed that Mina SSH sends msg 50 (SSH_MSG_USERAUTH_REQUEST) before key
exchange completes
{quote}
I cannot figure out what scenario can lead to this - authentication messages
are exchanged +after+ keys have been exchanged - never over insecure channel.
We have such checks in code:
{code:java}
@Override
public Boolean doAuth(Buffer buffer, boolean init) throws Exception {
ValidateUtils.checkTrue(init, "Instance not initialized");
ServerSession session = getServerSession();
if (!UserAuthMethodFactory.isSecureAuthenticationTransport(session)) {
if (log.isDebugEnabled()) {
log.debug("doAuth({}) session is not secure", session);
}
return false;
}
{code}
It may be the case that the server has some kind of race condition and has not
yet updated its internal state that KEX is completed when
{{SSH_MSG_USERAUTH_REQUEST}} arrives. Anyway, you have not mentioned what MINA
SSHD version you are using. Please try the latest one.
Another thing to notice is that some servers behave in a special way and expect
flows that are not always standard. To that effect, try and activate the
"delayed KEX" option in our code (see [this
link|https://github.com/apache/mina-sshd/blob/master/docs/client-setup.md#configuring-the-protocol-exchange-phase]).
This option delays KEX from the client side until the server has sent its
identification first. It might fix this issue entirely, but perhaps it may
introduce enough of a delay to overcome the suspected race condition in the
server.
> compatibility problem with cisco-2.0 ssh on ios-xr
> ---
>
> Key: SSHD-969
> URL: https://issues.apache.org/jira/browse/SSHD-969
> Project: MINA SSHD
> Issue Type: Bug
>Reporter: Yuefeng
>Priority: Major
>
> When mina ssh library is used to connect with a Cisco IOS-XR device, the
> connection is consistently reset by IOS-XR, here's mina log
>
>
> {code:java}
> 2020-02-25 20:12:15.531Z [sshd-SshClient[468ca10d]-nio2-thread-1] DEBUG
> o.a.s.c.session.ClientSessionImpl:271 -
> initializeProxyConnector(ClientSessionImpl[null@/10.90.10.193:22]) no proxy
> to initialize
> 2020-02-25 20:12:15.531Z [pool-171-thread-1] INFO
> com.forwardnetworks.client.a.f.a.d:20 - Connected to 'xrv9k'
> 2020-02-25 20:12:15.531Z [pool-171-thread-1] DEBUG
> o.a.s.c.session.ClientSessionImpl:204 -
> addPasswordIdentity(ClientSessionImpl[*@/10.90.10.193:22]) SHA256:**
> 2020-02-25 20:12:15.532Z [pool-171-thread-1] INFO
> com.forwardnetworks.client.a.f.a.d:28 - Authenticating client session to
> xrv9k for up to 1 seconds.
> 2020-02-25 20:12:15.532Z [pool-171-thread-1] DEBUG
> o.a.s.c.s.ClientUserAuthService:162 -
> auth(ClientSessionImpl[cisco@/10.90.10.193:22])[ssh-connection] send
> SSH_MSG_USERAUTH_REQUEST for 'none'
> 2020-02-25 20:12:15.532Z [pool-171-thread-1] DEBUG
> o.a.s.c.session.ClientSessionImpl:808 -
> writePacket(ClientSessionImpl[cisco@/10.90.10.193:22])[SSH_MSG_USERAUTH_REQUEST]
> Start flagging packets as pending until key exchange is done
> 2020-02-25 20:12:15.533Z [sshd-SshClient[468ca10d]-nio2-thread-5] DEBUG
> o.a.s.c.session.ClientSessionImpl:167 -
> signalAuthFailure(ClientSessionImpl[cisco@/10.90.10.193:22])
> type=IOException, signalled=true: Connection reset by peer
> {code}
>
> Ssh (openssh) successfully connects to the save device
> {code:java}
> fwd-collector@collector:/usr/local/fwd/logs$ ssh -vvv @10.90.10.193
> OpenSSH_7.2p2 Ubuntu-4ubuntu2.7, OpenSSL 1.0.2g 1 Mar 2016
> debug1: Reading configuration data /etc/ssh/ssh_config
> debug1: /etc/ssh/ssh_config line 19: Applying options for *
> debug2: resolving "10.90.10.193" port 22
> debug2: ssh_connect_direct: needpriv 0
> debug1: Connecting to 10.90.10.193 [10.90.10.193] port 22.
> debug1: Connection established.
> debug1: key_load_public: No such file or directory
> debug1: identity file /home/fwd-collector/.ssh/id_rsa type -1
> debug1: key_load_public: No such file or directory
> debug1: identity file /home/fwd-collector/.ssh/id_rsa-cert type -1
> debug1: key_load_public: No such file or directory
> debug1: identity file /home/fwd-collector/.ssh/id_dsa type -1
> debug1: key_load_public: No such file or directory
> debug1: identity file /home/fwd-collector/.ssh/id_dsa-cert type -1
> debug1: key_load_public: No such file or directory
> debug1: identity file /home/fwd-collector/.ssh/id_ecdsa type -1
> debug1: key_load_public: No such file or directory
> debug1: identity file /home/fwd-collector/.ssh/id_ecdsa-cert type -1
> debug1: key_load_public: No such file or directory
> debug1: identity file
