Ok, looked through the capture. The BMC violates the IPMI spec by not correctly setting the target address field. (e.g. you can see in wireshark that the protocol dissector blows up on the packet by saying 'Unkown command' and claiming it says 'BROADCAST', this is all because the byte at 0x38 is '00' and it should have been '0x81'). xCAT blows up for the same reason, the BMC is required to correctly address responses and xCAT recently started caring about that. xCAT actually blows up less than the dissector, the dissector actually gets confused about the correct offsets for the rest of the packet.
The good news is the reason for using multiple rqaddr values (why our
routing code had to actually check the value) was removed and thus we can
retire that check as well and tolerate the spec violation.
Can you see if the following change allows it to work again?
--- a/xCAT-server/lib/perl/xCAT/IPMI.pm
+++ b/xCAT-server/lib/perl/xCAT/IPMI.pm
@@ -865,7 +865,7 @@ sub parse_ipmi_payload {
my @payload = @_;
#for now, just trash the headers, this has been validated to death
anyway
#except seqlun, that one is important
- unless ($payload[4] == $self->{seqlun} and $payload[0] == $self->
{rqaddr} and $payload[1]>>2 == $self->{expectednetfn} and $payload[5] ==
$self->{expecte
+ unless ($payload[4] == $self->{seqlun} and $payload[1]>>2 == $self->
{expectednetfn} and $payload[5] == $self->{expectedcmd} ) {
#both sequence number and arqaddr must match, because we are using
rqaddr to extend the sequence number
#print "Successfully didn't get confused by stale response
".$payload[4]." and ".($self->{seqlun}-4)."\n";
#hexdump(@payload);
From: Sten Wolf <[email protected]>
To: xCAT Users Mailing list <[email protected]>,
Date: 01/29/2013 08:54 AM
Subject: Re: [xcat-user] ipmi commands not working if used as "rXXX"
comands, but work as ipmitool-xcat commands
Attached wireshark captures of the "conversation" stream.
tux401ipmi.cap has the issue.
Interestingly, when attaching the node to an older xcat server ( running on
centos 6.2 ) the result is correct. The capture for the same node, from the
older xcat server is attached as tux401ipmi-external.cap
trying perl -e 'use Digest::SHA1; use Digest::HMAC_SHA1;' didn't return
anything.
trying ipmitool -I lanplus -C 2 <etc...> returned correct result, same as
without -C 2
trying perl -e 'use Crypt::Rijndael; use Crypt::CBC;' returned error:
Can't locate Crypt/Rijndael.pm in @INC (@INC
contains: /usr/local/lib64/perl5 /usr/local/share/perl5
/usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl
/usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl
/usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .)
at -e line 1.
BEGIN failed--compilation aborted at -e line 1.
trying /opt/xcat/lib/perl/xCAT/IPMI.pm. After 'Digest::HMAC_HA1->import()'
there is a '1;'. Change that to '0;' :
the result was the same as with '1' - timeout.
should I have done anything else after modifying IPMI.pm?
from comparing the cap files, it seems the new server doesn't send the rmcp
+ open session request, sending the same "get channel" packet over and
over...
any idea what can be done?
On 28/01//2013 16:56, Jarrod B Johnson wrote:
It may be interesting to review a libpcap capture at the time of the
attempt. I can probably more specifically ascertain the issue.
I'll make a few speculative guesses as what could theoretically be
the problem.
-BMC is demanding IPMI 2, but xCAT's ipmi2support is failing. Try:
perl -e 'use Digest::SHA1; use Digest::HMAC_SHA1;'. If that works,
then xCAT should be doing IPMI 2 by default if the BMC advertises it.
I might should make those hard rpm dependencies, since they are
in-distro for almost everyone.
-xCAT is doing IPMI2 and BMC requires IPMI2, but it more specifically
is requiring cipher suite 3 and xCAT's prereqs for that are missing.
To test this, try ipmitool -I lanplus -C 2 <etc...>' and test for
xCAT Cipher suite 3 by doing: perl -e 'use Crypt::Rijndael; use
Crypt::CBC;'
-Service processor can do IPMI 2, but does not support mutual
authentication (protection against impersonating a BMC). ipmitool
works because it doesn't bother to authenticate the BMC at all (which
actually means rcons is hijackable so long as we use ipmitool). The
most secure workaround to try in such a case is to search for
'ipmi2support' in /opt/xcat/lib/perl/xCAT/IPMI.pm. After
'Digest::HMAC_HA1->import()' there is a '1;'. Change that to '0;' to
force IPMI 1.5 operation. *If* they don't support IPMI 2 mutual
authentication, then 1.5 would be more secure. Note that doesn't
mean that any functionality is lost, we still issue IPMI 2 version
commands when support detected even if we use IPMI 1.5 protocol to
convey them.
If you were interested in explicitly testing that last case, you'd
have to disable the check for matching authtype, the return statement
commented as 'not thinking about packets that do not match our
preferred auth type' would be that.
I probably should add a site entry to disable IPMI 2 support. I
could also more specifically relax the mutual authentication check,
but I'd much rather the relatively few manufacturers that can't pass
that fix their firmware since it's a much better security picture.
Sten Wolf ---01/27/2013 08:07:50 PM---Hi, I'm encountering a strange
issue with some SuperMicro machines, using xcat 2.7.6. All nodes are
From: Sten Wolf <[email protected]>
To: [email protected],
Date: 01/27/2013 08:07 PM
Subject: [xcat-user] ipmi commands not working if used as "rXXX"
comands, but work as ipmitool-xcat commands
Hi,
I'm encountering a strange issue with some SuperMicro machines, using
xcat 2.7.6. All nodes are centos 6.3 (both MN and compute nodes).
After configuring nodehm, ipmi tables the
rpower/rinv/rinstall/rvitals commands do not work. They simply time
out.
using the same credentials with ipmitool-xcat returns correct
results.
# ipmitool-xcat -I lanplus -H ipmi05 -U ADMIN -P ADMIN power status
Chassis Power is on
while
# rpower n05 status
n05: Error: timeout
Firewall and selinux both disabled.
lsdef n05 confirms it's bmc is ipmi05.
using xcatdebug -f enable -d enable also confirms the command is
trying to connect to the ipmi address, with the correct username and
password, but getting no reply.
The only possible issue I could see is
'ipmiversion' => '1.5',
I haven't tried ipmitool -I lan yet, but I wonder if this setting is
user configurable in xcat?
BTW - nodeset works correctly, if I power cycle the nodes manually,
or with ipmitool, they install without further issues.
------------------------------------------------------------------------------
Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
MVC, Windows 8 Apps, JavaScript and much more. Keep your skills
current
with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
MVPs and experts. ON SALE this month only -- learn more at:
http://p.sf.net/sfu/learnnow-d2d
_______________________________________________
xCAT-user mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/xcat-user
[attachment "tux401ipmi-external.cap" deleted by Jarrod B
Johnson/Raleigh/IBM] [attachment "tux401ipmi.cap" deleted by Jarrod B
Johnson/Raleigh/IBM]
------------------------------------------------------------------------------
Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
MVPs and experts. ON SALE this month only -- learn more at:
http://p.sf.net/sfu/learnnow-d2d
_______________________________________________
xCAT-user mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/xcat-user
<<inline: graycol.gif>>
------------------------------------------------------------------------------ Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnnow-d2d
_______________________________________________ xCAT-user mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/xcat-user
