Hi folks,
I've got some strange network behaviour that I'm wondering if anyone has come
across...
Setup:
1xT5240 with split bus to provide hardware redundancy etc
bash-3.00# ldm -V
Logical Domain Manager (v 2.0-patch)
Hypervisor control protocol v 1.6
Using Hypervisor MD v 1.3
System PROM:
Hypervisor v. 1.9.2 @(#)Hypervisor 1.9.2 2010/10/29
18:26\015
OpenBoot v. 4.32.2 @(#)OpenBoot 4.32.2 2010/10/29 15:36
Primary has 10Gb connection, dual HBA's and boots of internal disks.
Secondary has quad Gb, dual HBA's and boots of SAN.
3xGuests run through primary HBA's and 10Gb card and the secondary i/o's HBA
and quad Gb are assigned to the guests for failover with IPMP in the guests and
EMC PowerPath managing HBA's on the I/O domains.
For the secondary, I created an aggregate for the quad Gb card.
With all LDOM's "quiet" I see virtually no traffic going through the 10Gb or
the aggregate (I'm using nicstat). As soon as I assign a guest LDOM to have the
aggr as it's secondary vnet, network traffic suddenly jumps up on the aggr in
the secondary io domain.
Time Int rKB/s wKB/s rPk/s wPk/s rAvs wAvs %Util Sat
16:06:29 aggr1 1.17 0.29 13.37 2.00 89.43 150.8 0.00 0.00
16:06:34 aggr1 1.35 0.43 17.76 3.79 77.71 115.4 0.00 0.00
16:06:39 aggr1 1.69 0.27 22.95 1.80 75.20 153.1 0.00 0.00
16:06:44 aggr1 1.41 0.41 17.96 2.79 80.33 148.9 0.00 0.00
16:06:49 aggr1 190.8 0.29 356.6 1.99 547.8 150.8 0.16 0.00
16:06:54 aggr1 2974.4 0.75 3367.7 7.99 904.4 95.50 2.44 0.00
16:06:59 aggr1 1698.1 0.27 2727.9 1.80 637.4 156.7 1.39 0.00
16:07:04 aggr1 3480.8 0.27 3548.9 1.80 1004.4 154.9 2.85 0.00
16:07:09 aggr1 2359.5 0.29 3379.3 2.00 715.0 147.0 1.93 0.00
16:07:14 aggr1 918.4 0.53 2367.9 4.59 397.2 118.0 0.75 0.00
16:07:20 aggr1 1744.5 0.30 1655.4 1.99 1079.1 154.2 1.43 0.00
16:07:25 aggr1 1.12 0.22 13.58 1.40 84.56 162.0 0.00 0.00
16:07:30 aggr1 0.83 0.27 9.38 1.80 90.30 153.1 0.00 0.00
The first 4 lines are with no client assigned to the secondary vnet aggr and
the items in bold are when I add the vnet to the client and the last to entries
are when it is removed
If I plumb the aggregate as a vsw in the secondary, there is network traffic
regardless if a client has a vnet in the secondary domain.
Time Int rKB/s wKB/s rPk/s wPk/s rAvs wAvs %Util Sat
16:16:15 vsw0 3249.1 0.11 5438.5 0.80 611.8 140.0 2.66 0.00
16:16:20 vsw0 846.6 0.00 2665.7 0.00 325.2 0.00 0.69 0.00
16:16:25 vsw0 2814.8 0.00 3743.7 0.00 769.9 0.00 2.31 0.00
16:16:30 vsw0 2336.5 0.00 3250.5 0.00 736.1 0.00 1.91 0.00
Dladm shows:
bash-3.00# dladm show-aggr -L
key: 1 (0x0001) policy: L2 address: 0:21:28:b0:8c:8e (auto)
LACP mode: active LACP timer: short
device activity timeout aggregatable sync coll dist defaulted expired
nxge0 active short yes yes yes yes no no
nxge1 active short yes yes yes yes no no
nxge2 active short yes yes yes yes no no
nxge3 active short yes yes yes yes no no
bash-3.00# dladm show-aggr -s
key:1 ipackets rbytes opackets obytes %ipkts %opkts
Total 204172 78620370 26123 3919733
nxge0 110616 53315144 235 32864 54.2 0.9
nxge1 78835 23351865 234 32760 38.6 0.9
nxge2 7420 971077 23171 3229522 3.6 88.7
nxge3 7301 982284 2483 624587 3.6 9.5
Any thoughts ... not sure about the spread of packets through the aggr.
This may be normal behaviour, but it looks a bit "strange"!
Cheers
Sean
Sean Burke
UNIX Systems Engineer
Stryker S.A.
Cité-Centre, Grand-rue 92
1820 Montreux
Switzerland
tel.: + 41 21 966 13 54
mbl: +41 79 362 85 15
[email protected]
www.stryker.eu
________________________________
Stryker SA / Registered number: CH-550-0054924-4 / Registered office:
Cité-Centre, Grand rue 90, 1820 Montreux / VAT number: . 291987
Les informations contenues dans ce courriel ainsi que les éventuelles pièces
jointes sont exclusivement destinées à l'usage du ou des destinataires prévus.
Ce courriel peut contenir des informations privilégiées ou confidentielles aux
yeux de la loi ou le produit d'un travail soumis au secret professionnel de
l'avocat ou autrement protégé contre la divulgation. Aucune renonciation à un
quelconque privilège n'est envisagée. Si vous n'êtes pas le destinataire prévu,
toute interception, communication, diffusion ou copie est strictement interdite
et pourra donner lieu à des poursuites pénales ou civiles à votre encontre. Si
vous avez reçu ce courriel par erreur, veuillez en avertir l'expéditeur par
téléphone ou par retour de courriel et confirmer que vous détruirez ou
effacerez de votre système toutes les exemplaires de ce message et des
éventuelles pièces jointes.
The information contained in this email message and any attachments is
exclusively for the use of the intended recipient(s). It may contain legally
privileged, confidential information or work product subject to attorney-client
privilege or otherwise protected from disclosure. No waiver of any privilege is
intended. If you are not the intended recipient, any interception,
dissemination, distribution or copying is strictly prohibited, and may subject
you to criminal and/or civil penalties. If you have received this email message
in error, please notify the sender by telephone or by email reply and confirm
that you will destroy or delete all copies of the message and any attachments
from your system.
_______________________________________________
ldoms-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/ldoms-discuss
