The problem doesn't seem to be on the guest because the throughput on the vnet
remains minimal. The traffic on the IO domain however jumps. The traffic is the
same regardless (subnet traffic, broadcast etc) of whether there is a guest
assigned or not, but the through put just seems to jump as soon as a guest has
the aggregate assigned to it.
I performed a dladm show-aggr and the traffic appears to only utilise 2 of the
4 interfaces for input and the remaining 2 for output (this might be correct):
key:1 ipackets rbytes opackets obytes %ipkts %opkts
Total 4441 1545061 15 3250
nxge0 2477 1173436 0 0 55.8 0.0
nxge1 1961 371311 0 0 44.2 0.0
nxge2 1 124 15 3250 0.0 100.0
nxge3 2 190 0 0 0.0 0.0
When I remove the client, the through put drops....
key:1 ipackets rbytes opackets obytes %ipkts %opkts
Total 23 1680 0 0
nxge0 17 1084 0 0 73.9 -
nxge1 2 216 0 0 8.7 -
nxge2 1 124 0 0 4.3 -
nxge3 3 256 0 0 13.0 -
Cheers
Sean
-----Original Message-----
From: Octave Orgeron [mailto:[email protected]]
Sent: 31 March 2011 16:24
To: Burke Sean; [email protected]
Subject: Re: [ldoms-discuss] link based IPMP and LDOMS
What kind of traffic do you see from the guest through snoop?
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
Octave J. Orgeron
Solaris Virtualization Architect and Consultant
Web: http://unixconsole.blogspot.com
E-Mail: [email protected]
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
----- Original Message ----
From: Burke Sean <[email protected]>
To: "[email protected]" <[email protected]>
Sent: Thu, March 31, 2011 9:12:08 AM
Subject: Re: [ldoms-discuss] link based IPMP and LDOMS
I have a weird situation when I go the dladm route in that I constantly have
network traffic going over the aggregate as soon as I assign the vnet to a guest
as a secondary path...
The setup is a split bus T5240 with the guests primary vnet coming from the
primary IO's 10Gb card.
See below for nicstat capture of aggregate on the secondary IO domain
Time Int rKB/s wKB/s rPk/s wPk/s rAvs wAvs %Util Sat
16:04:59 aggr1 1.02 0.40 12.18 2.79 85.66 146.3 0.00 0.00
ldm add-vnet linkprop=phys-state guest-vnet1 secondary-vsw0 guest
16:05:04 aggr1 204.7 0.49 878.8 4.18 238.6 119.2 0.17 0.00
16:05:09 aggr1 1304.5 0.28 3579.4 2.00 373.2 145.4 1.07 0.00
16:05:14 aggr1 3601.7 0.29 4218.0 2.00 874.4 147.0 2.95 0.00
16:05:19 aggr1 2938.8 0.30 3365.1 1.99 894.3 154.2 2.41 0.00
ldm remove-vnet guest-vnet1 guest
16:05:24 aggr1 1.29 0.40 16.79 3.60 78.63 114.6 0.00 0.00
16:05:29 aggr1 1.00 0.35 12.17 2.39 84.23 150.3 0.00 0.00
bash-3.00# dladm show-aggr -L
key: 1 (0x0001) policy: L2 address: 0:21:28:b0:8c:8e (auto)
LACP mode: active LACP timer: short
device activity timeout aggregatable sync coll dist defaulted expired
nxge0 active short yes yes yes yes no no
nxge1 active short yes yes yes yes no no
nxge2 active short yes yes yes yes no no
nxge3 active short yes yes yes yes no no
Any thoughts?
Cheers
Sean
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Hudes, Dana
Sent: 31 March 2011 16:01
To: '[email protected]'; '[email protected]'
Subject: Re: [ldoms-discuss] link based IPMP and LDOMS
Instead of ipmp do dladm link aggregation and all your traffic uses both links.
In case of failure you have degraded performance for all rather than failure for
some. With solaris 11 and crossbow we will have traffic shaping to throttle
bandwidth per sub interface but I haven't got that running yet
----- Original Message -----
From: [email protected]
<[email protected]>
To: [email protected] <[email protected]>
Sent: Thu Mar 31 08:09:24 2011
Subject: Re: [ldoms-discuss] link based IPMP and LDOMS
> I see no benefit from doing IPMP in each LDOM instead
> of doing IPMP at the platform level and giving the
> resulting virtual interface to the vnet to the LDOM.
OK picture this. you have a domain controller with two interfaces
e1000g0 and nxge0 both are on the same vlan 192.168.1.x
joined together with IPMP. active interface is e1000g0. we have two different
LDOMS that are going
to be using this network LDOMa and LDOMb. each LDOM has 4 zones under it. This
gives us five
vnets on each LDOM ten in total. Since IPMP is active/passive all of your
traffic from ten different vnets
is going down your e1000g0 pipe.
now lets say we do our IPMP down inside the LDOMS/zones this would require
that I create 5 vnets on LDOMa under e1000g0 and 5 vnets on LDOMa under nxge0. 5
vnets on LDOMb under e1000g0 and 5 vnets on LDOMb under nxge0. Now down inside
the LDOMS/zones I alternate between which vnet is the primary. This allows me to
double the network bandwidth down inside the LDOMS/zones than doing all of the
IPMP at the domain controller level.
Next:
My environment has 4 to 6 LDOMS on a given domain controller all running 4
zones inside of them. Each of the LDOMS is tied to a vlan that I do not want or
need on the domain controller. I do not want a interface plumbed up on the
domain controller just to run IPMP on it instead of the LDOMS.
--
This message posted from opensolaris.org
_______________________________________________
ldoms-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/ldoms-discuss
_______________________________________________
ldoms-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/ldoms-discuss
Stryker SA / Registered number: CH-550-0054924-4 / Registered office:
Cité-Centre, Grand rue 90, 1820 Montreux / VAT number: . 291987
Les informations contenues dans ce courriel ainsi que les éventuelles pièces
jointes sont exclusivement destinées à l’usage du ou des destinataires prévus.
Ce courriel peut contenir des informations privilégiées ou confidentielles aux
yeux de la loi ou le produit d’un travail soumis au secret professionnel de
l’avocat ou autrement protégé contre la divulgation. Aucune renonciation à un
quelconque privilège n’est envisagée. Si vous n’êtes pas le destinataire prévu,
toute interception, communication, diffusion ou copie est strictement interdite
et pourra donner lieu à des poursuites pénales ou civiles à votre encontre. Si
vous avez reçu ce courriel par erreur, veuillez en avertir l’expéditeur par
téléphone ou par retour de courriel et confirmer que vous détruirez ou effacerez
de votre système toutes les exemplaires de ce message et des éventuelles pièces
jointes.
The information contained in this email message and any attachments is
exclusively for the use of the intended recipient(s). It may contain legally
privileged, confidential information or work product subject to attorney-client
privilege or otherwise protected from disclosure. No waiver of any privilege is
intended. If you are not the intended recipient, any interception,
dissemination, distribution or copying is strictly prohibited, and may subject
you to criminal and/or civil penalties. If you have received this email message
in error, please notify the sender by telephone or by email reply and confirm
that you will destroy or delete all copies of the message and any attachments
from your system.
_______________________________________________
ldoms-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/ldoms-discuss
Stryker SA / Registered number: CH-550-0054924-4 / Registered office:
Cité-Centre, Grand rue 90, 1820 Montreux / VAT number: . 291987
Les informations contenues dans ce courriel ainsi que les éventuelles pièces
jointes sont exclusivement destinées à l’usage du ou des destinataires prévus.
Ce courriel peut contenir des informations privilégiées ou confidentielles aux
yeux de la loi ou le produit d’un travail soumis au secret professionnel de
l’avocat ou autrement protégé contre la divulgation. Aucune renonciation à un
quelconque privilège n’est envisagée. Si vous n’êtes pas le destinataire prévu,
toute interception, communication, diffusion ou copie est strictement interdite
et pourra donner lieu à des poursuites pénales ou civiles à votre encontre. Si
vous avez reçu ce courriel par erreur, veuillez en avertir l’expéditeur par
téléphone ou par retour de courriel et confirmer que vous détruirez ou
effacerez de votre système toutes les exemplaires de ce message et des
éventuelles pièces jointes.
The information contained in this email message and any attachments is
exclusively for the use of the intended recipient(s). It may contain legally
privileged, confidential information or work product subject to attorney-client
privilege or otherwise protected from disclosure. No waiver of any privilege
is intended. If you are not the intended recipient, any interception,
dissemination, distribution or copying is strictly prohibited, and may subject
you to criminal and/or civil penalties. If you have received this email
message in error, please notify the sender by telephone or by email reply and
confirm that you will destroy or delete all copies of the message and any
attachments from your system.
_______________________________________________
ldoms-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/ldoms-discuss
