BTW, there are kstats in the domain that owns the vsw(usually the
control or a domain) that the vnets are attached to.
Here is are the details about vsw monitoring from the service domain
that provides the vsw:
#kstat -m vsw
...
module: vsw instance: 0
name: vswldc0xb class: net
brdcstrcv 27756BTW, there are kstats in
the domain that owns the vsw(usually the control or a domain) that the
vnets are attached to.
Here is are the details about vsw monitoring from the service domain
that provides the vsw:
#kstat -m vsw
...
module: vsw instance: 0
name: vswldc0xb class: net
brdcstrcv 27756BTW, there are kstats in
the domain that owns the vsw(usually the control or a domain) that the
vnets are attached to.
Here is are the details about vsw monitoring from the service domain
that provides the vsw:
#kstat -m vsw BTW, there are kstats in the domain that owns the
vsw(usually the control or a domain) that the vnets are attached to.
Here is are the details about vsw monitoring from the service domain
that provides the vsw:
#kstat -m vsw
...
module: vsw instance: 0
name: vswldc0xb class: net
brdcstrcv 27756
brdcstxmt 9479274
callbacks 28379983
crtime 118.05359603
dring_data_acks 19051064
dring_data_msgs 19051064
dring_stopped_acks 19051064
ierrors 0
ipackets 10904726
ipackets64 10904726
multircv 0
multixmt 6540175
norcvbuf 0
noxmtbuf 0
obytes 1509910808
obytes64 1509910808
oerrors 0
opackets 19969671
opackets64 19969671
rbytes 2515751743
rbytes64 2515751743
rx_allocb_fail 0
rx_lost_pkts 0
rx_pri_bytes 0
rx_pri_fail 0
rx_pri_packets 0
rx_vio_allocb_fail 0
snaptime 6902047.94257113
tx_no_desc 0
tx_pri_bytes 0
tx_pri_fail 0
tx_pri_packets 0
tx_qfull 0
BTW, there are kstats in the domain that owns the vsw(usually the
control or a domain) that the vnets are attached to.
Here is are the details about vsw monitoring from the service domain
that provides the vsw:
#kstat -m vsw
...
module: vsw instance: 0
name: vswldc0xb class: net
brdcstrcv 27756
brdcstxmt 9479274
callbacks 28379983
crtime 118.05359603
dring_data_acks 19051064
dring_data_msgs 19051064
dring_stopped_acks 19051064
ierrors 0
ipackets 10904726
ipackets64 10904726
multircv 0
multixmt 6540175
norcvbuf 0
noxmtbuf 0
obytes 1509910808
obytes64 1509910808
oerrors 0
opackets 19969671
opackets64 19969671
rbytes 2515751743
rbytes64 2515751743
rx_allocb_fail 0
rx_lost_pkts 0
rx_pri_bytes 0
rx_pri_fail 0
rx_pri_packets 0
rx_vio_allocb_fail 0
snaptime 6902047.94257113
tx_no_desc 0
tx_pri_bytes 0
tx_pri_fail 0
tx_pri_packets 0
tx_qfull 0
...
name: vsw0 - correponds to the vswitch as a device itself
name: vswldc0xb - is the ldomA ..
One issue is that vswldc<num> is not intutive today. Because the num
corresponds to the LDC channel connecting the vsw to the vnet in the
target domain.
ldm list -l -e will tell you what channel on the vswitch is connecting
to which vnet and that canbe used to construct the name of the stat.
#ldm list-bindings -e -p primary
-bash-3.00# ldm list-bindings -p -e primary
VERSION 1.4
DOMAIN|name=primary|state=active|flags=normal,control,vio-service|cons=SP|ncpu=8|mem=4294967296|util=0.5|uptime=6905408
MAC|mac-addr=00:14:4f:aa:c3:46
HOSTID|hostid=0x84aac346
CONTROL|failure-policy=ignore
...
VSW|name=primary-vsw0|mac-addr=00:14:4f:fa:b8:c5|net-dev=nxge0|dev=switch at
0|default-vlan-id=1|pvid=1|vid=|mode=|mtu=1500|linkprop=|id=0
|peer=vnet0 at ldomA|mac-addr=00:14:4f:f9:ab:4d|pvid=1|vid=|mtu=1500|ldc=0xb
|peer=vnet0 at ldomB|mac-addr=00:14:4f:fa:41:2d|pvid=1|vid=|mtu=1500|ldc=0x10
|peer=vnet0 at
secondary|mac-addr=00:14:4f:f9:66:b1|pvid=1|vid=|mtu=1500|ldc=0x15
VSW|name=primary-vsw1|mac-addr=00:14:4f:fa:ec:d0|net-dev=e1000g1|dev=switch at
1|default-vlan-id=1|pvid=1|vid=|mode=|mtu=1500|linkprop=|id=1
...
So from the above, we know that the kstat -m vsw -n vswldc0xb is the
vnet connection to the virtual switch vsw0 for ldomA.
...
name: vsw0 - correponds to the vswitch as a device itself
name: vswldc0xb - is the ldomA ..
One issue is that vswldc<num> is not intutive today. Because the num
corresponds to the LDC channel connecting the vsw to the vnet in the
target domain.
ldm list -l -e will tell you what channel on the vswitch is connecting
to which vnet and that canbe used to construct the name of the stat.
#ldm list-bindings -e -p primary
-bash-3.00# ldm list-bindings -p -e primary
VERSION 1.4
DOMAIN|name=primary|state=active|flags=normal,control,vio-service|cons=SP|ncpu=8|mem=4294967296|util=0.5|uptime=6905408
MAC|mac-addr=00:14:4f:aa:c3:46
HOSTID|hostid=0x84aac346
CONTROL|failure-policy=ignore
...
VSW|name=primary-vsw0|mac-addr=00:14:4f:fa:b8:c5|net-dev=nxge0|dev=switch at
0|default-vlan-id=1|pvid=1|vid=|mode=|mtu=1500|linkprop=|id=0
|peer=vnet0 at ldomA|mac-addr=00:14:4f:f9:ab:4d|pvid=1|vid=|mtu=1500|ldc=0xb
|peer=vnet0 at ldomB|mac-addr=00:14:4f:fa:41:2d|pvid=1|vid=|mtu=1500|ldc=0x10
|peer=vnet0 at
secondary|mac-addr=00:14:4f:f9:66:b1|pvid=1|vid=|mtu=1500|ldc=0x15
VSW|name=primary-vsw1|mac-addr=00:14:4f:fa:ec:d0|net-dev=e1000g1|dev=switch at
1|default-vlan-id=1|pvid=1|vid=|mode=|mtu=1500|linkprop=|id=1
...
So from the above, we know that the kstat -m vsw -n vswldc0xb is the
vnet connection to the virtual switch vsw0 for ldomA.
...
module: vsw instance: 0
name: vswldc0xb class: net
brdcstrcv 27756
brdcstxmt 9479274
callbacks 28379983
crtime 118.05359603
dring_data_acks 19051064
dring_data_msgs 19051064
dring_stopped_acks 19051064
ierrors 0
ipackets 10904726
ipackets64 10904726
multircv 0
multixmt 6540175
norcvbuf 0
noxmtbuf 0
obytes 1509910808
obytes64 1509910808
oerrors 0
opackets 19969671
opackets64 19969671
rbytes 2515751743
rbytes64 2515751743
rx_allocb_fail 0
rx_lost_pkts 0
rx_pri_bytes 0
rx_pri_fail 0
rx_pri_packets 0
rx_vio_allocb_fail 0
snaptime 6902047.94257113
tx_no_desc 0
tx_pri_bytes 0
tx_pri_fail 0
tx_pri_packets 0
tx_qfull 0
BTW, there are kstats in the domain that owns the vsw(usually the
control or a domain) that the vnets are attached to.
Here is are the details about vsw monitoring from the service domain
that provides the vsw:
#kstat -m vsw
...
module: vsw instance: 0
name: vswldc0xb class: net
brdcstrcv 27756
brdcstxmt 9479274
callbacks 28379983
crtime 118.05359603
dring_data_acks 19051064
dring_data_msgs 19051064
dring_stopped_acks 19051064
ierrors 0
ipackets 10904726
ipackets64 10904726
multircv 0
multixmt 6540175
norcvbuf 0
noxmtbuf 0
obytes 1509910808
obytes64 1509910808
oerrors 0
opackets 19969671
opackets64 19969671
rbytes 2515751743
rbytes64 2515751743
rx_allocb_fail 0
rx_lost_pkts 0
rx_pri_bytes 0
rx_pri_fail 0
rx_pri_packets 0
rx_vio_allocb_fail 0
snaptime 6902047.94257113
tx_no_desc 0
tx_pri_bytes 0
tx_pri_fail 0
tx_pri_packets 0
tx_qfull 0
...
name: vsw0 - correponds to the vswitch as a device itself
name: vswldc0xb - is the ldomA ..
One issue is that vswldc<num> is not intutive today. Because the num
corresponds to the LDC channel connecting the vsw to the vnet in the
target domain.
ldm list -l -e will tell you what channel on the vswitch is connecting
to which vnet and that canbe used to construct the name of the stat.
#ldm list-bindings -e -p primary
-bash-3.00# ldm list-bindings -p -e primary
VERSION 1.4
DOMAIN|name=primary|state=active|flags=normal,control,vio-service|cons=SP|ncpu=8|mem=4294967296|util=0.5|uptime=6905408
MAC|mac-addr=00:14:4f:aa:c3:46
HOSTID|hostid=0x84aac346
CONTROL|failure-policy=ignore
...
VSW|name=primary-vsw0|mac-addr=00:14:4f:fa:b8:c5|net-dev=nxge0|dev=switch at
0|default-vlan-id=1|pvid=1|vid=|mode=|mtu=1500|linkprop=|id=0
|peer=vnet0 at ldomA|mac-addr=00:14:4f:f9:ab:4d|pvid=1|vid=|mtu=1500|ldc=0xb
|peer=vnet0 at ldomB|mac-addr=00:14:4f:fa:41:2d|pvid=1|vid=|mtu=1500|ldc=0x10
|peer=vnet0 at
secondary|mac-addr=00:14:4f:f9:66:b1|pvid=1|vid=|mtu=1500|ldc=0x15
VSW|name=primary-vsw1|mac-addr=00:14:4f:fa:ec:d0|net-dev=e1000g1|dev=switch at
1|default-vlan-id=1|pvid=1|vid=|mode=|mtu=1500|linkprop=|id=1
...
So from the above, we know that the kstat -m vsw -n vswldc0xb is the
vnet connection to the virtual switch vsw0 for ldomA.
...
name: vsw0 - correponds to the vswitch as a device itself
name: vswldc0xb - is the ldomA ..
One issue is that vswldc<num> is not intutive today. Because the num
corresponds to the LDC channel connecting the vsw to the vnet in the
target domain.
ldm list -l -e will tell you what channel on the vswitch is connecting
to which vnet and that canbe used to construct the name of the stat.
#ldm list-bindings -e -p primary
-bash-3.00# ldm list-bindings -p -e primary
VERSION 1.4
DOMAIN|name=primary|state=active|flags=normal,control,vio-service|cons=SP|ncpu=8|mem=4294967296|util=0.5|uptime=6905408
MAC|mac-addr=00:14:4f:aa:c3:46
HOSTID|hostid=0x84aac346
CONTROL|failure-policy=ignore
...
VSW|name=primary-vsw0|mac-addr=00:14:4f:fa:b8:c5|net-dev=nxge0|dev=switch at
0|default-vlan-id=1|pvid=1|vid=|mode=|mtu=1500|linkprop=|id=0
|peer=vnet0 at ldomA|mac-addr=00:14:4f:f9:ab:4d|pvid=1|vid=|mtu=1500|ldc=0xb
|peer=vnet0 at ldomB|mac-addr=00:14:4f:fa:41:2d|pvid=1|vid=|mtu=1500|ldc=0x10
|peer=vnet0 at
secondary|mac-addr=00:14:4f:f9:66:b1|pvid=1|vid=|mtu=1500|ldc=0x15
VSW|name=primary-vsw1|mac-addr=00:14:4f:fa:ec:d0|net-dev=e1000g1|dev=switch at
1|default-vlan-id=1|pvid=1|vid=|mode=|mtu=1500|linkprop=|id=1
...
So from the above, we know that the kstat -m vsw -n vswldc0xb is the
vnet connection to the virtual switch vsw0 for ldomA.
brdcstxmt 9479274
callbacks 28379983
crtime 118.05359603
dring_data_acks 19051064
dring_data_msgs 19051064
dring_stopped_acks 19051064
ierrors 0
ipackets 10904726
ipackets64 10904726
multircv 0
multixmt 6540175
norcvbuf 0
noxmtbuf 0
obytes 1509910808
obytes64 1509910808
oerrors 0
opackets 19969671
opackets64 19969671
rbytes 2515751743
rbytes64 2515751743
rx_allocb_fail 0
rx_lost_pkts 0
rx_pri_bytes 0
rx_pri_fail 0
rx_pri_packets 0
rx_vio_allocb_fail 0
snaptime 6902047.94257113
tx_no_desc 0
tx_pri_bytes 0
tx_pri_fail 0
tx_pri_packets 0
tx_qfull 0
...
name: vsw0 - correponds to the vswitch as a device itself
name: vswldc0xb - is the ldomA ..
One issue is that vswldc<num> is not intutive today. Because the num
corresponds to the LDC channel connecting the vsw to the vnet in the
target domain.
ldm list -l -e will tell you what channel on the vswitch is connecting
to which vnet and that canbe used to construct the name of the stat.
#ldm list-bindings -e -p primary
-bash-3.00# ldm list-bindings -p -e primary
VERSION 1.4
DOMAIN|name=primary|state=active|flags=normal,control,vio-service|cons=SP|ncpu=8|mem=4294967296|util=0.5|uptime=6905408
MAC|mac-addr=00:14:4f:aa:c3:46
HOSTID|hostid=0x84aac346
CONTROL|failure-policy=ignore
...
VSW|name=primary-vsw0|mac-addr=00:14:4f:fa:b8:c5|net-dev=nxge0|dev=switch at
0|default-vlan-id=1|pvid=1|vid=|mode=|mtu=1500|linkprop=|id=0
|peer=vnet0 at ldomA|mac-addr=00:14:4f:f9:ab:4d|pvid=1|vid=|mtu=1500|ldc=0xb
|peer=vnet0 at ldomB|mac-addr=00:14:4f:fa:41:2d|pvid=1|vid=|mtu=1500|ldc=0x10
|peer=vnet0 at
secondary|mac-addr=00:14:4f:f9:66:b1|pvid=1|vid=|mtu=1500|ldc=0x15
VSW|name=primary-vsw1|mac-addr=00:14:4f:fa:ec:d0|net-dev=e1000g1|dev=switch at
1|default-vlan-id=1|pvid=1|vid=|mode=|mtu=1500|linkprop=|id=1
...
So from the above, we know that the kstat -m vsw -n vswldc0xb is the
vnet connection to the virtual switch vsw0 for ldomA.
brdcstxmt 9479274
callbacks 28379983
crtime 118.05359603
dring_data_acks 19051064
dring_data_msgs 19051064
dring_stopped_acks 19051064
ierrors 0
ipackets 10904726
ipackets64 10904726
multircv 0
multixmt 6540175
norcvbuf 0
noxmtbuf 0
obytes 1509910808
obytes64 1509910808
oerrors 0
opackets 19969671
opackets64 19969671
rbytes 2515751743
rbytes64 2515751743
rx_allocb_fail 0
rx_lost_pkts 0
rx_pri_bytes 0
rx_pri_fail 0
rx_pri_packets 0
rx_vio_allocb_fail 0
snaptime 6902047.94257113
tx_no_desc 0
tx_pri_bytes 0
tx_pri_fail 0
tx_pri_packets 0
tx_qfull 0
...
name: vsw0 - correponds to the vswitch as a device itself
name: vswldc0xb - is the ldomA ..
One issue is that vswldc<num> is not intutive today. Because the num
corresponds to the LDC channel connecting the vsw to the vnet in the
target domain.
ldm list -l -e will tell you what channel on the vswitch is connecting
to which vnet and that canbe used to construct the name of the stat.
#ldm list-bindings -e -p primary
-bash-3.00# ldm list-bindings -p -e primary
VERSION 1.4
DOMAIN|name=primary|state=active|flags=normal,control,vio-service|cons=SP|ncpu=8|mem=4294967296|util=0.5|uptime=6905408
MAC|mac-addr=00:14:4f:aa:c3:46
HOSTID|hostid=0x84aac346
CONTROL|failure-policy=ignore
...
VSW|name=primary-vsw0|mac-addr=00:14:4f:fa:b8:c5|net-dev=nxge0|dev=switch at
0|default-vlan-id=1|pvid=1|vid=|mode=|mtu=1500|linkprop=|id=0
|peer=vnet0 at ldomA|mac-addr=00:14:4f:f9:ab:4d|pvid=1|vid=|mtu=1500|ldc=0xb
|peer=vnet0 at ldomB|mac-addr=00:14:4f:fa:41:2d|pvid=1|vid=|mtu=1500|ldc=0x10
|peer=vnet0 at
secondary|mac-addr=00:14:4f:f9:66:b1|pvid=1|vid=|mtu=1500|ldc=0x15
VSW|name=primary-vsw1|mac-addr=00:14:4f:fa:ec:d0|net-dev=e1000g1|dev=switch at
1|default-vlan-id=1|pvid=1|vid=|mode=|mtu=1500|linkprop=|id=1
...
So from the above, we know that the kstat -m vsw -n vswldc0xb is the
vnet connection to the virtual switch vsw0 for ldomA.
Stefan Hinker - Systems Practice - Sun Microsystems Germany wrote:
> Hi Jorge,
>
> I'll give it a try ;-) Please see inline.
>
> Jorge Chiang - Oracle wrote:
>
>> 1) How can I get an overall view of the performance of a T5240 when
>> using LDoms? Is there a tool to gather performance data on the control
>> and guest LDoms and give me a "big picture" view?
>>
>
> Performance is always an attribute of the application, not the platform. If
> you are looking for platform utilization, you might try
>
> "ldm list"
>
> on the control domain to get a big picture of the CPU utilization of all
> domains.
>
>
>> 2) When debugging performance issues in an LDom, is there a way to trace
>> through from the application in the guest LDom to the external interface
>> (NIC or FC HBA) to isolate one guest LDom's traffic from the traffic
>> generated by other LDoms?
>>
>
> The virtualization components (vSwitch, vDisk Server, etc.) are transparent
> for the guests. Their service endpoints (vnet, vdisk device drivers) in the
> guests are the furthest you can trace using Solaris from within a guest, at
> least afaik. In the IO domains or outside the chassis, you can of course
> monitor the MAC-addresses of the guests, at least wrt network traffic.
> For normal debugging, you can treat a vSwitch just like a regular ethernet
> component. For disk IO, you'll likely need to monitor service-times both in
> the guest and in the IO-domain, correlating what you know about the disk
> backend. No easy wins to be expected.
>
> I'd hope for someone from engineering on this list to elaborate a bit more
> on this point.
>
>
>> 3) What are the key performance issues that I should be looking for in
>> the Control and IO Domains? I suspect that all the work is being done
>> at the Kernel level with the vdsdev/vdisk and vsw/vnet modules and I
>> have no idea how to measure/monitor those.
>>
>
> Yes and no. Make sure you have enough CPU (use mpstat, look for at least
> some idle time). If using ZFS in the IO domains, make sure you have
> sufficient RAM for the ARC. However, some of the work is being done in the
> hypervisor, which borrows CPU cycles from these domains. This does *not*
> show up in Solaris.
>
> hth
> stefan
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> ldoms-discuss mailing list
> ldoms-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/ldoms-discuss
>
--
-----------------------------------------------------------------------
______ Joseph Balenzano
/_____/\ ISV Engineering
/____ \\ \ Sun Microsystems Inc.
/_____\ \\ / joseph.balenzano at sun.com
/_____/ \/ / / Phone/Fax 203.653.4186
/_____/ / \//\
\_____\//\ / /
\_____/ / /\ /
\_____/ \\ \
\_____\ \\
\_____\/
----------------------------------------------------------------------
