Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.

2010-05-12 Thread Patrick Mahan 

Check to make sure the links are all full-duplex.  We started seeing
bad performance with the em(4) driver on our HP Proliant  360DL G5's
using 1000Mbits.  It turned out that switch was setting it's port to
half-duplex and the emX interface was following suit.

HTH,

Patrick

joe wrote:

On 05/08/2010 01:31 PM, Jack Vogel wrote:

Looks like something to do with system C, you might isolate it, and try
a back
to back connection with its NICs, change cables, look at BIOS settings,
change
the slot the nic is in... All just off the top of my head.

Jack


On Sat, May 8, 2010 at 9:41 AM, joe mailto:j...@hostedcontent.com>> wrote:

On 05/08/2010 11:17 AM, Ian FREISLICH wrote:

joe wrote:

On 05/08/2010 06:55 AM, Ian FREISLICH wrote:

joe wrote:

   I have just tried your suggeston and it has
no effect for me ;(


Do you have another brand of NIC that you can try?  At
least that
will isolate whether it's igb(4) or something else.


I will grab a new nic today and try...my options are limited
though.
Here are the nics i can get my hands on

TP-LINK TL-TG3468, 10/100/1000Mbps PCIe Adapter (supported
by fbsd?)


Based on the RTL8168B chip.  Should be supported by the re(4)
driver.

Intel (EXPI9301CT) Gigabit CT Desktop Adapter (yet another
intel nic)


i82574L chip.  Should be supported by the em(4) driver.  I 
have had

good performance in the past with this driver and less than
satisfactory performance with the igb(4) driver.

That may not be your problem though.  Before you go out and buy,
have a look at the amount of interrupt time your slow machine 
spends
in 'top' or 'systat -vm'.  systat will also show the interrupt 
rate

for each driver, perhaps it's not doing interrupt moderation
properly.
This will manifest as more than about a 1000 per second.  
There are

loader tunables for the driver to increase the number of transfer
descriptors and to tune interrupt moderation.

You could try running trafshow (port) on the interface while
performing the transfer.  Perhaps promiscuous mode will turn off
some hardware feature that will improve things.  It may however
break hardware vlanning as it does on my 82575GB 4 port igb card.

Ian

--
Ian Freislich


I bought those two cards anyways, im in a rush to figure out this
problem. That being said i am still encountering the exact same
problem regardless on which network card i am running. I am at a
complete loss. I am about to try a raid card to see if the problem
might lay within the onboard sata ports. I did pull the server and
brought it home so that i can test more things quicker.

I am going to try using a raid card instead of the onboard sata
ports and see if i still encounter the same problem. I would love
any suggestions you may have on where to go from here to figure out
where the problem might be.

joe

___
freebsd-current@freebsd.org 
mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to
"freebsd-current-unsubscr...@freebsd.org
"




I think it might have something to so with the nics / switch, and their 
features. I brought the box home, plugged into my gb switch, and i am 
able to FTP data to the server at around 35MB/sec.


I dont know what would cause this other than some sort of issue with the 
the 3 different types of nics and the switch i am using.


Any suggestions?
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"



___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.

2010-05-09 Thread Jack Vogel 
On Sun, May 9, 2010 at 2:54 AM, joe  wrote:

> On 05/08/2010 02:21 PM, Jack Vogel wrote:
>
>> The cable, its a simple thing but make SURE you try that, a slightly
>> damaged one can do weird things and its quick to check, don't overlook
>> it.
>>
>> Jack
>>
>>
>> On Sat, May 8, 2010 at 10:22 AM, joe > > wrote:
>>
>>On 05/08/2010 01:53 PM, Jack Vogel wrote:
>>
>>I still am not clear on this system, how many ports are on it,
>>and its
>>an 82576?
>>Sounds to me like you've proven its not on the box if you can do
>>fine
>>when its
>>on its own. So change ports in the switch, as I said, change
>>cables, must be
>>something in that environment.
>>
>>Jack
>>
>>
>>On Sat, May 8, 2010 at 10:04 AM, joe >
>>>>
>>wrote:
>>
>>On 05/08/2010 01:31 PM, Jack Vogel wrote:
>>
>>Looks like something to do with system C, you might
>>isolate it,
>>and try
>>a back
>>to back connection with its NICs, change cables, look at
>>BIOS
>>settings,
>>change
>>the slot the nic is in... All just off the top of my head.
>>
>>Jack
>>
>>
>>On Sat, May 8, 2010 at 9:41 AM, joe
>>mailto:j...@hostedcontent.com>
>>>
>>
>>>
>>wrote:
>>
>>On 05/08/2010 11:17 AM, Ian FREISLICH wrote:
>>
>>joe wrote:
>>
>>On 05/08/2010 06:55 AM, Ian FREISLICH wrote:
>>
>>joe wrote:
>>
>>   I have just tried your
>>suggeston and
>>it has
>>no effect for me ;(
>>
>>
>>Do you have another brand of NIC that
>>you can
>>try?  At
>>least that
>>will isolate whether it's igb(4) or
>>something else.
>>
>>
>>I will grab a new nic today and try...my
>>options are
>>limited
>>though.
>>Here are the nics i can get my hands on
>>
>>TP-LINK TL-TG3468, 10/100/1000Mbps PCIe Adapter
>>(supported
>>by fbsd?)
>>
>>
>>Based on the RTL8168B chip.  Should be supported
>>by the
>>re(4)
>>driver.
>>
>>Intel (EXPI9301CT) Gigabit CT Desktop
>>Adapter (yet
>>another
>>intel nic)
>>
>>
>>i82574L chip.  Should be supported by the em(4)
>>driver.
>>  I have had
>>good performance in the past with this driver
>>and less than
>>satisfactory performance with the igb(4) driver.
>>
>>That may not be your problem though.  Before you
>>go out
>>and buy,
>>have a look at the amount of interrupt time your
>>slow
>>machine spends
>>in 'top' or 'systat -vm'.  systat will also show
>> the
>>interrupt rate
>>for each driver, perhaps it's not doing
>>interrupt moderation
>>properly.
>>This will manifest as more than about a 1000 per
>>second.
>>  There are
>>loader tunables for the driver to increase the
>>number of
>>transfer
>>descriptors and to tune interrupt moderation.
>>
>>You could try running trafshow (port) on the
>>interface while
>>performing the transfer.  Perhaps promiscuous
>>mode will
>>turn off
>>some hardware feature that will improve things.
>>  It may
>>however
>>break hardware vlanning as it does on my 82575GB
>>4 port
>>igb card.
>>
>>Ian
>>
>>--
>>Ian Freislich
>>
>>
>>I bought those two cards anyways, im in a rush to
>>figure out
>>this
>>problem. That being said i am still encounte

Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.

2010-05-09 Thread joe 

On 05/08/2010 02:21 PM, Jack Vogel wrote:

The cable, its a simple thing but make SURE you try that, a slightly
damaged one can do weird things and its quick to check, don't overlook
it.

Jack


On Sat, May 8, 2010 at 10:22 AM, joe mailto:j...@hostedcontent.com>> wrote:

On 05/08/2010 01:53 PM, Jack Vogel wrote:

I still am not clear on this system, how many ports are on it,
and its
an 82576?
Sounds to me like you've proven its not on the box if you can do
fine
when its
on its own. So change ports in the switch, as I said, change
cables, must be
something in that environment.

Jack


On Sat, May 8, 2010 at 10:04 AM, joe mailto:j...@hostedcontent.com>
>>
wrote:

On 05/08/2010 01:31 PM, Jack Vogel wrote:

Looks like something to do with system C, you might
isolate it,
and try
a back
to back connection with its NICs, change cables, look at
BIOS
settings,
change
the slot the nic is in... All just off the top of my head.

Jack


On Sat, May 8, 2010 at 9:41 AM, joe
mailto:j...@hostedcontent.com>
>

Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.

2010-05-08 Thread joe 

On 05/08/2010 02:21 PM, Jack Vogel wrote:

The cable, its a simple thing but make SURE you try that, a slightly
damaged one can do weird things and its quick to check, don't overlook
it.

Jack


On Sat, May 8, 2010 at 10:22 AM, joe mailto:j...@hostedcontent.com>> wrote:

On 05/08/2010 01:53 PM, Jack Vogel wrote:

I still am not clear on this system, how many ports are on it,
and its
an 82576?
Sounds to me like you've proven its not on the box if you can do
fine
when its
on its own. So change ports in the switch, as I said, change
cables, must be
something in that environment.

Jack


On Sat, May 8, 2010 at 10:04 AM, joe mailto:j...@hostedcontent.com>
>>
wrote:

On 05/08/2010 01:31 PM, Jack Vogel wrote:

Looks like something to do with system C, you might
isolate it,
and try
a back
to back connection with its NICs, change cables, look at
BIOS
settings,
change
the slot the nic is in... All just off the top of my head.

Jack


On Sat, May 8, 2010 at 9:41 AM, joe
mailto:j...@hostedcontent.com>
>

Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.

2010-05-08 Thread Jack Vogel 
The cable, its a simple thing but make SURE you try that, a slightly
damaged one can do weird things and its quick to check, don't overlook
it.

Jack


On Sat, May 8, 2010 at 10:22 AM, joe  wrote:

> On 05/08/2010 01:53 PM, Jack Vogel wrote:
>
>> I still am not clear on this system, how many ports are on it, and its
>> an 82576?
>> Sounds to me like you've proven its not on the box if you can do fine
>> when its
>> on its own. So change ports in the switch, as I said, change cables, must
>> be
>> something in that environment.
>>
>> Jack
>>
>>
>> On Sat, May 8, 2010 at 10:04 AM, joe > > wrote:
>>
>>On 05/08/2010 01:31 PM, Jack Vogel wrote:
>>
>>Looks like something to do with system C, you might isolate it,
>>and try
>>a back
>>to back connection with its NICs, change cables, look at BIOS
>>settings,
>>change
>>the slot the nic is in... All just off the top of my head.
>>
>>Jack
>>
>>
>>On Sat, May 8, 2010 at 9:41 AM, joe >
>>>>
>>
>>wrote:
>>
>>On 05/08/2010 11:17 AM, Ian FREISLICH wrote:
>>
>>joe wrote:
>>
>>On 05/08/2010 06:55 AM, Ian FREISLICH wrote:
>>
>>joe wrote:
>>
>>   I have just tried your suggeston and
>>it has
>>no effect for me ;(
>>
>>
>>Do you have another brand of NIC that you can
>>try?  At
>>least that
>>will isolate whether it's igb(4) or something else.
>>
>>
>>I will grab a new nic today and try...my options are
>>limited
>>though.
>>Here are the nics i can get my hands on
>>
>>TP-LINK TL-TG3468, 10/100/1000Mbps PCIe Adapter
>>(supported
>>by fbsd?)
>>
>>
>>Based on the RTL8168B chip.  Should be supported by the
>>re(4)
>>driver.
>>
>>Intel (EXPI9301CT) Gigabit CT Desktop Adapter (yet
>>another
>>intel nic)
>>
>>
>>i82574L chip.  Should be supported by the em(4) driver.
>>  I have had
>>good performance in the past with this driver and less than
>>satisfactory performance with the igb(4) driver.
>>
>>That may not be your problem though.  Before you go out
>>and buy,
>>have a look at the amount of interrupt time your slow
>>machine spends
>>in 'top' or 'systat -vm'.  systat will also show the
>>interrupt rate
>>for each driver, perhaps it's not doing interrupt
>> moderation
>>properly.
>>This will manifest as more than about a 1000 per second.
>>  There are
>>loader tunables for the driver to increase the number of
>>transfer
>>descriptors and to tune interrupt moderation.
>>
>>You could try running trafshow (port) on the interface
>> while
>>performing the transfer.  Perhaps promiscuous mode will
>>turn off
>>some hardware feature that will improve things.  It may
>>however
>>break hardware vlanning as it does on my 82575GB 4 port
>>igb card.
>>
>>Ian
>>
>>--
>>Ian Freislich
>>
>>
>>I bought those two cards anyways, im in a rush to figure out
>>this
>>problem. That being said i am still encountering the exact same
>>problem regardless on which network card i am running. I am at
>> a
>>complete loss. I am about to try a raid card to see if the
>>problem
>>might lay within the onboard sata ports. I did pull the
>>server and
>>brought it home so that i can test more things quicker.
>>
>>I am going to try using a raid card instead of the onboard sata
>>ports and see if i still encounter the same problem. I would
>>love
>>any suggestions you may have on where to go from here to
>>figure out
>>where the problem might be.
>>
>>joe
>>
>>___
>>freebsd-current@freebsd.org 
>>>>
>>
>>mailing list
>>http://lists.freebsd.org/mailman/listinfo/freebsd-current
>>To unsubscribe, send any mail to
>>"freebsd-current-unsubscr...@freebsd.org
>>
>>

Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.

2010-05-08 Thread joe 

On 05/08/2010 01:53 PM, Jack Vogel wrote:

I still am not clear on this system, how many ports are on it, and its
an 82576?
Sounds to me like you've proven its not on the box if you can do fine
when its
on its own. So change ports in the switch, as I said, change cables, must be
something in that environment.

Jack


On Sat, May 8, 2010 at 10:04 AM, joe mailto:j...@hostedcontent.com>> wrote:

On 05/08/2010 01:31 PM, Jack Vogel wrote:

Looks like something to do with system C, you might isolate it,
and try
a back
to back connection with its NICs, change cables, look at BIOS
settings,
change
the slot the nic is in... All just off the top of my head.

Jack


On Sat, May 8, 2010 at 9:41 AM, joe mailto:j...@hostedcontent.com>
>>
wrote:

On 05/08/2010 11:17 AM, Ian FREISLICH wrote:

joe wrote:

On 05/08/2010 06:55 AM, Ian FREISLICH wrote:

joe wrote:

   I have just tried your suggeston and
it has
no effect for me ;(


Do you have another brand of NIC that you can
try?  At
least that
will isolate whether it's igb(4) or something else.


I will grab a new nic today and try...my options are
limited
though.
Here are the nics i can get my hands on

TP-LINK TL-TG3468, 10/100/1000Mbps PCIe Adapter
(supported
by fbsd?)


Based on the RTL8168B chip.  Should be supported by the
re(4)
driver.

Intel (EXPI9301CT) Gigabit CT Desktop Adapter (yet
another
intel nic)


i82574L chip.  Should be supported by the em(4) driver.
  I have had
good performance in the past with this driver and less than
satisfactory performance with the igb(4) driver.

That may not be your problem though.  Before you go out
and buy,
have a look at the amount of interrupt time your slow
machine spends
in 'top' or 'systat -vm'.  systat will also show the
interrupt rate
for each driver, perhaps it's not doing interrupt moderation
properly.
This will manifest as more than about a 1000 per second.
  There are
loader tunables for the driver to increase the number of
transfer
descriptors and to tune interrupt moderation.

You could try running trafshow (port) on the interface while
performing the transfer.  Perhaps promiscuous mode will
turn off
some hardware feature that will improve things.  It may
however
break hardware vlanning as it does on my 82575GB 4 port
igb card.

Ian

--
Ian Freislich


I bought those two cards anyways, im in a rush to figure out
this
problem. That being said i am still encountering the exact same
problem regardless on which network card i am running. I am at a
complete loss. I am about to try a raid card to see if the
problem
might lay within the onboard sata ports. I did pull the
server and
brought it home so that i can test more things quicker.

I am going to try using a raid card instead of the onboard sata
ports and see if i still encounter the same problem. I would
love
any suggestions you may have on where to go from here to
figure out
where the problem might be.

joe

___
freebsd-current@freebsd.org 
>

mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to
"freebsd-current-unsubscr...@freebsd.org

>"



I think it might have something to so with the nics / switch, and
their features. I brought the box home, plugged into my gb switch,
and i am able to FTP data to the server at around 35MB/sec.

I dont know what would cause this other than some sort of issue with
the the 3 different types of nics and the switch i am using.

Any suggestions?




There are two embedded intel 8

Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.

2010-05-08 Thread Jack Vogel 
I still am not clear on this system, how many ports are on it, and its an
82576?
Sounds to me like you've proven its not on the box if you can do fine when
its
on its own. So change ports in the switch, as I said, change cables, must be
something in that environment.

Jack


On Sat, May 8, 2010 at 10:04 AM, joe  wrote:

> On 05/08/2010 01:31 PM, Jack Vogel wrote:
>
>> Looks like something to do with system C, you might isolate it, and try
>> a back
>> to back connection with its NICs, change cables, look at BIOS settings,
>> change
>> the slot the nic is in... All just off the top of my head.
>>
>> Jack
>>
>>
>> On Sat, May 8, 2010 at 9:41 AM, joe > > wrote:
>>
>>On 05/08/2010 11:17 AM, Ian FREISLICH wrote:
>>
>>joe wrote:
>>
>>On 05/08/2010 06:55 AM, Ian FREISLICH wrote:
>>
>>joe wrote:
>>
>>   I have just tried your suggeston and it has
>>no effect for me ;(
>>
>>
>>Do you have another brand of NIC that you can try?  At
>>least that
>>will isolate whether it's igb(4) or something else.
>>
>>
>>I will grab a new nic today and try...my options are limited
>>though.
>>Here are the nics i can get my hands on
>>
>>TP-LINK TL-TG3468, 10/100/1000Mbps PCIe Adapter (supported
>>by fbsd?)
>>
>>
>>Based on the RTL8168B chip.  Should be supported by the re(4)
>>driver.
>>
>>Intel (EXPI9301CT) Gigabit CT Desktop Adapter (yet another
>>intel nic)
>>
>>
>>i82574L chip.  Should be supported by the em(4) driver.  I have had
>>good performance in the past with this driver and less than
>>satisfactory performance with the igb(4) driver.
>>
>>That may not be your problem though.  Before you go out and buy,
>>have a look at the amount of interrupt time your slow machine
>> spends
>>in 'top' or 'systat -vm'.  systat will also show the interrupt rate
>>for each driver, perhaps it's not doing interrupt moderation
>>properly.
>>This will manifest as more than about a 1000 per second.  There are
>>loader tunables for the driver to increase the number of transfer
>>descriptors and to tune interrupt moderation.
>>
>>You could try running trafshow (port) on the interface while
>>performing the transfer.  Perhaps promiscuous mode will turn off
>>some hardware feature that will improve things.  It may however
>>break hardware vlanning as it does on my 82575GB 4 port igb card.
>>
>>Ian
>>
>>--
>>Ian Freislich
>>
>>
>>I bought those two cards anyways, im in a rush to figure out this
>>problem. That being said i am still encountering the exact same
>>problem regardless on which network card i am running. I am at a
>>complete loss. I am about to try a raid card to see if the problem
>>might lay within the onboard sata ports. I did pull the server and
>>brought it home so that i can test more things quicker.
>>
>>I am going to try using a raid card instead of the onboard sata
>>ports and see if i still encounter the same problem. I would love
>>any suggestions you may have on where to go from here to figure out
>>where the problem might be.
>>
>>joe
>>
>>___
>>freebsd-current@freebsd.org 
>>
>>mailing list
>>http://lists.freebsd.org/mailman/listinfo/freebsd-current
>>To unsubscribe, send any mail to
>>"freebsd-current-unsubscr...@freebsd.org
>>"
>>
>>
>>
> I think it might have something to so with the nics / switch, and their
> features. I brought the box home, plugged into my gb switch, and i am able
> to FTP data to the server at around 35MB/sec.
>
> I dont know what would cause this other than some sort of issue with the
> the 3 different types of nics and the switch i am using.
>
> Any suggestions?
>
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.

2010-05-08 Thread joe 

On 05/08/2010 01:31 PM, Jack Vogel wrote:

Looks like something to do with system C, you might isolate it, and try
a back
to back connection with its NICs, change cables, look at BIOS settings,
change
the slot the nic is in... All just off the top of my head.

Jack


On Sat, May 8, 2010 at 9:41 AM, joe mailto:j...@hostedcontent.com>> wrote:

On 05/08/2010 11:17 AM, Ian FREISLICH wrote:

joe wrote:

On 05/08/2010 06:55 AM, Ian FREISLICH wrote:

joe wrote:

   I have just tried your suggeston and it has
no effect for me ;(


Do you have another brand of NIC that you can try?  At
least that
will isolate whether it's igb(4) or something else.


I will grab a new nic today and try...my options are limited
though.
Here are the nics i can get my hands on

TP-LINK TL-TG3468, 10/100/1000Mbps PCIe Adapter (supported
by fbsd?)


Based on the RTL8168B chip.  Should be supported by the re(4)
driver.

Intel (EXPI9301CT) Gigabit CT Desktop Adapter (yet another
intel nic)


i82574L chip.  Should be supported by the em(4) driver.  I have had
good performance in the past with this driver and less than
satisfactory performance with the igb(4) driver.

That may not be your problem though.  Before you go out and buy,
have a look at the amount of interrupt time your slow machine spends
in 'top' or 'systat -vm'.  systat will also show the interrupt rate
for each driver, perhaps it's not doing interrupt moderation
properly.
This will manifest as more than about a 1000 per second.  There are
loader tunables for the driver to increase the number of transfer
descriptors and to tune interrupt moderation.

You could try running trafshow (port) on the interface while
performing the transfer.  Perhaps promiscuous mode will turn off
some hardware feature that will improve things.  It may however
break hardware vlanning as it does on my 82575GB 4 port igb card.

Ian

--
Ian Freislich


I bought those two cards anyways, im in a rush to figure out this
problem. That being said i am still encountering the exact same
problem regardless on which network card i am running. I am at a
complete loss. I am about to try a raid card to see if the problem
might lay within the onboard sata ports. I did pull the server and
brought it home so that i can test more things quicker.

I am going to try using a raid card instead of the onboard sata
ports and see if i still encounter the same problem. I would love
any suggestions you may have on where to go from here to figure out
where the problem might be.

joe

___
freebsd-current@freebsd.org 
mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to
"freebsd-current-unsubscr...@freebsd.org
"




I think it might have something to so with the nics / switch, and their 
features. I brought the box home, plugged into my gb switch, and i am 
able to FTP data to the server at around 35MB/sec.


I dont know what would cause this other than some sort of issue with the 
the 3 different types of nics and the switch i am using.


Any suggestions?
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.

2010-05-08 Thread Jack Vogel 
Looks like something to do with system C, you might isolate it, and try a
back
to back connection with its NICs, change cables, look at BIOS settings,
change
the slot the nic is in... All just off the top of my head.

Jack


On Sat, May 8, 2010 at 9:41 AM, joe  wrote:

> On 05/08/2010 11:17 AM, Ian FREISLICH wrote:
>
>> joe wrote:
>>
>>> On 05/08/2010 06:55 AM, Ian FREISLICH wrote:
>>>
 joe wrote:

>   I have just tried your suggeston and it has no effect for me ;(
>

 Do you have another brand of NIC that you can try?  At least that
 will isolate whether it's igb(4) or something else.

>>>
>>> I will grab a new nic today and try...my options are limited though.
>>> Here are the nics i can get my hands on
>>>
>>> TP-LINK TL-TG3468, 10/100/1000Mbps PCIe Adapter (supported by fbsd?)
>>>
>>
>> Based on the RTL8168B chip.  Should be supported by the re(4) driver.
>>
>>  Intel (EXPI9301CT) Gigabit CT Desktop Adapter (yet another intel nic)
>>>
>>
>> i82574L chip.  Should be supported by the em(4) driver.  I have had
>> good performance in the past with this driver and less than
>> satisfactory performance with the igb(4) driver.
>>
>> That may not be your problem though.  Before you go out and buy,
>> have a look at the amount of interrupt time your slow machine spends
>> in 'top' or 'systat -vm'.  systat will also show the interrupt rate
>> for each driver, perhaps it's not doing interrupt moderation properly.
>> This will manifest as more than about a 1000 per second.  There are
>> loader tunables for the driver to increase the number of transfer
>> descriptors and to tune interrupt moderation.
>>
>> You could try running trafshow (port) on the interface while
>> performing the transfer.  Perhaps promiscuous mode will turn off
>> some hardware feature that will improve things.  It may however
>> break hardware vlanning as it does on my 82575GB 4 port igb card.
>>
>> Ian
>>
>> --
>> Ian Freislich
>>
>
> I bought those two cards anyways, im in a rush to figure out this problem.
> That being said i am still encountering the exact same problem regardless on
> which network card i am running. I am at a complete loss. I am about to try
> a raid card to see if the problem might lay within the onboard sata ports. I
> did pull the server and brought it home so that i can test more things
> quicker.
>
> I am going to try using a raid card instead of the onboard sata ports and
> see if i still encounter the same problem. I would love any suggestions you
> may have on where to go from here to figure out where the problem might be.
>
> joe
>
> ___
> freebsd-current@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
>
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.

2010-05-08 Thread joe 

On 05/08/2010 11:17 AM, Ian FREISLICH wrote:

joe wrote:

On 05/08/2010 06:55 AM, Ian FREISLICH wrote:

joe wrote:

   I have just tried your suggeston and it has no effect for me ;(


Do you have another brand of NIC that you can try?  At least that
will isolate whether it's igb(4) or something else.


I will grab a new nic today and try...my options are limited though.
Here are the nics i can get my hands on

TP-LINK TL-TG3468, 10/100/1000Mbps PCIe Adapter (supported by fbsd?)


Based on the RTL8168B chip.  Should be supported by the re(4) driver.


Intel (EXPI9301CT) Gigabit CT Desktop Adapter (yet another intel nic)


i82574L chip.  Should be supported by the em(4) driver.  I have had
good performance in the past with this driver and less than
satisfactory performance with the igb(4) driver.

That may not be your problem though.  Before you go out and buy,
have a look at the amount of interrupt time your slow machine spends
in 'top' or 'systat -vm'.  systat will also show the interrupt rate
for each driver, perhaps it's not doing interrupt moderation properly.
This will manifest as more than about a 1000 per second.  There are
loader tunables for the driver to increase the number of transfer
descriptors and to tune interrupt moderation.

You could try running trafshow (port) on the interface while
performing the transfer.  Perhaps promiscuous mode will turn off
some hardware feature that will improve things.  It may however
break hardware vlanning as it does on my 82575GB 4 port igb card.

Ian

--
Ian Freislich


I bought those two cards anyways, im in a rush to figure out this 
problem. That being said i am still encountering the exact same problem 
regardless on which network card i am running. I am at a complete loss. 
I am about to try a raid card to see if the problem might lay within the 
onboard sata ports. I did pull the server and brought it home so that i 
can test more things quicker.


I am going to try using a raid card instead of the onboard sata ports 
and see if i still encounter the same problem. I would love any 
suggestions you may have on where to go from here to figure out where 
the problem might be.


joe
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.

2010-05-08 Thread Ian FREISLICH 
joe wrote:
> On 05/08/2010 06:55 AM, Ian FREISLICH wrote:
> > joe wrote:
> >>   I have just tried your suggeston and it has no effect for me ;(
> >
> > Do you have another brand of NIC that you can try?  At least that
> > will isolate whether it's igb(4) or something else.
> 
> I will grab a new nic today and try...my options are limited though. 
> Here are the nics i can get my hands on
> 
> TP-LINK TL-TG3468, 10/100/1000Mbps PCIe Adapter (supported by fbsd?)

Based on the RTL8168B chip.  Should be supported by the re(4) driver.

> Intel (EXPI9301CT) Gigabit CT Desktop Adapter (yet another intel nic)

i82574L chip.  Should be supported by the em(4) driver.  I have had
good performance in the past with this driver and less than
satisfactory performance with the igb(4) driver.

That may not be your problem though.  Before you go out and buy,
have a look at the amount of interrupt time your slow machine spends
in 'top' or 'systat -vm'.  systat will also show the interrupt rate
for each driver, perhaps it's not doing interrupt moderation properly.
This will manifest as more than about a 1000 per second.  There are
loader tunables for the driver to increase the number of transfer
descriptors and to tune interrupt moderation.

You could try running trafshow (port) on the interface while
performing the transfer.  Perhaps promiscuous mode will turn off
some hardware feature that will improve things.  It may however
break hardware vlanning as it does on my 82575GB 4 port igb card.

Ian

--
Ian Freislich
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.

2010-05-08 Thread joe 

On 05/08/2010 06:55 AM, Ian FREISLICH wrote:

joe wrote:

  I have just tried your suggeston and it has no effect for me ;(


Do you have another brand of NIC that you can try?  At least that
will isolate whether it's igb(4) or something else.

Ian

--
Ian Freislich



I will grab a new nic today and try...my options are limited though. 
Here are the nics i can get my hands on


TP-LINK TL-TG3468, 10/100/1000Mbps PCIe Adapter (supported by fbsd?)
Intel (EXPI9301CT) Gigabit CT Desktop Adapter (yet another intel nic)

These are the only two nics the local computer store has in stock. 
Anyone know if the TP LINK is supported? I couldnt find anything to 
suggest it is on google.

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.

2010-05-08 Thread Joe 
I have 3 boxes, each with two nics. One nic for the private network and 
one for the public network.
The private network is all on the same vlan. All 6 nics are on the same 
switch. All connections are 1000tx Full Duplex.


I will call the servers Box A, Box B, and Box C.

When i FTP data between Box A & B i get abou 25MB/sec.
When i FTP data from Box C to Box A or B, i get about 20MB/sec.
When i FTP data from Box A to C i get 10MB/sec
When i FTP data from Box B to C i get 200KB/sec...

Can anyone suggest why i might only be getting 200KB when transfering 
data from Box B to C but not when transferring data from Box A to C?


I tried installing ubuntu to see if the problem was still there and it 
is not. I can FTP data from Box A or B to box C at 10MB/sec, still not 
the gigabit speeds i should be seeing but not the 200KB/sec  i am 
getting with freebsd.


I logged into my switch and it reports that the ports are 1000tx full 
duplex, the same as what freebsd is reporting. The switch & freebsd also 
report no errors.


I really dont know what to do. Nothing anywhere reports showing a 
problem but obviously there is a problem somewhere. I've tried drivers 
1.9.5, 1.8.4, and 1.7.3. If i use 1.8.4 or 1.7.3 i get 50KB/sec 
transferring data from box B whereas if i use 1.9.5 i get 200-300KB sec. 
I should be getting like 50MB/sec ;(


Any help would be grateful. Or, maybe someone could put me in touch with 
the person responsible for the intel nic drivers and i can work with 
them on resolving this issue?


Thanks in advance.


NameMtu Network   Address  Ipkts Ierrs Idrop
Opkts Oerrs  Coll
igb0   1500   00:30:48:9f:11:0425853 0 0
18340 0 0
igb0   1500 216.105.91.14 216.105.91.145   19040 - -
18343 - -
igb1*  1500   00:30:48:9f:11:050 0 0
0 0 0
lo0   16384   72 0 0   
72 0 0
lo0   16384 127.0.0.0/8   127.0.0.1   72 - -   
72 - -


Sysctl info for igb0/1
dev.igb.0.%desc: Intel(R) PRO/1000 Network Connection version - 1.8.4
dev.igb.0.%driver: igb
dev.igb.0.%location: slot=0 function=0
dev.igb.0.%pnpinfo: vendor=0x8086 device=0x10c9 subvendor=0x15d9 
subdevice=0x0100 class=0x02

dev.igb.0.%parent: pci1
dev.igb.0.debug: -1
dev.igb.0.stats: -1
dev.igb.0.flow_control: 3
dev.igb.0.enable_aim: 1
dev.igb.0.low_latency: 128
dev.igb.0.ave_latency: 450
dev.igb.0.bulk_latency: 1200
dev.igb.0.rx_processing_limit: 100

Debug info for igb0
May  8 09:17:57 debbie kernel: igb0: TX(1) Packets sent = 1295
May  8 09:17:57 debbie kernel: igb0: Queue(2) tdh = 106, tdt = 106
May  8 09:17:57 debbie kernel: igb0: TX(2) no descriptors avail event = 0
May  8 09:17:57 debbie kernel: igb0: TX(2) MSIX IRQ Handled = 5353
May  8 09:17:57 debbie kernel: igb0: TX(2) Packets sent = 5450
May  8 09:17:57 debbie kernel: igb0: Queue(3) tdh = 1687, tdt = 1687
May  8 09:17:57 debbie kernel: igb0: TX(3) no descriptors avail event = 0
May  8 09:17:57 debbie kernel: igb0: TX(3) MSIX IRQ Handled = 1335
May  8 09:17:57 debbie kernel: igb0: TX(3) Packets sent = 1354
May  8 09:17:57 debbie kernel: igb0: Queue(0) rdh = 425, rdt = 424
May  8 09:17:57 debbie kernel: igb0: RX(0) Packets received = 16809
May  8 09:17:57 debbie kernel: igb0: RX(0) Split Packets = 12262
May  8 09:17:57 debbie kernel: igb0: RX(0) Byte count = 25894129
May  8 09:17:57 debbie kernel: igb0: RX(0) MSIX IRQ Handled = 43428
May  8 09:17:57 debbie kernel: igb0: RX(0) LRO Queued= 13601
May  8 09:17:57 debbie kernel: igb0: RX(0) LRO Flushed= 9966
May  8 09:17:57 debbie kernel: igb0: Queue(1) rdh = 1700, rdt = 1699
May  8 09:17:57 debbie kernel: igb0: RX(1) Packets received = 1700
May  8 09:17:57 debbie kernel: igb0: RX(1) Split Packets = 937
May  8 09:17:57 debbie kernel: igb0: RX(1) Byte count = 1392691
May  8 09:17:57 debbie kernel: igb0: RX(1) MSIX IRQ Handled = 31888
May  8 09:17:57 debbie kernel: igb0: RX(1) LRO Queued= 1362
May  8 09:17:57 debbie kernel: igb0: RX(1) LRO Flushed= 1351
May  8 09:17:57 debbie kernel: igb0: Queue(2) rdh = 33, rdt = 32
May  8 09:17:57 debbie kernel: igb0: RX(2) Packets received = 6177
May  8 09:17:57 debbie kernel: igb0: RX(2) Split Packets = 1442
May  8 09:17:57 debbie kernel: igb0: RX(2) Byte count = 2518498
May  8 09:17:57 debbie kernel: igb0: RX(2) MSIX IRQ Handled = 36258
May  8 09:17:57 debbie kernel: igb0: RX(2) LRO Queued= 5794
May  8 09:17:57 debbie kernel: igb0: RX(2) LRO Flushed= 5673
May  8 09:17:57 debbie kernel: igb0: Queue(3) rdh = 1418, rdt = 1417
May  8 09:17:57 debbie kernel: igb0: RX(3) Packets received = 1418
May  8 09:17:57 debbie kernel: igb0: RX(3) Split Packets = 939
May  8 09:17:57 debbie kernel: igb0: RX(3) Byte count = 1700710
May  8 09:17:57 debbie kernel: igb0: RX(3) MSIX IRQ Handled = 31399
May  8 09:17:57 debbie kernel: igb0: RX(3) LRO Queued= 1102
May  8 09:17:57 debbie kernel: igb0: RX(3) LRO Flushed= 882
May  8 09:17:57 debbie kernel:

Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.

2010-05-08 Thread Ian FREISLICH 
joe wrote:
>  I have just tried your suggeston and it has no effect for me ;(

Do you have another brand of NIC that you can try?  At least that
will isolate whether it's igb(4) or something else.

Ian

--
Ian Freislich
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.

2010-05-08 Thread joe 

On 05/08/2010 05:54 AM, Fabien Thomas wrote:

Have you tried to disable TSO / LRO?

Fabien

   

I have 3 boxes, each with two nics. One nic for the private network and one for 
the public network.
The private network is all on the same vlan. All 6 nics are on the same switch. 
All connections are 1000tx Full Duplex.

I will call the servers Box A, Box B, and Box C.

When i FTP data between Box A&  B i get abou 25MB/sec.
When i FTP data from Box C to Box A or B, i get about 20MB/sec.
When i FTP data from Box A to C i get 10MB/sec
When i FTP data from Box B to C i get 200KB/sec...

Can anyone suggest why i might only be getting 200KB when transfering data from 
Box B to C but not when transferring data from Box A to C?

I tried installing ubuntu to see if the problem was still there and it is not. 
I can FTP data from Box A or B to box C at 10MB/sec, still not the gigabit 
speeds i should be seeing but not the 200KB/sec  i am getting with freebsd.

I logged into my switch and it reports that the ports are 1000tx full duplex, the 
same as what freebsd is reporting. The switch&  freebsd also report no errors.

I really dont know what to do. Nothing anywhere reports showing a problem but 
obviously there is a problem somewhere. I've tried drivers 1.9.5, 1.8.4, and 
1.7.3. If i use 1.8.4 or 1.7.3 i get 50KB/sec transferring data from box B 
whereas if i use 1.9.5 i get 200-300KB sec. I should be getting like 50MB/sec ;(

Any help would be grateful. Or, maybe someone could put me in touch with the 
person responsible for the intel nic drivers and i can work with them on 
resolving this issue?

Thanks in advance.


NameMtu Network   Address  Ipkts Ierrs IdropOpkts Oerrs 
 Coll
igb0   1500   00:30:48:9f:11:0425853 0 018340 0 
0
igb0   1500 216.105.91.14 216.105.91.145   19040 - -18343 - 
-
igb1*  1500   00:30:48:9f:11:050 0 00 0 
0
lo0   16384   72 0 0   72 0 
0
lo0   16384 127.0.0.0/8   127.0.0.1   72 - -   72 - 
-

Sysctl info for igb0/1
dev.igb.0.%desc: Intel(R) PRO/1000 Network Connection version - 1.8.4
dev.igb.0.%driver: igb
dev.igb.0.%location: slot=0 function=0
dev.igb.0.%pnpinfo: vendor=0x8086 device=0x10c9 subvendor=0x15d9 
subdevice=0x0100 class=0x02
dev.igb.0.%parent: pci1
dev.igb.0.debug: -1
dev.igb.0.stats: -1
dev.igb.0.flow_control: 3
dev.igb.0.enable_aim: 1
dev.igb.0.low_latency: 128
dev.igb.0.ave_latency: 450
dev.igb.0.bulk_latency: 1200
dev.igb.0.rx_processing_limit: 100

Debug info for igb0
May  8 09:17:57 debbie kernel: igb0: TX(1) Packets sent = 1295
May  8 09:17:57 debbie kernel: igb0: Queue(2) tdh = 106, tdt = 106
May  8 09:17:57 debbie kernel: igb0: TX(2) no descriptors avail event = 0
May  8 09:17:57 debbie kernel: igb0: TX(2) MSIX IRQ Handled = 5353
May  8 09:17:57 debbie kernel: igb0: TX(2) Packets sent = 5450
May  8 09:17:57 debbie kernel: igb0: Queue(3) tdh = 1687, tdt = 1687
May  8 09:17:57 debbie kernel: igb0: TX(3) no descriptors avail event = 0
May  8 09:17:57 debbie kernel: igb0: TX(3) MSIX IRQ Handled = 1335
May  8 09:17:57 debbie kernel: igb0: TX(3) Packets sent = 1354
May  8 09:17:57 debbie kernel: igb0: Queue(0) rdh = 425, rdt = 424
May  8 09:17:57 debbie kernel: igb0: RX(0) Packets received = 16809
May  8 09:17:57 debbie kernel: igb0: RX(0) Split Packets = 12262
May  8 09:17:57 debbie kernel: igb0: RX(0) Byte count = 25894129
May  8 09:17:57 debbie kernel: igb0: RX(0) MSIX IRQ Handled = 43428
May  8 09:17:57 debbie kernel: igb0: RX(0) LRO Queued= 13601
May  8 09:17:57 debbie kernel: igb0: RX(0) LRO Flushed= 9966
May  8 09:17:57 debbie kernel: igb0: Queue(1) rdh = 1700, rdt = 1699
May  8 09:17:57 debbie kernel: igb0: RX(1) Packets received = 1700
May  8 09:17:57 debbie kernel: igb0: RX(1) Split Packets = 937
May  8 09:17:57 debbie kernel: igb0: RX(1) Byte count = 1392691
May  8 09:17:57 debbie kernel: igb0: RX(1) MSIX IRQ Handled = 31888
May  8 09:17:57 debbie kernel: igb0: RX(1) LRO Queued= 1362
May  8 09:17:57 debbie kernel: igb0: RX(1) LRO Flushed= 1351
May  8 09:17:57 debbie kernel: igb0: Queue(2) rdh = 33, rdt = 32
May  8 09:17:57 debbie kernel: igb0: RX(2) Packets received = 6177
May  8 09:17:57 debbie kernel: igb0: RX(2) Split Packets = 1442
May  8 09:17:57 debbie kernel: igb0: RX(2) Byte count = 2518498
May  8 09:17:57 debbie kernel: igb0: RX(2) MSIX IRQ Handled = 36258
May  8 09:17:57 debbie kernel: igb0: RX(2) LRO Queued= 5794
May  8 09:17:57 debbie kernel: igb0: RX(2) LRO Flushed= 5673
May  8 09:17:57 debbie kernel: igb0: Queue(3) rdh = 1418, rdt = 1417
May  8 09:17:57 debbie kernel: igb0: RX(3) Packets received = 1418
May  8 09:17:57 debbie kernel: igb0: RX(3) Split Packets = 939
May  8 09:17:57 debbie kernel: igb0: RX(3) Byte count = 1700710
May  8 09:17:57 debbie kernel: igb0: RX(3) MSIX IRQ Handled = 31399
May  8 09:17:57 debbie kernel: igb0: RX(3) LRO Queued= 1102
May

Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.

2010-05-08 Thread joe 

On 05/08/2010 05:08 AM, joe wrote:
I have 3 boxes, each with two nics. One nic for the private network 
and one for the public network.
The private network is all on the same vlan. All 6 nics are on the 
same switch. All connections are 1000tx Full Duplex.


I will call the servers Box A, Box B, and Box C.

When i FTP data between Box A & B i get abou 25MB/sec.
When i FTP data from Box C to Box A or B, i get about 20MB/sec.
When i FTP data from Box A to C i get 10MB/sec
When i FTP data from Box B to C i get 200KB/sec...

Can anyone suggest why i might only be getting 200KB when transfering 
data from Box B to C but not when transferring data from Box A to C?


I tried installing ubuntu to see if the problem was still there and it 
is not. I can FTP data from Box A or B to box C at 10MB/sec, still not 
the gigabit speeds i should be seeing but not the 200KB/sec  i am 
getting with freebsd.


I logged into my switch and it reports that the ports are 1000tx full 
duplex, the same as what freebsd is reporting. The switch & freebsd 
also report no errors.


I really dont know what to do. Nothing anywhere reports showing a 
problem but obviously there is a problem somewhere. I've tried drivers 
1.9.5, 1.8.4, and 1.7.3. If i use 1.8.4 or 1.7.3 i get 50KB/sec 
transferring data from box B whereas if i use 1.9.5 i get 200-300KB 
sec. I should be getting like 50MB/sec ;(


Any help would be grateful. Or, maybe someone could put me in touch 
with the person responsible for the intel nic drivers and i can work 
with them on resolving this issue?


Thanks in advance.


NameMtu Network   Address  Ipkts Ierrs Idrop
Opkts Oerrs  Coll
igb0   1500   00:30:48:9f:11:0425853 0 0
18340 0 0
igb0   1500 216.105.91.14 216.105.91.145   19040 - -
18343 - -
igb1*  1500   00:30:48:9f:11:050 0 
00 0 0
lo0   16384   72 0 0   
72 0 0
lo0   16384 127.0.0.0/8   127.0.0.1   72 - -   
72 - -


Sysctl info for igb0/1
dev.igb.0.%desc: Intel(R) PRO/1000 Network Connection version - 1.8.4
dev.igb.0.%driver: igb
dev.igb.0.%location: slot=0 function=0
dev.igb.0.%pnpinfo: vendor=0x8086 device=0x10c9 subvendor=0x15d9 
subdevice=0x0100 class=0x02

dev.igb.0.%parent: pci1
dev.igb.0.debug: -1
dev.igb.0.stats: -1
dev.igb.0.flow_control: 3
dev.igb.0.enable_aim: 1
dev.igb.0.low_latency: 128
dev.igb.0.ave_latency: 450
dev.igb.0.bulk_latency: 1200
dev.igb.0.rx_processing_limit: 100

Debug info for igb0
May  8 09:17:57 debbie kernel: igb0: TX(1) Packets sent = 1295
May  8 09:17:57 debbie kernel: igb0: Queue(2) tdh = 106, tdt = 106
May  8 09:17:57 debbie kernel: igb0: TX(2) no descriptors avail event = 0
May  8 09:17:57 debbie kernel: igb0: TX(2) MSIX IRQ Handled = 5353
May  8 09:17:57 debbie kernel: igb0: TX(2) Packets sent = 5450
May  8 09:17:57 debbie kernel: igb0: Queue(3) tdh = 1687, tdt = 1687
May  8 09:17:57 debbie kernel: igb0: TX(3) no descriptors avail event = 0
May  8 09:17:57 debbie kernel: igb0: TX(3) MSIX IRQ Handled = 1335
May  8 09:17:57 debbie kernel: igb0: TX(3) Packets sent = 1354
May  8 09:17:57 debbie kernel: igb0: Queue(0) rdh = 425, rdt = 424
May  8 09:17:57 debbie kernel: igb0: RX(0) Packets received = 16809
May  8 09:17:57 debbie kernel: igb0: RX(0) Split Packets = 12262
May  8 09:17:57 debbie kernel: igb0: RX(0) Byte count = 25894129
May  8 09:17:57 debbie kernel: igb0: RX(0) MSIX IRQ Handled = 43428
May  8 09:17:57 debbie kernel: igb0: RX(0) LRO Queued= 13601
May  8 09:17:57 debbie kernel: igb0: RX(0) LRO Flushed= 9966
May  8 09:17:57 debbie kernel: igb0: Queue(1) rdh = 1700, rdt = 1699
May  8 09:17:57 debbie kernel: igb0: RX(1) Packets received = 1700
May  8 09:17:57 debbie kernel: igb0: RX(1) Split Packets = 937
May  8 09:17:57 debbie kernel: igb0: RX(1) Byte count = 1392691
May  8 09:17:57 debbie kernel: igb0: RX(1) MSIX IRQ Handled = 31888
May  8 09:17:57 debbie kernel: igb0: RX(1) LRO Queued= 1362
May  8 09:17:57 debbie kernel: igb0: RX(1) LRO Flushed= 1351
May  8 09:17:57 debbie kernel: igb0: Queue(2) rdh = 33, rdt = 32
May  8 09:17:57 debbie kernel: igb0: RX(2) Packets received = 6177
May  8 09:17:57 debbie kernel: igb0: RX(2) Split Packets = 1442
May  8 09:17:57 debbie kernel: igb0: RX(2) Byte count = 2518498
May  8 09:17:57 debbie kernel: igb0: RX(2) MSIX IRQ Handled = 36258
May  8 09:17:57 debbie kernel: igb0: RX(2) LRO Queued= 5794
May  8 09:17:57 debbie kernel: igb0: RX(2) LRO Flushed= 5673
May  8 09:17:57 debbie kernel: igb0: Queue(3) rdh = 1418, rdt = 1417
May  8 09:17:57 debbie kernel: igb0: RX(3) Packets received = 1418
May  8 09:17:57 debbie kernel: igb0: RX(3) Split Packets = 939
May  8 09:17:57 debbie kernel: igb0: RX(3) Byte count = 1700710
May  8 09:17:57 debbie kernel: igb0: RX(3) MSIX IRQ Handled = 31399
May  8 09:17:57 debbie kernel: igb0: RX(3) LRO Queued= 1102
May  8 09:17:57 debbie kernel: igb0: RX(3) LRO Flushed

igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.

2010-05-08 Thread joe 
I have 3 boxes, each with two nics. One nic for the private network and 
one for the public network.
The private network is all on the same vlan. All 6 nics are on the same 
switch. All connections are 1000tx Full Duplex.


I will call the servers Box A, Box B, and Box C.

When i FTP data between Box A & B i get abou 25MB/sec.
When i FTP data from Box C to Box A or B, i get about 20MB/sec.
When i FTP data from Box A to C i get 10MB/sec
When i FTP data from Box B to C i get 200KB/sec...

Can anyone suggest why i might only be getting 200KB when transfering 
data from Box B to C but not when transferring data from Box A to C?


I tried installing ubuntu to see if the problem was still there and it 
is not. I can FTP data from Box A or B to box C at 10MB/sec, still not 
the gigabit speeds i should be seeing but not the 200KB/sec  i am 
getting with freebsd.


I logged into my switch and it reports that the ports are 1000tx full 
duplex, the same as what freebsd is reporting. The switch & freebsd also 
report no errors.


I really dont know what to do. Nothing anywhere reports showing a 
problem but obviously there is a problem somewhere. I've tried drivers 
1.9.5, 1.8.4, and 1.7.3. If i use 1.8.4 or 1.7.3 i get 50KB/sec 
transferring data from box B whereas if i use 1.9.5 i get 200-300KB sec. 
I should be getting like 50MB/sec ;(


Any help would be grateful. Or, maybe someone could put me in touch with 
the person responsible for the intel nic drivers and i can work with 
them on resolving this issue?


Thanks in advance.


NameMtu Network   Address  Ipkts Ierrs Idrop
Opkts Oerrs  Coll
igb0   1500   00:30:48:9f:11:0425853 0 0
18340 0 0
igb0   1500 216.105.91.14 216.105.91.145   19040 - -
18343 - -
igb1*  1500   00:30:48:9f:11:050 0 0
0 0 0
lo0   16384   72 0 0   
72 0 0
lo0   16384 127.0.0.0/8   127.0.0.1   72 - -   
72 - -


Sysctl info for igb0/1
dev.igb.0.%desc: Intel(R) PRO/1000 Network Connection version - 1.8.4
dev.igb.0.%driver: igb
dev.igb.0.%location: slot=0 function=0
dev.igb.0.%pnpinfo: vendor=0x8086 device=0x10c9 subvendor=0x15d9 
subdevice=0x0100 class=0x02

dev.igb.0.%parent: pci1
dev.igb.0.debug: -1
dev.igb.0.stats: -1
dev.igb.0.flow_control: 3
dev.igb.0.enable_aim: 1
dev.igb.0.low_latency: 128
dev.igb.0.ave_latency: 450
dev.igb.0.bulk_latency: 1200
dev.igb.0.rx_processing_limit: 100

Debug info for igb0
May  8 09:17:57 debbie kernel: igb0: TX(1) Packets sent = 1295
May  8 09:17:57 debbie kernel: igb0: Queue(2) tdh = 106, tdt = 106
May  8 09:17:57 debbie kernel: igb0: TX(2) no descriptors avail event = 0
May  8 09:17:57 debbie kernel: igb0: TX(2) MSIX IRQ Handled = 5353
May  8 09:17:57 debbie kernel: igb0: TX(2) Packets sent = 5450
May  8 09:17:57 debbie kernel: igb0: Queue(3) tdh = 1687, tdt = 1687
May  8 09:17:57 debbie kernel: igb0: TX(3) no descriptors avail event = 0
May  8 09:17:57 debbie kernel: igb0: TX(3) MSIX IRQ Handled = 1335
May  8 09:17:57 debbie kernel: igb0: TX(3) Packets sent = 1354
May  8 09:17:57 debbie kernel: igb0: Queue(0) rdh = 425, rdt = 424
May  8 09:17:57 debbie kernel: igb0: RX(0) Packets received = 16809
May  8 09:17:57 debbie kernel: igb0: RX(0) Split Packets = 12262
May  8 09:17:57 debbie kernel: igb0: RX(0) Byte count = 25894129
May  8 09:17:57 debbie kernel: igb0: RX(0) MSIX IRQ Handled = 43428
May  8 09:17:57 debbie kernel: igb0: RX(0) LRO Queued= 13601
May  8 09:17:57 debbie kernel: igb0: RX(0) LRO Flushed= 9966
May  8 09:17:57 debbie kernel: igb0: Queue(1) rdh = 1700, rdt = 1699
May  8 09:17:57 debbie kernel: igb0: RX(1) Packets received = 1700
May  8 09:17:57 debbie kernel: igb0: RX(1) Split Packets = 937
May  8 09:17:57 debbie kernel: igb0: RX(1) Byte count = 1392691
May  8 09:17:57 debbie kernel: igb0: RX(1) MSIX IRQ Handled = 31888
May  8 09:17:57 debbie kernel: igb0: RX(1) LRO Queued= 1362
May  8 09:17:57 debbie kernel: igb0: RX(1) LRO Flushed= 1351
May  8 09:17:57 debbie kernel: igb0: Queue(2) rdh = 33, rdt = 32
May  8 09:17:57 debbie kernel: igb0: RX(2) Packets received = 6177
May  8 09:17:57 debbie kernel: igb0: RX(2) Split Packets = 1442
May  8 09:17:57 debbie kernel: igb0: RX(2) Byte count = 2518498
May  8 09:17:57 debbie kernel: igb0: RX(2) MSIX IRQ Handled = 36258
May  8 09:17:57 debbie kernel: igb0: RX(2) LRO Queued= 5794
May  8 09:17:57 debbie kernel: igb0: RX(2) LRO Flushed= 5673
May  8 09:17:57 debbie kernel: igb0: Queue(3) rdh = 1418, rdt = 1417
May  8 09:17:57 debbie kernel: igb0: RX(3) Packets received = 1418
May  8 09:17:57 debbie kernel: igb0: RX(3) Split Packets = 939
May  8 09:17:57 debbie kernel: igb0: RX(3) Byte count = 1700710
May  8 09:17:57 debbie kernel: igb0: RX(3) MSIX IRQ Handled = 31399
May  8 09:17:57 debbie kernel: igb0: RX(3) LRO Queued= 1102
May  8 09:17:57 debbie kernel: igb0: RX(3) LRO Flushed= 882
May  8 09:17:57 debbie kernel: