Re: [Discuss-gnuradio] UHD host resources and system suggestions

[Marc Epard]

On Sep 7, 2010, at 11:46 AM, Josh Blum wrote:

> sequence errors are not normal, something may be wrong with the fpga image. 
> we are trying to duplicate it out over here -josh

Thanks, Josh. If it's helpful, you'll find the source to my app at 
.

-Marc


___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
http://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] UHD host resources and system suggestions

[Josh Blum]

sequence errors are not normal, something may be wrong with the fpga 
image. we are trying to duplicate it out over here -josh


On 09/07/2010 12:22 PM, Marc Epard wrote:

On Sep 3, 2010, at 5:57 PM, Marcus D. Leech wrote:


I'll comment that a 1.6GHz machine that's getting close to 4 years old
isn't particularly "beefy" by todays standards.  Certainly not
  beefy enough to sustain two 25Msps threads without "problems".


Thanks, Marcus. That was helpful and as I suspected. So over the weekend I built* a new 
system that should eliminate the speed concern (for now ;-). It's an Intel Core i7-980x 
with 6GB of the "world's fastest" XMP-certified RAM. 64-bit Ubuntu screams on 
it. Running make with --jobs=8 builds gnu-radio and Ettus UHD rather quickly. My UHD app 
is running much better and the GUI is actually usable while I'm doing a transmit+receive 
run.

However, the transmit side is still troublesome. Periodically, a burst of Us 
comes out of the UART and there are always a lot of 
EVENT_CODE_SEQ_ERROR_IN_BURST events -- like several thousands per second. I've 
tried this on two different ethernet NICs identified by Ubuntu as an Intel 
82567V-2 (on the motherboard) and a Realtek RTL8111/8166B (in a PCIe slot). I 
have the transmit and receive buffers set to 500,000,000.

I'm digging into the host and USRP2 source to figure out where 
EVENT_CODE_SEQ_ERROR_IN_BURST come from and how transmit flow control works.

Any pointers would be appreciated.

-Marc

* It's the first time I've built a system as I normally leave that to Apple. It 
was fun.


___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
http://lists.gnu.org/mailman/listinfo/discuss-gnuradio


___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
http://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] UHD host resources and system suggestions

[Marc Epard]

On Sep 3, 2010, at 5:57 PM, Marcus D. Leech wrote:

> I'll comment that a 1.6GHz machine that's getting close to 4 years old
> isn't particularly "beefy" by todays standards.  Certainly not
>  beefy enough to sustain two 25Msps threads without "problems".

Thanks, Marcus. That was helpful and as I suspected. So over the weekend I 
built* a new system that should eliminate the speed concern (for now ;-). It's 
an Intel Core i7-980x with 6GB of the "world's fastest" XMP-certified RAM. 
64-bit Ubuntu screams on it. Running make with --jobs=8 builds gnu-radio and 
Ettus UHD rather quickly. My UHD app is running much better and the GUI is 
actually usable while I'm doing a transmit+receive run. 

However, the transmit side is still troublesome. Periodically, a burst of Us 
comes out of the UART and there are always a lot of 
EVENT_CODE_SEQ_ERROR_IN_BURST events -- like several thousands per second. I've 
tried this on two different ethernet NICs identified by Ubuntu as an Intel 
82567V-2 (on the motherboard) and a Realtek RTL8111/8166B (in a PCIe slot). I 
have the transmit and receive buffers set to 500,000,000.

I'm digging into the host and USRP2 source to figure out where 
EVENT_CODE_SEQ_ERROR_IN_BURST come from and how transmit flow control works. 

Any pointers would be appreciated.

-Marc

* It's the first time I've built a system as I normally leave that to Apple. It 
was fun.


___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
http://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] UHD host resources and system suggestions

[Marcus D. Leech]

On 09/03/2010 04:52 PM, Marc Epard wrote:
> Sorry to reply to myself, but we just got the new Ethernet NIC so both USRP2s 
> report WE_TX flow control. Nothing changed.
>
> And I have a correction for the types of ethernet cards. The onboard NIC is 
> the Broadcom. Ethernet 2 (and now 3) appear to be Realtek RTL8111/8168B.
>
> On Sep 3, 2010, at 3:10 PM, Marc Epard wrote:
>
>   
I'll comment that a 1.6GHz machine that's getting close to 4 years old
isn't particularly "beefy" by todays standards.  Certainly not
  beefy enough to sustain two 25Msps threads without "problems".

Just the interrupt load alone is rather frightening.  The packets used
by the USRP2 aren't terribly big, something like 300-500 bytes
  I think.  So, 25e6 / 3e2 = 83Kpackets/second--depending on your NIC,
each of those packets could be creating an interrupt!

Then you have to actually process the darn things -- copy up to user
space, "do stuff", and present the result of that "stuff" either back out
  to the network, or into your software (FFT display, whatever).

To give some perspective.  I run my radio astronomy Rx-only "stack" on a
6-core machine doing "stuff" that's  of the same order of
  complexity as modern telecom-type signal processing.  At 5 or 6MHz
bandwidth, the machine is working *really* hard.  That's a machine
  with an AMD Phenom II X6 1090t, running at 3.2GHz/core with 6GB of
*fast* memory.  [And I suspect a flaky power supply, but that's
  a topic for another day :-) ].


-- 
Marcus Leech
Principal Investigator
Shirleys Bay Radio Astronomy Consortium
http://www.sbrac.org



___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
http://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] UHD host resources and system suggestions

[Marc Epard]

Sorry to reply to myself, but we just got the new Ethernet NIC so both USRP2s 
report WE_TX flow control. Nothing changed.

And I have a correction for the types of ethernet cards. The onboard NIC is the 
Broadcom. Ethernet 2 (and now 3) appear to be Realtek RTL8111/8168B.

On Sep 3, 2010, at 3:10 PM, Marc Epard wrote:

> I'm a puzzled by some UHD behavior and wondering if I need a new host system. 
> I'm not doing any signal processing in the host -- just transmitting from a 
> buffer and/or receiving from a buffer. Later, we'll be wanting to do more 
> processing with two USRP2s both transmitting and receiving at the same time 
> and may want 25 Msps in some cases.
> 
> I've managed to get simultaneous receive threads going for each of two 
> USRP2s, both at 25 Msps (big buffers are good). The GUI still updates, but is 
> a little sluggish. It's fragile, though. If I do much at all with the GUI 
> (sometimes just moving the mouse), recv's start timing out. System Monitor 
> shows no memory pressure and both CPUs at 50-60%. FWIW, I'm running my C++ 
> app via SSH from another machine.
> 
> Transmitting is worse. Even a solo transmit thread at 25 Msps (plus a thread 
> calling recv_async_msg) causes the GUI to be unusable and I get frequent Us 
> on both the host and the USRP2's serial port, as well as tons of 
> EVENT_CODE_SEQ_ERROR and EVENT_CODE_SEQ_ERROR_IN_BURST messages from 
> recv_async_msg. I can't tell what the CPU utilization is because System 
> Monitor stops updating. 6.25 Msps is better, but I still get a few Us and a 
> bunch of EVENT_CODE_SEQ_ERROR and EVENT_CODE_SEQ_ERROR_IN_BURST messages. 
> System Monitor manages to update and shows CPU utilization of 80% and 55%. 
> Again, I'm only transmitting from a buffer that I fill once up front and send 
> over and over again  with no signal processing.
> 
> I'm specifying recv_buff_size=100e6 and send_buff_size=100e6; 
> net.core.rmem_max and net.core.wmem_max are set to 1. I'm calling 
> uhd::set_thread_priority_safe from each of my threads. I'm using the latest 
> gnuradio and UHD software/firmware/FPGA (the recent release with USRP1 
> support). I have Ubuntu 10.4 with stock ethernet drivers. 
> 
> I've itemized a few details of the hardware setup below. What systems are you 
> all using for MIMO-like work with USRP2s and UHD? What systems do you 
> recommend or wish you had? Are there other changes I should make to my 
> current system?
> 
> TIA
> 
> -Marc
> 
> Dell DCDO tower manufactured in 2007
> Dual-core Xeon 5110 @ 1.60 GHz
> 2 GB RAM
> Lots of slots - mix of PCI, PCIe, PCI-X
> 
> Ethernet 0:  USRP2 with DBSRX
> Onboard 10/100/1000 Ethernet. 
> USRP2 says no flow control and we're getting another PCI Express card.
> 
> Ethernet 1: network connection
> D-Link PCI NIC
> No flow control
> 
> Ethernet 2:  USRP2 with XCVR2450
> StarTech.com PCI Express 10/100/1000 Ethernet NIC 
> Ubuntu identifies it as Broadcom NetXtreme BCM5752
> USRP2 says it gets WE_TX flow control
> 
> 
> ___
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> http://lists.gnu.org/mailman/listinfo/discuss-gnuradio


___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
http://lists.gnu.org/mailman/listinfo/discuss-gnuradio

[Discuss-gnuradio] UHD host resources and system suggestions

[Marc Epard]

I'm a puzzled by some UHD behavior and wondering if I need a new host system. 
I'm not doing any signal processing in the host -- just transmitting from a 
buffer and/or receiving from a buffer. Later, we'll be wanting to do more 
processing with two USRP2s both transmitting and receiving at the same time and 
may want 25 Msps in some cases.

I've managed to get simultaneous receive threads going for each of two USRP2s, 
both at 25 Msps (big buffers are good). The GUI still updates, but is a little 
sluggish. It's fragile, though. If I do much at all with the GUI (sometimes 
just moving the mouse), recv's start timing out. System Monitor shows no memory 
pressure and both CPUs at 50-60%. FWIW, I'm running my C++ app via SSH from 
another machine.

Transmitting is worse. Even a solo transmit thread at 25 Msps (plus a thread 
calling recv_async_msg) causes the GUI to be unusable and I get frequent Us on 
both the host and the USRP2's serial port, as well as tons of 
EVENT_CODE_SEQ_ERROR and EVENT_CODE_SEQ_ERROR_IN_BURST messages from 
recv_async_msg. I can't tell what the CPU utilization is because System Monitor 
stops updating. 6.25 Msps is better, but I still get a few Us and a bunch of 
EVENT_CODE_SEQ_ERROR and EVENT_CODE_SEQ_ERROR_IN_BURST messages. System Monitor 
manages to update and shows CPU utilization of 80% and 55%. Again, I'm only 
transmitting from a buffer that I fill once up front and send over and over 
again  with no signal processing.
 
I'm specifying recv_buff_size=100e6 and send_buff_size=100e6; net.core.rmem_max 
and net.core.wmem_max are set to 1. I'm calling 
uhd::set_thread_priority_safe from each of my threads. I'm using the latest 
gnuradio and UHD software/firmware/FPGA (the recent release with USRP1 
support). I have Ubuntu 10.4 with stock ethernet drivers. 

I've itemized a few details of the hardware setup below. What systems are you 
all using for MIMO-like work with USRP2s and UHD? What systems do you recommend 
or wish you had? Are there other changes I should make to my current system?

TIA

-Marc

Dell DCDO tower manufactured in 2007
Dual-core Xeon 5110 @ 1.60 GHz
2 GB RAM
Lots of slots - mix of PCI, PCIe, PCI-X

Ethernet 0:  USRP2 with DBSRX
Onboard 10/100/1000 Ethernet. 
USRP2 says no flow control and we're getting another PCI Express card.

Ethernet 1: network connection
D-Link PCI NIC
No flow control

Ethernet 2:  USRP2 with XCVR2450
StarTech.com PCI Express 10/100/1000 Ethernet NIC 
Ubuntu identifies it as Broadcom NetXtreme BCM5752
USRP2 says it gets WE_TX flow control


___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
http://lists.gnu.org/mailman/listinfo/discuss-gnuradio