I am not certain, but I believe that the benchmark_rate example in UHD 3.12 does not support "sync-ing" of multiple devices. The recent versions of benchmark_rate do support this. In any case, it is not difficult to add if you want (configuring all devices to use external Ref/PPS and then resetting the clocks for all devices at a common PPS). The "sequence" errors below seem consistent with this hypothesis that the devices are not sync-ed.
So, it seems that you do not have "performance issues" if you are able to run 4x100 MS/s in the single device case. In your original email, you mentioned that you were running "rx_samples_to_file". Maybe you could try changing the file to "/dev/null" (I think that is right) in order to simply "throw away" the received samples. Or, you could just modify the source code to skip the file writing. This might let you know if the file writing is causing havoc. Rob On Wed, Dec 20, 2023 at 6:41 AM Anabel Almodovar <anabel.almodo...@gmail.com> wrote: > When I run the benchmark_rate sometimes work, sometimes no. With 4 > channels there is no problem. This is one of the error: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *rs3_lab@RS3-lab:~/workarea-uhd/uhd/host/examples/build$ sudo > ./benchmark_rate > --args="addr0=192.168.60.2,second_addr0=192.168.50.2,addr1=192.168.40.2,second_addr1=192.168.30.2,recv_buff_size=900000000" > --rx_subdev="A:0 A:1 B:0 B:1" --rx_rate 100e6 --ref="external" > --pps="external" --channels="0,1,2,3,4,5,6,7"[INFO] [UHD] linux; GNU C++ > version 5.4.0 20160609; Boost_105800; > UHD_3.12.0.HEAD-0-gec786351[00:00:00.000016] Creating the usrp device with: > addr0=192.168.60.2,second_addr0=192.168.50.2,addr1=192.168.40.2,second_addr1=192.168.30.2,recv_buff_size=900000000...[INFO] > [X300] X300 initialization sequence...[INFO] [X300] Maximum frame size: > 8000 bytes.[INFO] [X300] Maximum frame size: 8000 bytes.[INFO] [X300] > Maximum frame size: 8000 bytes.[INFO] [X300] Maximum frame size: 8000 > bytes.[INFO] [X300] Radio 1x clock: 200 MHz[INFO] [X300] Radio 1x clock: > 200 MHz[INFO] [0/DmaFIFO_0] Initializing block control (NOC ID: > 0xF1F0D00000000000)[INFO] [0/DmaFIFO_0] BIST passed (Throughput: 1292 > MB/s)[INFO] [0/DmaFIFO_0] BIST passed (Throughput: 1308 MB/s)[INFO] > [0/Radio_0] Initializing block control (NOC ID: 0x12AD100000000001)[INFO] > [0/Radio_1] Initializing block control (NOC ID: 0x12AD100000000001)[INFO] > [0/DDC_0] Initializing block control (NOC ID: 0xDDC0000000000000)[INFO] > [0/DDC_1] Initializing block control (NOC ID: 0xDDC0000000000000)[INFO] > [0/DUC_0] Initializing block control (NOC ID: 0xD0C0000000000000)[INFO] > [0/DUC_1] Initializing block control (NOC ID: 0xD0C0000000000000)[WARNING] > [X300] Cannot update master clock rate! X300 Series does not allow changing > the clock rate during runtime.[INFO] [GPS] No GPSDO found[INFO] > [1/DmaFIFO_0] Initializing block control (NOC ID: 0xF1F0D00000000000)[INFO] > [1/DmaFIFO_0] BIST passed (Throughput: 1316 MB/s)[INFO] [1/DmaFIFO_0] BIST > passed (Throughput: 1315 MB/s)[INFO] [1/Radio_0] Initializing block control > (NOC ID: 0x12AD100000000001)[INFO] [1/Radio_1] Initializing block control > (NOC ID: 0x12AD100000000001)[INFO] [1/DDC_0] Initializing block control > (NOC ID: 0xDDC0000000000000)[INFO] [1/DDC_1] Initializing block control > (NOC ID: 0xDDC0000000000000)[INFO] [1/DUC_0] Initializing block control > (NOC ID: 0xD0C0000000000000)[INFO] [1/DUC_1] Initializing block control > (NOC ID: 0xD0C0000000000000)[WARNING] [X300] Cannot update master clock > rate! X300 Series does not allow changing the clock rate during > runtime.[WARNING] [X300 RADIO] Requesting invalid sampling rate from > device: 200 MHz. Actual rate is: 100 MHz.[WARNING] [X300 RADIO] Requesting > invalid sampling rate from device: 200 MHz. Actual rate is: 100 > MHz.[WARNING] [X300 RADIO] Requesting invalid sampling rate from device: > 200 MHz. Actual rate is: 100 MHz.[WARNING] [X300 RADIO] Requesting invalid > sampling rate from device: 200 MHz. Actual rate is: 100 MHz.Using Device: > Multi USRP: Device: X-Series Device Mboard 0: X310 Mboard 1: X310 RX > Channel: 0 RX DSP: 0 RX Dboard: A RX Subdev: TwinRX RX0 RX > Channel: 1 RX DSP: 1 RX Dboard: A RX Subdev: TwinRX RX1 RX > Channel: 2 RX DSP: 0 RX Dboard: B RX Subdev: TwinRX RX0 RX > Channel: 3 RX DSP: 1 RX Dboard: B RX Subdev: TwinRX RX1 RX > Channel: 4 RX DSP: 0 RX Dboard: A RX Subdev: TwinRX RX0 RX > Channel: 5 RX DSP: 1 RX Dboard: A RX Subdev: TwinRX RX1 RX > Channel: 6 RX DSP: 0 RX Dboard: B RX Subdev: TwinRX RX0 RX > Channel: 7 RX DSP: 1 RX Dboard: B RX Subdev: TwinRX RX1 TX > Channel: 0 TX DSP: 0 TX Dboard: A TX Subdev: Unknown (0xffff) - 0 > TX Channel: 1 TX DSP: 0 TX Dboard: B TX Subdev: Unknown (0xffff) - > 0 TX Channel: 2 TX DSP: 0 TX Dboard: A TX Subdev: Unknown > (0xffff) - 0 TX Channel: 3 TX DSP: 0 TX Dboard: B TX Subdev: > Unknown (0xffff) - 0Now confirming lock on clock > signals...[00:00:04.509398] Setting device timestamp to 0...[INFO] > [MULTI_USRP] 1) catch time transition at pps edge[INFO] [MULTI_USRP] > 2) set times next pps (synchronously)[00:00:06.357833] Testing receive > rate 100.000000 Msps on 8 channelsD[00:00:06.408574] Detected Rx sequence > error.D[00:00:06.427168] Detected Rx sequence error.[00:00:06.532780] > Receiver error: ERROR_CODE_TIMEOUT, continuing...[00:00:06.632926] Receiver > error: ERROR_CODE_TIMEOUT, continuing...[00:00:06.733147] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[00:00:06.833421] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[00:00:06.933685] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[00:00:07.033952] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[00:00:07.134184] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[00:00:07.234537] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[00:00:07.334828] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[00:00:07.435189] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[00:00:07.535456] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[00:00:07.635749] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[ERROR] [X300] 192.168.40.2 > <http://192.168.40.2>: x300 fw communication failure #1EnvironmentError: > IOError: x300 fw poke32 - reply timed out[00:00:07.736035] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[00:00:07.836326] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[00:00:07.936631] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[00:00:08.036924] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[00:00:08.137192] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[00:00:08.237608] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[00:00:08.337885] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[00:00:08.438185] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[00:00:08.538479] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[00:00:08.638731] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[ERROR] [X300] 192.168.40.2 > <http://192.168.40.2>: x300 fw communication failure #2EnvironmentError: > IOError: x300 fw poke32 - reply timed out[00:00:08.738960] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[00:00:08.839209] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[00:00:08.939471] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[00:00:09.039712] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[00:00:09.139951] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[00:00:09.240207] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[00:00:09.340462] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[00:00:09.440678] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[00:00:09.540942] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[00:00:09.641176] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[ERROR] [X300] 192.168.40.2 > <http://192.168.40.2>: x300 fw communication failure #3EnvironmentError: > IOError: x300 fw poke32 - reply timed out[ERROR] [UHD] An unexpected > exception was caught in a task loop.The task loop will now exit, things may > not work.EnvironmentError: IOError: 192.168.40.2 <http://192.168.40.2>: > x300 fw communication failure #3EnvironmentError: IOError: x300 fw poke32 - > reply timed out[00:00:09.741400] Receiver error: ERROR_CODE_TIMEOUT, > continuing...[00:00:09.841666] Receiver error: ERROR_CODE_TIMEOUT, > continuing...[ERROR] [STREAMER] The receive packet handler failed to > time-align packets. 1002 received packets were processed by the handler. > However, a timestamp match could not be determined.[00:00:11.039474] > Timestamp after overrun recovery ahead of error timestamp! Unable to > calculate number of dropped samples.(Delta: -14354204 > ticks)[D00:00:11.039576] Detected Rx sequence error.O[00:00:11.282342] > Receiver error: ERROR_CODE_TIMEOUT, continuing...D[00:00:11.282549] > Detected Rx sequence error.O[00:00:11.533164] Receiver error: > ERROR_CODE_TIMEOUT, continuing...[00:00:11.533285] Detected Rx sequence > error.DO[00:00:11.777357] Receiver error: ERROR_CODE_TIMEOUT, > continuing...[00:00:11.777444] Detected Rx sequence > error.DO[00:00:12.024195] Receiver error: ERROR_CODE_TIMEOUT, > continuing...[00:00:12.024298D] Detected Rx sequence > error.OO[00:00:12.352749] Receiver error: ERROR_CODE_TIMEOUT, > continuing...[00:00:12.452949] Receiver error: ERROR_CODE_TIMEOUT, > continuing...[00:00:12.553142] Receiver error: ERROR_CODE_TIMEOUT, > continuing...[00:00:12.653400] Receiver error: ERROR_CODE_TIMEOUT, > continuing...[00:00:12.753649] Receiver error: ERROR_CODE_TIMEOUT, > continuing...[00:00:12.853960] Receiver error: ERROR_CODE_TIMEOUT, > continuing...[00:00:12.954233] Receiver error: ERROR_CODE_TIMEOUT, > continuing...[00:00:13.054580] Receiver error: ERROR_CODE_TIMEOUT, > continuing...[00:00:13.154806] Receiver error: ERROR_CODE_TIMEOUT, > continuing...[00:00:13.255107] Receiver error: ERROR_CODE_TIMEOUT, > continuing...[00:00:13.355408] Receiver error: ERROR_CODE_TIMEOUT, > continuing...[00:00:13.455681] Receiver error: ERROR_CODE_TIMEOUT, > continuing...[00:00:13.555943] Receiver error: ERROR_CODE_TIMEOUT, > continuing...[00:00:13.656214] Receiver error: ERROR_CODE_TIMEOUT, > continuing...[00:00:13.756477] Receiver error: ERROR_CODE_TIMEOUT, > continuing...[00:00:13.856742] Receiver error: ERROR_CODE_TIMEOUT, > continuing...O[00:00:14.813040] Receiver error: ERROR_CODE_TIMEOUT, > continuing...D[00:00:14.813167] Detected Rx sequence > error.O[00:00:15.039704] Receiver error: ERROR_CODE_TIMEOUT, > continuing...D[00:00:15.039807] Detected Rx sequence > error.O[00:00:15.295388] Receiver error: ERROR_CODE_TIMEOUT, > continuing...[00:00:15.395547] Receiver error: ERROR_CODE_TIMEOUT, > continuing...[00:00:15.495668] Receiver error: ERROR_CODE_TIMEOUT, > continuing...[00:00:15.595768] Receiver error: ERROR_CODE_TIMEOUT, > continuing...[00:00:15.695879] Receiver error: ERROR_CODE_TIMEOUT, > continuing...[00:00:15.795982] Receiver error: ERROR_CODE_TIMEOUT, > continuing...[00:00:15.896079] Receiver error: ERROR_CODE_TIMEOUT, > continuing...[00:00:15.996190] Receiver error: ERROR_CODE_TIMEOUT, > continuing...[00:00:16.096293] Receiver error: ERROR_CODE_TIMEOUT, > continuing...[00:00:16.196395] Receiver error: ERROR_CODE_TIMEOUT, > continuing...[00:00:16.296500] Receiver error: ERROR_CODE_TIMEOUT, > continuing...[00:00:17.096016] Benchmark complete.Benchmark rate summary: > Num received samples: 14802336 Num dropped samples: 31269466 Num > overruns detected: 9 Num transmitted samples: 0 Num sequence errors > (Tx): 0 Num sequence errors (Rx): 9 Num underruns detected: 0 Num late > commands: 0 Num timeouts (Tx): 0 Num timeouts (Rx): > 67Done!* > > And this is other: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *rs3_lab@RS3-lab:~/workarea-uhd/uhd/host/examples/build$ sudo > ./benchmark_rate > --args="addr0=192.168.60.2,second_addr0=192.168.50.2,addr1=192.168.40.2,second_addr1=192.168.30.2,recv_buff_size=900000000" > --rx_subdev="A:0 A:1 B:0 B:1" --rx_rate 100e6 --ref="external" > --pps="external" --channels="0,1,2,3,4,5,6,7"[INFO] [UHD] linux; GNU C++ > version 5.4.0 20160609; Boost_105800; > UHD_3.12.0.HEAD-0-gec786351[00:00:00.000014] Creating the usrp device with: > addr0=192.168.60.2,second_addr0=192.168.50.2,addr1=192.168.40.2,second_addr1=192.168.30.2,recv_buff_size=900000000...[INFO] > [X300] X300 initialization sequence...[INFO] [X300] Maximum frame size: > 8000 bytes.[INFO] [X300] Maximum frame size: 8000 bytes.[INFO] [X300] > Maximum frame size: 8000 bytes.[INFO] [X300] Maximum frame size: 8000 > bytes.[INFO] [X300] Radio 1x clock: 200 MHz[INFO] [0/DmaFIFO_0] > Initializing block control (NOC ID: 0xF1F0D00000000000)[INFO] [X300] Radio > 1x clock: 200 MHz[INFO] [0/DmaFIFO_0] BIST passed (Throughput: 1317 > MB/s)[INFO] [0/DmaFIFO_0] BIST passed (Throughput: 1312 MB/s)[INFO] > [1/DmaFIFO_0] Initializing block control (NOC ID: 0xF1F0D00000000000)[INFO] > [1/DmaFIFO_0] BIST passed (Throughput: 1319 MB/s)[INFO] [1/DmaFIFO_0] BIST > passed (Throughput: 1301 MB/s)[INFO] [0/Radio_0] Initializing block control > (NOC ID: 0x12AD100000000001)[INFO] [1/Radio_0] Initializing block control > (NOC ID: 0x12AD100000000001)[INFO] [0/Radio_1] Initializing block control > (NOC ID: 0x12AD100000000001)[INFO] [1/Radio_1] Initializing block control > (NOC ID: 0x12AD100000000001)[INFO] [0/DDC_0] Initializing block control > (NOC ID: 0xDDC0000000000000)[INFO] [1/DDC_0] Initializing block control > (NOC ID: 0xDDC0000000000000)[INFO] [0/DDC_1] Initializing block control > (NOC ID: 0xDDC0000000000000)[INFO] [1/DDC_1] Initializing block control > (NOC ID: 0xDDC0000000000000)[INFO] [0/DUC_0] Initializing block control > (NOC ID: 0xD0C0000000000000)[INFO] [1/DUC_0] Initializing block control > (NOC ID: 0xD0C0000000000000)[INFO] [0/DUC_1] Initializing block control > (NOC ID: 0xD0C0000000000000)[INFO] [1/DUC_1] Initializing block control > (NOC ID: 0xD0C0000000000000)[WARNING] [X300] Cannot update master clock > rate! X300 Series does not allow changing the clock rate during > runtime.[WARNING] [X300] Cannot update master clock rate! X300 Series does > not allow changing the clock rate during runtime.[WARNING] [X300 RADIO] > Requesting invalid sampling rate from device: 200 MHz. Actual rate is: 100 > MHz.[WARNING] [X300 RADIO] Requesting invalid sampling rate from device: > 200 MHz. Actual rate is: 100 MHz.[WARNING] [X300 RADIO] Requesting invalid > sampling rate from device: 200 MHz. Actual rate is: 100 MHz.[WARNING] [X300 > RADIO] Requesting invalid sampling rate from device: 200 MHz. Actual rate > is: 100 MHz.Using Device: Multi USRP: Device: X-Series Device Mboard 0: > X310 Mboard 1: X310 RX Channel: 0 RX DSP: 0 RX Dboard: A RX > Subdev: TwinRX RX0 RX Channel: 1 RX DSP: 1 RX Dboard: A RX > Subdev: TwinRX RX1 RX Channel: 2 RX DSP: 0 RX Dboard: B RX > Subdev: TwinRX RX0 RX Channel: 3 RX DSP: 1 RX Dboard: B RX > Subdev: TwinRX RX1 RX Channel: 4 RX DSP: 0 RX Dboard: A RX > Subdev: TwinRX RX0 RX Channel: 5 RX DSP: 1 RX Dboard: A RX > Subdev: TwinRX RX1 RX Channel: 6 RX DSP: 0 RX Dboard: B RX > Subdev: TwinRX RX0 RX Channel: 7 RX DSP: 1 RX Dboard: B RX > Subdev: TwinRX RX1 TX Channel: 0 TX DSP: 0 TX Dboard: A TX > Subdev: Unknown (0xffff) - 0 TX Channel: 1 TX DSP: 0 TX Dboard: B > TX Subdev: Unknown (0xffff) - 0 TX Channel: 2 TX DSP: 0 TX Dboard: > A TX Subdev: Unknown (0xffff) - 0 TX Channel: 3 TX DSP: 0 TX > Dboard: B TX Subdev: Unknown (0xffff) - 0Now confirming lock on clock > signals...[00:00:03.698257] Setting device timestamp to 0...[INFO] > [MULTI_USRP] 1) catch time transition at pps edge[INFO] [MULTI_USRP] > 2) set times next pps (synchronously)[00:00:04.914946] Testing receive > rate 100.000000 Msps on 8 channelsD[00:00:04.985150] Detected Rx sequence > error.D[00:00:04.985335] Detected Rx sequence error.D[00:00:04.985416] > Detected Rx sequence error.D[00:00:04.985481] Detected Rx sequence > error.D[00:00:04.985547] Detected Rx sequence error.D[00:00:04.985627] > Detected Rx sequence error.D[00:00:04.985698] Detected Rx sequence > error.[00:00:04.985783] Detected Rx sequence error.DD[00:00:04.985837] > Detected Rx sequence error.D[00:00:04.985890] Detected Rx sequence > error.D[00:00:04.985936] Detected Rx sequence error.D[00:00:04.985990] > Detected Rx sequence error.D[00:00:04.986057] Detected Rx sequence > error.D[00:00:04.986136] Detected Rx sequence error.[00:00:04.986233] > Detected Rx sequence error.D[00:00:04.986316] Timestamp after overrun > recovery ahead of error timestamp! Unable to calculate number of dropped > samples.(Delta: -9980 ticks)[D00:00:04.986366] Detected Rx sequence > error.[00:00:04.986424] Detected Rx sequence error.D[00:00:04.986491] > Detected Rx sequence error.D[00:00:04.986595] Detected Rx sequence > error.DD[00:00:04.986838] Detected Rx sequence error.[00:00:04.987167] > Detected Rx sequence error.D[00:00:04.987334] Detected Rx sequence > error.DD[00:00:04.987400] Detected Rx sequence error.D[00:00:04.992004] > Detected Rx sequence error.[00:00:04.992086] Detected Rx sequence > error.DD[00:00:04.992132] Detected Rx sequence error.D[00:00:04.992183] > Detected Rx sequence error.D[00:00:04.992231] Detected Rx sequence > error.D[00:00:04.992279] Detected Rx sequence error.[00:00:04.992335] > Detected Rx sequence error.D[00:00:04.992392] Detected Rx sequence > error.D[00:00:04.992453] Detected Rx sequence error.D[00:00:04.992506] > Detected Rx sequence error.DD[00:00:04.992545] Detected Rx sequence > error.[00:00:04.992592] Detected Rx sequence error.D[00:00:04.992657] > Detected Rx sequence error.DD[00:00:04.992711] Detected Rx sequence > error.[00:00:04.992763] Detected Rx sequence error.D[00:00:04.992834D] > Detected Rx sequence error.[00:00:04.992889] Detected Rx sequence > error.D[00:00:04.992947] Detected Rx sequence error.D[00:00:04.993006] > Detected Rx sequence error.D[00:00:04.993071] Detected Rx sequence > error.DD[00:00:04.993121] Detected Rx sequence error.[00:00:04.993173] > Detected Rx sequence error.DD[00:00:04.993213] Detected Rx sequence > error.[00:00:04.993254D] Detected Rx sequence error.[00:00:04.993310] > Detected Rx sequence error.D[00:00:04.993366] Detected Rx sequence > error.DD[00:00:04.993411] Detected Rx sequence error.[00:00:04.993465] > Detected Rx sequence error.D[00:00:04.993521] Detected Rx sequence > error.DD[00:00:04.993567] Detected Rx sequence error.D[00:00:04.993611] > Detected Rx sequence error.[00:00:04.993663D] Detected Rx sequence > error.[00:00:04.993713] Detected Rx sequence error.DD[00:00:04.993752] > Detected Rx sequence error.[00:00:04.993803] Detected Rx sequence > error.D[00:00:04.993859] Detected Rx sequence error.D[00:00:04.993919D] > Detected Rx sequence error.[00:00:04.993977] Detected Rx sequence > error.D[00:00:04.994022] Detected Rx sequence error.D[00:00:04.994070D] > Detected Rx sequence error.[00:00:04.994122] Detected Rx sequence > error.D[00:00:04.994168] Detected Rx sequence error.DD[00:00:04.994218] > Detected Rx sequence error.[00:00:04.994250] Detected Rx sequence > error.D[00:00:08.051291] Detected Rx sequence error.D[00:00:08.051352] > Detected Rx sequence error.D[00:00:08.051403] Detected Rx sequence > error.D[00:00:08.051453] Detected Rx sequence error.DD[00:00:09.406237] > Detected Rx sequence error.[00:00:09.406340] Detected Rx sequence > error.D[00:00:09.406379] Detected Rx sequence error.D[00:00:09.406422] > Detected Rx sequence error.DD[00:00:09.406477] Detected Rx sequence > error.[00:00:09.406525] Detected Rx sequence error.D[00:00:09.406541] > Detected Rx sequence error.D[00:00:09.406577] Detected Rx sequence > error.D[00:00:09.406623] Timestamp after overrun recovery ahead of error > timestamp! Unable to calculate number of dropped samples.(Delta: -1996 > ticks)[00:00:15.715955] Benchmark complete.Benchmark rate summary: Num > received samples: 7958249432 Num dropped samples: 49903 Num > overruns detected: 0 Num transmitted samples: 0 Num sequence errors > (Tx): 0 Num sequence errors (Rx): 79 Num underruns detected: 0 Num > late commands: 0 Num timeouts (Tx): 0 Num timeouts (Rx): > 0Done!* > > > > El lun, 18 dic 2023 a las 19:10, Rob Kossler (<rkoss...@nd.edu>) escribió: > >> How about if you use an unmodified "benchmark_rate"? >> >> On Mon, Dec 18, 2023 at 11:43 AM Anabel Almodovar < >> anabel.almodo...@gmail.com> wrote: >> >>> Even with a single card I still get the same error. >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> *Creating the usrp device with: >>> addr0=192.168.60.2,second_addr0=192.168.50.2,recv_buff_size=900000000...Creating >>> the subdevice with: A:0 A:1 B:0 B:1...Using Device: Single USRP: Device: >>> X-Series Device Mboard 0: X310 RX Channel: 0 RX DSP: 0 RX Dboard: >>> A RX Subdev: TwinRX RX0 RX Channel: 1 RX DSP: 1 RX Dboard: A >>> RX Subdev: TwinRX RX1 RX Channel: 2 RX DSP: 0 RX Dboard: B RX >>> Subdev: TwinRX RX0 RX Channel: 3 RX DSP: 1 RX Dboard: B RX >>> Subdev: TwinRX RX1 TX Channel: 0 TX DSP: 0 TX Dboard: A TX >>> Subdev: Unknown (0xffff) - 0 TX Channel: 1 TX DSP: 0 TX Dboard: B >>> TX Subdev: Unknown (0xffff) - 0Número de canales detectados: 4Número de >>> tarjetas detectadas: 1Actual RX Rate: 50.000000 Msps...Actual Acquisition >>> Time to: 0.000000 Seconds.Actual RX Freq: 800.000000 MHz...Actual RX Gain: >>> 5.000000 dB...Actual RX Bandwidth: 50.000000 MHz...Setting LO >>> source...[INFO] [MULTI_USRP] 1) catch time transition at pps edge[INFO] >>> [MULTI_USRP] 2) set times next pps (synchronously)Press Ctrl + C to >>> stop streaming...STAR SAMPLING ...D[ERROR] [STREAMER] The receive packet >>> handler failed to time-align packets. 1002 received packets were processed >>> by the handler. However, a timestamp match could not be determined.D[ERROR] >>> [STREAMER] The receive packet handler failed to time-align packets. 1002 >>> received packets were processed by the handler. However, a timestamp match >>> could not be determined.^CReceived 199995208 samples in 6.703929 >>> seconds29.8325 MspsDone!* >>> >>> >>> El lun, 18 dic 2023 a las 17:13, Rob Kossler (<rkoss...@nd.edu>) >>> escribió: >>> >>>> Several comments: >>>> >>>> - It seems like multiple things are going on. You mentioned the >>>> original "failed to time align" error and below you mentioned 'O' and >>>> 'D'. >>>> - The time-align error I did not expect had anything to do with >>>> "performance". This seemed to me that the first packet arriving from >>>> device >>>> 1 had a different time stamp than the first packet arriving from >>>> device 2. >>>> Now, I'm not so sure >>>> - But, the 'O' and 'D' are often related to "performance". >>>> Overflow 'O' occurs when the 'Radio' block has A/D samples that it >>>> needs to >>>> put in a FIFO but the FIFO is full because it is not being emptied >>>> fast >>>> enough (presumably by the host PC). A dropped packet (or sequence >>>> error) >>>> 'D' occurs when the host PC detects that the incoming packets are >>>> not in >>>> sequential order. This can occur if the host PC Ethernet handling >>>> becomes >>>> overwhelmed and simply discards a set of incoming packets for a time >>>> period. Both 'O' and 'D' can imply that the host PC is not keeping >>>> up with >>>> incoming data >>>> - In any case, you may want to simplify the problem by dropping >>>> from two devices to one device. See if you can eliminate some or all of >>>> these problems when using only 4 channels. >>>> - If you drop to one device, you can use benchmark_rate to test >>>> performance. If you do not use "second_addr", you should be able to get >>>> 4x50 MS/s. If you use "second_addr", you should be able to get 4x100 >>>> MS/s. >>>> - Regarding your needed changes to "rx_samples_to_file", I guess I >>>> was thinking about the latest version which supports multiple channels. >>>> Perhaps UHD 3.12 has a version of this example that only supports a >>>> single >>>> channel. You could compare your version to the latest version (say, UHD >>>> 4.6) to see the improvements to this example. >>>> >>>> >>>> On Mon, Dec 18, 2023 at 7:44 AM Anabel Almodovar < >>>> anabel.almodo...@gmail.com> wrote: >>>> >>>>> Hi Rob, >>>>> >>>>> Thanks for the suggestions. I have tried deleting the LO sharing, and >>>>> nothing changes. Then removing the second addr, and that leads me to get >>>>> 'Ds' in addition to the error already mentioned, as it is not able to >>>>> handle that much information. Although I don't quite understand the >>>>> difference between 'O' and 'D'. >>>>> >>>>> From what I understand the program is set up for a single channel, so >>>>> I need to modify it to get more than one. >>>>> >>>>> Thank you in advance. >>>>> >>>>> Regards, >>>>> Anabel >>>>> >>>>> El vie, 15 dic 2023 a las 15:39, Rob Kossler (<rkoss...@nd.edu>) >>>>> escribió: >>>>> >>>>>> Hmmm. I'm not sure. Here are a few comments: >>>>>> >>>>>> - Try running without "second_addr". I realize that you will >>>>>> need it for full rate (4 x 100MS/s for each X310), but you could run >>>>>> at 50 >>>>>> MS/s without second_addr >>>>>> - Try running without shared LO. I don't think you would need to >>>>>> physically change any sharing cables. >>>>>> - I am curious why you needed to modify the streamer, pointer >>>>>> buffer and file writing. I would expect that this would scale with >>>>>> the >>>>>> number of channels such that you didn't need to modify any code in >>>>>> this area >>>>>> - If you were using a more recent UHD, I would recommend >>>>>> switching to the example benchmark_rate which natively supports >>>>>> external >>>>>> PPS and multiple devices. I noticed that even the most >>>>>> recent rx_samples_to_file does not support external PPS (without >>>>>> modifying >>>>>> the code) >>>>>> - I know you mentioned you were unable to upgrade because of >>>>>> compatibility constraints. But, even if you cannot upgrade the OS, >>>>>> would >>>>>> you be able to install a new UHD - perhaps in a local folder that did >>>>>> not >>>>>> interfere with the system-wide UHD 3.12 installation. I typically >>>>>> have >>>>>> multiple UHD versions on my system and switch among them by >>>>>> "sourcing" a >>>>>> given setup file as needed. >>>>>> >>>>>> >>>>>> On Fri, Dec 15, 2023 at 12:52 AM Anabel Almodovar < >>>>>> anabel.almodo...@gmail.com> wrote: >>>>>> >>>>>>> Dear Rob, >>>>>>> >>>>>>> Yes, I use an external clock configuration. >>>>>>> >>>>>>> *std::this_thread::sleep_for( std::chrono::milliseconds(int64_t(1000 >>>>>>> * setup_time) );* >>>>>>> >>>>>>> >>>>>>> >>>>>>> *usrp->set_clock_source("external", >>>>>>> uhd::usrp::multi_usrp::ALL_MBOARDS); >>>>>>> usrp->set_time_source("external", >>>>>>> uhd::usrp::multi_usrp::ALL_MBOARDS);usrp->set_time_unknown_pps(uhd::time_spec_t(0.0));std::this_thread::sleep_for(std::chrono::seconds(1));* >>>>>>> >>>>>>> I have only modified the code to get 8 channels and LO sharing. I >>>>>>> want to get a continuous acquisition setup without losing samples. But >>>>>>> I am >>>>>>> starting with something basic at the moment. Any suggestions are >>>>>>> welcome. >>>>>>> So I've modified the streamer, the pointer buffer, and added writing >>>>>>> files. >>>>>>> >>>>>>> my line is >>>>>>> *$sudo ./rx_samples_to_file_v1 >>>>>>> --args="addr0=192.168.60.2,second_addr0=192.168.50.2,addr1=192.168.40.2,second_addr1=192.168.30.2,recv_buff_size=900000000" >>>>>>> --subdev="A:0 A:1 B:0 B:1" --channel="0,1,2,3,4,5,6,7" --freq 800e6 >>>>>>> --rate >>>>>>> 25e6 --bw 25e6 --gain 70 * >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> *Anabel* >>>>>>> >>>>>>> >>>>>>> >>>>>>> El jue, 14 dic 2023 a las 18:25, Rob Kossler (<rkoss...@nd.edu>) >>>>>>> escribió: >>>>>>> >>>>>>>> Hi Anabel, >>>>>>>> In my opinion, the error you are experiencing has nothing to do >>>>>>>> with streaming performance settings (such as "performance" governor, >>>>>>>> network descriptors, MTU size, etc). The issue seems to be that the two >>>>>>>> X310 devices do not have synchronized clocks. In addition to the >>>>>>>> physical >>>>>>>> synchronization using OctoClock, you must also configure each device >>>>>>>> to use >>>>>>>> external synchronization and you must tell each device to reset its >>>>>>>> FPGA >>>>>>>> clock count at a common PPS. The typical sequence is: (1) wait for a >>>>>>>> PPS to >>>>>>>> occur (by querying either device), (2) tell each device to reset its >>>>>>>> FPGA >>>>>>>> clock at the subsequent PPS (this step must complete before the next >>>>>>>> PPS >>>>>>>> arrives). >>>>>>>> >>>>>>>> You mentioned that you are using rx_samples_to_file. Did you modify >>>>>>>> it in any way? What is your exact command line with all parameters? >>>>>>>> Rob >>>>>>>> >>>>>>>> On Thu, Dec 14, 2023 at 10:29 AM Anabel Almodovar < >>>>>>>> anabel.almodo...@gmail.com> wrote: >>>>>>>> >>>>>>>>> Dear Rob >>>>>>>>> >>>>>>>>> Thank you for your answer. >>>>>>>>> I make use of the CDA-2990 octoclock as a source of >>>>>>>>> synchronization between both USRPs, in addition to local oscillator >>>>>>>>> sharing. I use dual 10GigEth connections and a MTU of 9000 to connect >>>>>>>>> the >>>>>>>>> USRPs to the PC. >>>>>>>>> >>>>>>>>> Due to various compatibility issues I am unable to upgrade the >>>>>>>>> system. >>>>>>>>> >>>>>>>>> When I work with a sample rate of 10MSps I don't get problems but >>>>>>>>> when I increase to 25MSps I start to get the error. I need them >>>>>>>>> working >>>>>>>>> with 100MSps. I have tried changing the CPU governor to "performance" >>>>>>>>> to >>>>>>>>> get the maximum working frequency (*sudo cpupower frequency-set >>>>>>>>> --governor performance*), as well as changing the number of >>>>>>>>> network interface descriptors to maximum (*sudo ethtool -G >>>>>>>>> <interface> tx <max> rx <max>*), or increasing the dirty memory >>>>>>>>> buffer (*sudo sysctl -w vm.dirty_ratio=80; sudo sysctl -w >>>>>>>>> vm.dirty_background_ratio=50*), but I still get the same problem. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Anabel >>>>>>>>> >>>>>>>>> El jue, 14 dic 2023 a las 15:38, Rob Kossler (<rkoss...@nd.edu>) >>>>>>>>> escribió: >>>>>>>>> >>>>>>>>>> Hi Anabel, >>>>>>>>>> How are you sync-ing the clocks on the two units? Do you have an >>>>>>>>>> external PPS source and are you configuring both devices to use this >>>>>>>>>> external source? >>>>>>>>>> >>>>>>>>>> Finally, do you have the ability to upgrade your OS and your UHD >>>>>>>>>> versions? There aren't many user's that are using UHD 3.12 so if you >>>>>>>>>> have >>>>>>>>>> issues, it will be hard to get support. >>>>>>>>>> Rob >>>>>>>>>> >>>>>>>>>> On Thu, Dec 14, 2023 at 5:20 AM Anabel Almodovar < >>>>>>>>>> anabel.almodo...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Dear all, >>>>>>>>>>> >>>>>>>>>>> I am trying to make an acquisition with two X310 equipped with >>>>>>>>>>> two TwinRx. I am using Ubuntu 16.04 LTS 64-bit and UHD 3.12. My PC >>>>>>>>>>> contains >>>>>>>>>>> 128GB RAM and an Intel® Xeon(R) Silver 4114 CPU @ 2.20GHz × 40 and >>>>>>>>>>> a 4TB >>>>>>>>>>> SSD. I have modified the example rx_samples_to_file.cpp code to get >>>>>>>>>>> 8 >>>>>>>>>>> channels and I get the following error: >>>>>>>>>>> >>>>>>>>>>> *D* >>>>>>>>>>> *[ERROR] [STREAMER] The receive packet handler failed to >>>>>>>>>>> time-align packets. 1002 received packets were processed by the >>>>>>>>>>> handler. >>>>>>>>>>> However, a timestamp match could not be determined.* >>>>>>>>>>> *Timeout while streaming* >>>>>>>>>>> >>>>>>>>>>> *[ERROR] [X300] 192.168.60.2 <http://192.168.60.2>: x300 fw >>>>>>>>>>> communication failure #1* >>>>>>>>>>> *EnvironmentError: IOError: x300 fw poke32 - reply timed out* >>>>>>>>>>> *[ERROR] [UHD] An unexpected exception was caught in a task >>>>>>>>>>> loop.The task loop will now exit, things may not >>>>>>>>>>> work.AssertionError: >>>>>>>>>>> reply.sequence == request.sequence* >>>>>>>>>>> * in virtual void >>>>>>>>>>> x300_ctrl_iface_enet::__poke32(uhd::wb_iface::wb_addr_type, >>>>>>>>>>> uint32_t)* >>>>>>>>>>> * at >>>>>>>>>>> /home/rs3_lab/workarea-uhd/uhd/host/lib/usrp/x300/x300_fw_ctrl.cpp:135* >>>>>>>>>>> >>>>>>>>>>> I don't know how to solve the Timeout problem, I have tried to >>>>>>>>>>> start the acquisition 1.1 sg in time. But it didn't work. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> * stream_cmd.stream_now = false; >>>>>>>>>>> stream_cmd.time_spec = usrp->get_time_last_pps(0)+1.1;* >>>>>>>>>>> >>>>>>>>>>> What is the problem and how can I fix it? >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> Anabel >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> USRP-users mailing list -- usrp-users@lists.ettus.com >>>>>>>>>>> To unsubscribe send an email to usrp-users-le...@lists.ettus.com >>>>>>>>>>> >>>>>>>>>>
_______________________________________________ USRP-users mailing list -- usrp-users@lists.ettus.com To unsubscribe send an email to usrp-users-le...@lists.ettus.com
