Re: [USRP-users] x300 latency over 10GigE
A quick update ...
I added
#include
to my includes and the following code to UHD_SAFE_MAIN:
=
uhd::device3::sptr usrp3 = usrp->get_device3();
uhd::rfnoc::dma_fifo_block_ctrl::sptr dmafifo_block_ctrl =
usrp3->get_block_ctrl(
uhd::rfnoc::block_id_t(0,"DmaFIFO"));
int fifoSize = 4*1024*1024;
int numChannels = usrp->get_tx_num_channels();
for (int chanIdx = 0; chanIdxget_depth(0);
// uint32_t currBaseAddr = dmafifo_block_ctrl->get_base_addr(0);
// std::cerr << "DMA chan " << chanIdx << ": base / depth : " <<
// currBaseAddr << " / " << currDepth << std::endl;
std::cerr << "Attempting to resize channel " << chanIdx <<
std::endl;
dmafifo_block_ctrl->resize( chanIdx*fifoSize, /* base address */
fifoSize, /* depth */
chanIdx );
}
=
I started with 16MB, then 8MB, etc ...
At 4MB, latency is 1/8 of what I see at 32MB as expected ... about 21.33
ms. I'm sure I'll need to tune this a little more once I apply it to my
application, but I can now control it.
I greatly appreciate the help, Brian!
Best,
Doug
On Wed, Mar 10, 2021 at 2:46 PM Doug Blackburn wrote:
> Brian --
>
> Thanks so much! I sprinkled my comments in below :
>
> On Wed, Mar 10, 2021 at 1:42 PM Brian Padalino
> wrote:
>
>> On Wed, Mar 10, 2021 at 12:39 PM Doug Blackburn
>> wrote:
>>
>>> Brian,
>>>
>>> I've seen this using UHD-3.14 and UHD-3.15.LTS.
>>>
>>
>> The DMA FIFO block default size is set here in the source code for
>> UHD-3.15.LTS:
>>
>>
>> https://github.com/EttusResearch/uhd/blob/UHD-3.15.LTS/host/lib/rfnoc/dma_fifo_block_ctrl_impl.cpp#L25
>>
>> And the interface in the header file provides a way to resize it:
>>
>>
>> https://github.com/EttusResearch/uhd/blob/UHD-3.15.LTS/host/include/uhd/rfnoc/dma_fifo_block_ctrl.hpp#L33
>>
>> I'd probably resize it before sending any data to it.
>>
>> That should help with your latency question I think.
>>
>
> This is super helpful. I'll give it a shot and see what happens!
>
>
>>
>>
>>>
>>> I have performed some follow-on testing that raises more questions,
>>> particularly about the usage of end_of_burst and start_of_burst. I talk
>>> through my tests and observations below; the questions that these generated
>>> are at the end ...
>>>
>>> I thought it would be interesting to modify benchmark_rate.cpp to
>>> attempt to place a timestamp on each buffer that was sent out to see if I
>>> could observe the same behavior. I haven't seen thorough explanations of
>>> what exactly the start_of_burst and end_of_burst metadata fields do at a
>>> low level beyond this posting --
>>> http://lists.ettus.com/pipermail/usrp-users_lists.ettus.com/2016-November/050555.html
>>> and a note about start_of_burst resetting the CORDICs (I'd appreciate being
>>> pointed in the right direction if I've missed it, thank you!) -- so I
>>> wanted to test the effect on timing when has_time_spec is true and the SOB
>>> and EOB fields are either false or true. I initially set my test up in the
>>> following way (I hope the pseudocode makes sense) to make observations
>>> easy. I watched for the LO on a spectrum analyzer. Per the code below, I
>>> would expect a burst every 2 seconds if the time_spec was followed ...
>>>
>>> ==
>>> max_samps_per_packet = 50e5; // 100ms at 50 MSPS
>>> start_of_burst =
>>> end_of_burst =
>>> has_time_spec = true;
>>> while( not burst_timer_elapsed)
>>> {
>>> tx_stream->send();
>>> start_of_burst =
>>> end_of_burst =
>>> time_spec = time_spec + 2.0;
>>> }
>>>
>>
>> A few things. I'd expect a burst every 2 seconds if you set sob = true,
>> eob = true outside the loop, and never change it and only change the
>> time_spec for every send. Does that not work for you?
>>
>>
> Yes -- that does work, too. I tried all the different combinations ... So
> for example, if sob/eob were true/true outside the loop and false/false
> inside the loop, I'd see a two second pause after the first burst and then
> we'd roll through the rest of them contiguously.
>
>
>> Next, The sizing of packets can be really important here. The way the
>> DUC works is a little unintuitive. The DUC works by creating N packets
>> from 1 input packet. To this end, if you have an extra 1 sample, it will
>> repeat that small 1 sample packet N times - very processing inefficient.
>>
>> Furthermore, when I tried doing this I would run into weird edge cases
>> with the DMA FIFO where the send() call would block indefinitely. My
>> workaround was to manually zero stuff and keep the transmit FIFO constantly
>> going - not using any eob flags at all. My system would actually use a
>> software FIFO for bursts that wanted to go out, and I had a software thread
>> in a tight loop that would check if the FIFO had anything in it. If it
>> didn't, it would zero stuff some small amount of transmit samples (1 packet
Re: [USRP-users] x300 latency over 10GigE
Brian --
Thanks so much! I sprinkled my comments in below :
On Wed, Mar 10, 2021 at 1:42 PM Brian Padalino wrote:
> On Wed, Mar 10, 2021 at 12:39 PM Doug Blackburn wrote:
>
>> Brian,
>>
>> I've seen this using UHD-3.14 and UHD-3.15.LTS.
>>
>
> The DMA FIFO block default size is set here in the source code for
> UHD-3.15.LTS:
>
>
> https://github.com/EttusResearch/uhd/blob/UHD-3.15.LTS/host/lib/rfnoc/dma_fifo_block_ctrl_impl.cpp#L25
>
> And the interface in the header file provides a way to resize it:
>
>
> https://github.com/EttusResearch/uhd/blob/UHD-3.15.LTS/host/include/uhd/rfnoc/dma_fifo_block_ctrl.hpp#L33
>
> I'd probably resize it before sending any data to it.
>
> That should help with your latency question I think.
>
This is super helpful. I'll give it a shot and see what happens!
>
>
>>
>> I have performed some follow-on testing that raises more questions,
>> particularly about the usage of end_of_burst and start_of_burst. I talk
>> through my tests and observations below; the questions that these generated
>> are at the end ...
>>
>> I thought it would be interesting to modify benchmark_rate.cpp to attempt
>> to place a timestamp on each buffer that was sent out to see if I could
>> observe the same behavior. I haven't seen thorough explanations of what
>> exactly the start_of_burst and end_of_burst metadata fields do at a low
>> level beyond this posting --
>> http://lists.ettus.com/pipermail/usrp-users_lists.ettus.com/2016-November/050555.html
>> and a note about start_of_burst resetting the CORDICs (I'd appreciate being
>> pointed in the right direction if I've missed it, thank you!) -- so I
>> wanted to test the effect on timing when has_time_spec is true and the SOB
>> and EOB fields are either false or true. I initially set my test up in the
>> following way (I hope the pseudocode makes sense) to make observations
>> easy. I watched for the LO on a spectrum analyzer. Per the code below, I
>> would expect a burst every 2 seconds if the time_spec was followed ...
>>
>> ==
>> max_samps_per_packet = 50e5; // 100ms at 50 MSPS
>> start_of_burst =
>> end_of_burst =
>> has_time_spec = true;
>> while( not burst_timer_elapsed)
>> {
>> tx_stream->send();
>> start_of_burst =
>> end_of_burst =
>> time_spec = time_spec + 2.0;
>> }
>>
>
> A few things. I'd expect a burst every 2 seconds if you set sob = true,
> eob = true outside the loop, and never change it and only change the
> time_spec for every send. Does that not work for you?
>
>
Yes -- that does work, too. I tried all the different combinations ... So
for example, if sob/eob were true/true outside the loop and false/false
inside the loop, I'd see a two second pause after the first burst and then
we'd roll through the rest of them contiguously.
> Next, The sizing of packets can be really important here. The way the DUC
> works is a little unintuitive. The DUC works by creating N packets from 1
> input packet. To this end, if you have an extra 1 sample, it will repeat
> that small 1 sample packet N times - very processing inefficient.
>
> Furthermore, when I tried doing this I would run into weird edge cases
> with the DMA FIFO where the send() call would block indefinitely. My
> workaround was to manually zero stuff and keep the transmit FIFO constantly
> going - not using any eob flags at all. My system would actually use a
> software FIFO for bursts that wanted to go out, and I had a software thread
> in a tight loop that would check if the FIFO had anything in it. If it
> didn't, it would zero stuff some small amount of transmit samples (1 packet
> I believe). If it did, it would send the burst. You may want to do
> something similar even with a synchronized system and counting outgoing
> samples.
>
:) This is what led me here; the application I was working on essentially
did that. I'd have some data I'd want to send at a specific time. I'd
translate that time to a number of buffers past the start of my transmit
(with some extra bookkeeping and buffer magic to align samples, etc), and
found that I could only get this to work if my requested transmit time was
at least 167 ms in the future. This didn't quite reconcile with the 1ms
of latency I could demonstrate with 'latency_test' -- which uses a single
packet -- hence my trip down the rabbit hole. If I can lower that number a
little by modifying the FIFO block, I think I'll be happy, but ...
>
>
>>
>> My observations were as follows: if end_of_burst for the prior burst was
>> set to true, my code adhered to the time_spec. The value of start_of_burst
>> had no effect on whether or not the expected timing was followed. If
>> end_of_burst was set to false, the time_spec for the following burst is
>> ignored and the packet is transmitted as soon as possible.
>>
>> I then followed this up with another test -- I replaced
>> time_spec = time_spec + 2.0;
>> with the equivalent of
>> time_spec = time_spec + 0.100;
>>
>> And
Re: [USRP-users] x300 latency over 10GigE
On Wed, Mar 10, 2021 at 12:39 PM Doug Blackburn wrote:
> Brian,
>
> I've seen this using UHD-3.14 and UHD-3.15.LTS.
>
The DMA FIFO block default size is set here in the source code for
UHD-3.15.LTS:
https://github.com/EttusResearch/uhd/blob/UHD-3.15.LTS/host/lib/rfnoc/dma_fifo_block_ctrl_impl.cpp#L25
And the interface in the header file provides a way to resize it:
https://github.com/EttusResearch/uhd/blob/UHD-3.15.LTS/host/include/uhd/rfnoc/dma_fifo_block_ctrl.hpp#L33
I'd probably resize it before sending any data to it.
That should help with your latency question I think.
>
> I have performed some follow-on testing that raises more questions,
> particularly about the usage of end_of_burst and start_of_burst. I talk
> through my tests and observations below; the questions that these generated
> are at the end ...
>
> I thought it would be interesting to modify benchmark_rate.cpp to attempt
> to place a timestamp on each buffer that was sent out to see if I could
> observe the same behavior. I haven't seen thorough explanations of what
> exactly the start_of_burst and end_of_burst metadata fields do at a low
> level beyond this posting --
> http://lists.ettus.com/pipermail/usrp-users_lists.ettus.com/2016-November/050555.html
> and a note about start_of_burst resetting the CORDICs (I'd appreciate being
> pointed in the right direction if I've missed it, thank you!) -- so I
> wanted to test the effect on timing when has_time_spec is true and the SOB
> and EOB fields are either false or true. I initially set my test up in the
> following way (I hope the pseudocode makes sense) to make observations
> easy. I watched for the LO on a spectrum analyzer. Per the code below, I
> would expect a burst every 2 seconds if the time_spec was followed ...
>
> ==
> max_samps_per_packet = 50e5; // 100ms at 50 MSPS
> start_of_burst =
> end_of_burst =
> has_time_spec = true;
> while( not burst_timer_elapsed)
> {
> tx_stream->send();
> start_of_burst =
> end_of_burst =
> time_spec = time_spec + 2.0;
> }
>
A few things. I'd expect a burst every 2 seconds if you set sob = true,
eob = true outside the loop, and never change it and only change the
time_spec for every send. Does that not work for you?
Next, The sizing of packets can be really important here. The way the DUC
works is a little unintuitive. The DUC works by creating N packets from 1
input packet. To this end, if you have an extra 1 sample, it will repeat
that small 1 sample packet N times - very processing inefficient.
Furthermore, when I tried doing this I would run into weird edge cases with
the DMA FIFO where the send() call would block indefinitely. My workaround
was to manually zero stuff and keep the transmit FIFO constantly going -
not using any eob flags at all. My system would actually use a software
FIFO for bursts that wanted to go out, and I had a software thread in a
tight loop that would check if the FIFO had anything in it. If it didn't,
it would zero stuff some small amount of transmit samples (1 packet I
believe). If it did, it would send the burst. You may want to do
something similar even with a synchronized system and counting outgoing
samples.
>
> My observations were as follows: if end_of_burst for the prior burst was
> set to true, my code adhered to the time_spec. The value of start_of_burst
> had no effect on whether or not the expected timing was followed. If
> end_of_burst was set to false, the time_spec for the following burst is
> ignored and the packet is transmitted as soon as possible.
>
> I then followed this up with another test -- I replaced
> time_spec = time_spec + 2.0;
> with the equivalent of
> time_spec = time_spec + 0.100;
>
> And set end_of_burst and start_of_burst to true.
>
> I figured if I can run this continuously by setting has_time_spec to
> 'false' after the first burst and easily push data into the FIFO buffer,
> that doing this should not be a problem ... but I'm presented with a stream
> of lates and no actual transmission.
>
> I understand that 100ms is not an integer multiple of packet size returned
> by get_max_num_samps() -- so I tried an integer multiple of the packet
> size, too, with an appropriately updated time_spec. This also resulted with
> a lates through the entire transmit.
>
> So here are my additional questions:
>
> Is the only effect of "start_of_burst = true" to cause the CORDICs to
> reset?
> What is end_of_burst doing to enable a following time_spec to be used?
> What additional work is being performed when I set end_of_burst and
> has_time_spec to 'true' such that I get latest throughout the entire
> attempted transmission?
>
I don't know the answer to these questions. Try the suggestions above and
see if they help you out or not.
Good luck!
Brian
>
___
USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com
Re: [USRP-users] x300 latency over 10GigE
Brian,
I've seen this using UHD-3.14 and UHD-3.15.LTS.
I have performed some follow-on testing that raises more questions,
particularly about the usage of end_of_burst and start_of_burst. I talk
through my tests and observations below; the questions that these generated
are at the end ...
I thought it would be interesting to modify benchmark_rate.cpp to attempt
to place a timestamp on each buffer that was sent out to see if I could
observe the same behavior. I haven't seen thorough explanations of what
exactly the start_of_burst and end_of_burst metadata fields do at a low
level beyond this posting --
http://lists.ettus.com/pipermail/usrp-users_lists.ettus.com/2016-November/050555.html
and a note about start_of_burst resetting the CORDICs (I'd appreciate being
pointed in the right direction if I've missed it, thank you!) -- so I
wanted to test the effect on timing when has_time_spec is true and the SOB
and EOB fields are either false or true. I initially set my test up in the
following way (I hope the pseudocode makes sense) to make observations
easy. I watched for the LO on a spectrum analyzer. Per the code below, I
would expect a burst every 2 seconds if the time_spec was followed ...
==
max_samps_per_packet = 50e5; // 100ms at 50 MSPS
start_of_burst =
end_of_burst =
has_time_spec = true;
while( not burst_timer_elapsed)
{
tx_stream->send();
start_of_burst =
end_of_burst =
time_spec = time_spec + 2.0;
}
My observations were as follows: if end_of_burst for the prior burst was
set to true, my code adhered to the time_spec. The value of start_of_burst
had no effect on whether or not the expected timing was followed. If
end_of_burst was set to false, the time_spec for the following burst is
ignored and the packet is transmitted as soon as possible.
I then followed this up with another test -- I replaced
time_spec = time_spec + 2.0;
with the equivalent of
time_spec = time_spec + 0.100;
And set end_of_burst and start_of_burst to true.
I figured if I can run this continuously by setting has_time_spec to
'false' after the first burst and easily push data into the FIFO buffer,
that doing this should not be a problem ... but I'm presented with a stream
of lates and no actual transmission.
I understand that 100ms is not an integer multiple of packet size returned
by get_max_num_samps() -- so I tried an integer multiple of the packet
size, too, with an appropriately updated time_spec. This also resulted with
a lates through the entire transmit.
So here are my additional questions:
Is the only effect of "start_of_burst = true" to cause the CORDICs to
reset?
What is end_of_burst doing to enable a following time_spec to be used?
What additional work is being performed when I set end_of_burst and
has_time_spec to 'true' such that I get latest throughout the entire
attempted transmission?
Best Regards,
Doug
On Tue, Mar 9, 2021 at 11:51 PM Brian Padalino wrote:
> On Tue, Mar 9, 2021 at 10:03 PM Doug Blackburn via USRP-users <
> usrp-users@lists.ettus.com> wrote:
>
>> Hello --
>>
>> I've got some questions re: latency with the x300 over the 10GigE
>> interface.
>>
>> If I use the latency_test example operating at a rate of 50 MSPS, I have
>> no issues with a latency of 1ms. The latency test receives data, examines
>> the time stamp, and transmits a single packet.
>>
>> I have an application where I'd like to run the transmitter continuously,
>> and I got curious about the latency involved in that operation. My
>> application is similar to the benchmark_rate example. I added the
>> following lines to the benchmark_rate example at line 256 after the line.
>>
>> md.has_time_spec = false;
>>
>>
>> if ( (num_tx_samps % 5000) < 4*max_samps_per_packet )
>> {
>> uhd::time_spec_t expectedTime = startTime + (double) ( num_tx_samps )
>> / (double)usrp->get_tx_rate();
>> uhd::time_spec_t timeAtLog = usrp->get_time_now();
>> timeAtLog = usrp->get_time_now();
>> std::cerr << " Actual time " << std::endl;
>> std::cerr << " " << timeAtLog.get_full_secs() << " / "
>> << timeAtLog.get_frac_secs() << std::endl;
>> std::cerr << " Expected time " << std::endl;
>> std::cerr << " " << expectedTime.get_full_secs() << " / "
>> << expectedTime.get_frac_secs() << std::endl;
>> }
>>
>>
>> The intent of this insertion is to log the time at which we return from
>> tx_stream->send() and the time at which the first sample of that sent data
>> should be transmitted -- at approximately once per second when running at
>> 50 MSPS.
>>
>> After the first second, I consistently saw the following results:
>>
>> Actual time
>> 1 / 0.10517
>> Expected time
>> 1 / 0.27253
>>
>> Actual time
>> 1 / 0.105419
>> Expected time
>> 1 / 0.27255
>>
>> Which indicates to me that there is a latency of
Re: [USRP-users] x300 latency over 10GigE
On Tue, Mar 9, 2021 at 10:03 PM Doug Blackburn via USRP-users <
usrp-users@lists.ettus.com> wrote:
> Hello --
>
> I've got some questions re: latency with the x300 over the 10GigE
> interface.
>
> If I use the latency_test example operating at a rate of 50 MSPS, I have
> no issues with a latency of 1ms. The latency test receives data, examines
> the time stamp, and transmits a single packet.
>
> I have an application where I'd like to run the transmitter continuously,
> and I got curious about the latency involved in that operation. My
> application is similar to the benchmark_rate example. I added the
> following lines to the benchmark_rate example at line 256 after the line.
>
> md.has_time_spec = false;
>
>
> if ( (num_tx_samps % 5000) < 4*max_samps_per_packet )
> {
> uhd::time_spec_t expectedTime = startTime + (double) ( num_tx_samps )
> / (double)usrp->get_tx_rate();
> uhd::time_spec_t timeAtLog = usrp->get_time_now();
> timeAtLog = usrp->get_time_now();
> std::cerr << " Actual time " << std::endl;
> std::cerr << " " << timeAtLog.get_full_secs() << " / "
> << timeAtLog.get_frac_secs() << std::endl;
> std::cerr << " Expected time " << std::endl;
> std::cerr << " " << expectedTime.get_full_secs() << " / "
> << expectedTime.get_frac_secs() << std::endl;
> }
>
>
> The intent of this insertion is to log the time at which we return from
> tx_stream->send() and the time at which the first sample of that sent data
> should be transmitted -- at approximately once per second when running at
> 50 MSPS.
>
> After the first second, I consistently saw the following results:
>
> Actual time
> 1 / 0.10517
> Expected time
> 1 / 0.27253
>
> Actual time
> 1 / 0.105419
> Expected time
> 1 / 0.27255
>
> Which indicates to me that there is a latency of approximately 167ms when
> transmitting data. That is, the send() function is returning 167ms earlier
> than I expect the data to actually be transmitted. If I halve the sample
> rate to 25 MSPS, the latency doubles.
>
> What is the source of the latency when running in a continuous mode?
> Initially, I had thought that this might be related to the
> send_buffer_size, but making changes to send_buffer_size seem to not have
> an effect. FWIW, 167ms at 50 MSPS is suspiciously close to the value for
> wmem_max (33554432) suggested in the x300 system configuration ... but
> neither changing that value or send_buffer_size seems to make a difference.
>
> Is this latency tunable?
>
I suspect it's the DMA FIFO which uses the DRAM in the X310 as a buffer.
The default buffer size is 32MB.
Which version of UHD are you using?
Brian
___
USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com
Re: [USRP-users] x300 latency over 10GigE
Hah. Yes. Sorry for the confusion.
Sent from my iPhone
> On Mar 9, 2021, at 10:36 PM, Doug Blackburn wrote:
>
>
> Marcus,
>
> Thanks for your response; I believe I am. The math is spread over two lines:
>
> uhd::time_spec_t expectedTime = startTime + (double) ( num_tx_samps )
> / (double)usrp->get_tx_rate();
>
> Best,
> Doug
>
>
>> On Tue, Mar 9, 2021 at 10:20 PM Marcus D Leech
>> wrote:
>> Shouldn’t you be scaling your num_tx_samples by the time per sample when
>> calculating the expectedTime?
>>
>> Sent from my iPhone
>>
On Mar 9, 2021, at 10:03 PM, Doug Blackburn via USRP-users
wrote:
>>>
>>> Hello --
>>>
>>> I've got some questions re: latency with the x300 over the 10GigE
>>> interface.
>>>
>>> If I use the latency_test example operating at a rate of 50 MSPS, I have no
>>> issues with a latency of 1ms. The latency test receives data, examines the
>>> time stamp, and transmits a single packet.
>>>
>>> I have an application where I'd like to run the transmitter continuously,
>>> and I got curious about the latency involved in that operation. My
>>> application is similar to the benchmark_rate example. I added the
>>> following lines to the benchmark_rate example at line 256 after the line.
>>>
>>> md.has_time_spec = false;
>>>
>>>
>>> if ( (num_tx_samps % 5000) < 4*max_samps_per_packet )
>>> {
>>> uhd::time_spec_t expectedTime = startTime + (double) ( num_tx_samps )
>>> / (double)usrp->get_tx_rate();
>>> uhd::time_spec_t timeAtLog = usrp->get_time_now();
>>> timeAtLog = usrp->get_time_now();
>>> std::cerr << " Actual time " << std::endl;
>>> std::cerr << " " << timeAtLog.get_full_secs() << " / "
>>> << timeAtLog.get_frac_secs() << std::endl;
>>> std::cerr << " Expected time " << std::endl;
>>> std::cerr << " " << expectedTime.get_full_secs() << " / "
>>> << expectedTime.get_frac_secs() << std::endl;
>>> }
>>>
>>>
>>> The intent of this insertion is to log the time at which we return from
>>> tx_stream->send() and the time at which the first sample of that sent data
>>> should be transmitted -- at approximately once per second when running at
>>> 50 MSPS.
>>>
>>> After the first second, I consistently saw the following results:
>>>
>>> Actual time
>>> 1 / 0.10517
>>> Expected time
>>> 1 / 0.27253
>>>
>>> Actual time
>>> 1 / 0.105419
>>> Expected time
>>> 1 / 0.27255
>>>
>>> Which indicates to me that there is a latency of approximately 167ms when
>>> transmitting data. That is, the send() function is returning 167ms earlier
>>> than I expect the data to actually be transmitted. If I halve the sample
>>> rate to 25 MSPS, the latency doubles.
>>>
>>> What is the source of the latency when running in a continuous mode?
>>> Initially, I had thought that this might be related to the
>>> send_buffer_size, but making changes to send_buffer_size seem to not have
>>> an effect. FWIW, 167ms at 50 MSPS is suspiciously close to the value for
>>> wmem_max (33554432) suggested in the x300 system configuration ... but
>>> neither changing that value or send_buffer_size seems to make a difference.
>>>
>>> Is this latency tunable?
>>>
>>> Thank you for your help!
>>>
>>> Best Regards,
>>> Doug Blackburn
>>>
>>> ___
>>> USRP-users mailing list
>>> USRP-users@lists.ettus.com
>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com
___
USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com
Re: [USRP-users] x300 latency over 10GigE
Marcus,
Thanks for your response; I believe I am. The math is spread over two
lines:
uhd::time_spec_t expectedTime = startTime + (double) ( num_tx_samps )
/ (double)usrp->get_tx_rate();
Best,
Doug
On Tue, Mar 9, 2021 at 10:20 PM Marcus D Leech
wrote:
> Shouldn’t you be scaling your num_tx_samples by the time per sample when
> calculating the expectedTime?
>
> Sent from my iPhone
>
> On Mar 9, 2021, at 10:03 PM, Doug Blackburn via USRP-users <
> usrp-users@lists.ettus.com> wrote:
>
>
> Hello --
>
> I've got some questions re: latency with the x300 over the 10GigE
> interface.
>
> If I use the latency_test example operating at a rate of 50 MSPS, I have
> no issues with a latency of 1ms. The latency test receives data, examines
> the time stamp, and transmits a single packet.
>
> I have an application where I'd like to run the transmitter continuously,
> and I got curious about the latency involved in that operation. My
> application is similar to the benchmark_rate example. I added the
> following lines to the benchmark_rate example at line 256 after the line.
>
> md.has_time_spec = false;
>
>
> if ( (num_tx_samps % 5000) < 4*max_samps_per_packet )
> {
> uhd::time_spec_t expectedTime = startTime + (double) ( num_tx_samps )
> / (double)usrp->get_tx_rate();
> uhd::time_spec_t timeAtLog = usrp->get_time_now();
> timeAtLog = usrp->get_time_now();
> std::cerr << " Actual time " << std::endl;
> std::cerr << " " << timeAtLog.get_full_secs() << " / "
> << timeAtLog.get_frac_secs() << std::endl;
> std::cerr << " Expected time " << std::endl;
> std::cerr << " " << expectedTime.get_full_secs() << " / "
> << expectedTime.get_frac_secs() << std::endl;
> }
>
>
> The intent of this insertion is to log the time at which we return from
> tx_stream->send() and the time at which the first sample of that sent data
> should be transmitted -- at approximately once per second when running at
> 50 MSPS.
>
> After the first second, I consistently saw the following results:
>
> Actual time
> 1 / 0.10517
> Expected time
> 1 / 0.27253
>
> Actual time
> 1 / 0.105419
> Expected time
> 1 / 0.27255
>
> Which indicates to me that there is a latency of approximately 167ms when
> transmitting data. That is, the send() function is returning 167ms earlier
> than I expect the data to actually be transmitted. If I halve the sample
> rate to 25 MSPS, the latency doubles.
>
> What is the source of the latency when running in a continuous mode?
> Initially, I had thought that this might be related to the
> send_buffer_size, but making changes to send_buffer_size seem to not have
> an effect. FWIW, 167ms at 50 MSPS is suspiciously close to the value for
> wmem_max (33554432) suggested in the x300 system configuration ... but
> neither changing that value or send_buffer_size seems to make a difference.
>
> Is this latency tunable?
>
> Thank you for your help!
>
> Best Regards,
> Doug Blackburn
>
> ___
> USRP-users mailing list
> USRP-users@lists.ettus.com
> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com
>
>
___
USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com
Re: [USRP-users] x300 latency over 10GigE
Shouldn’t you be scaling your num_tx_samples by the time per sample when
calculating the expectedTime?
Sent from my iPhone
> On Mar 9, 2021, at 10:03 PM, Doug Blackburn via USRP-users
> wrote:
>
>
> Hello --
>
> I've got some questions re: latency with the x300 over the 10GigE interface.
>
> If I use the latency_test example operating at a rate of 50 MSPS, I have no
> issues with a latency of 1ms. The latency test receives data, examines the
> time stamp, and transmits a single packet.
>
> I have an application where I'd like to run the transmitter continuously, and
> I got curious about the latency involved in that operation. My application
> is similar to the benchmark_rate example. I added the following lines to the
> benchmark_rate example at line 256 after the line.
>
> md.has_time_spec = false;
>
>
> if ( (num_tx_samps % 5000) < 4*max_samps_per_packet )
> {
> uhd::time_spec_t expectedTime = startTime + (double) ( num_tx_samps )
> / (double)usrp->get_tx_rate();
> uhd::time_spec_t timeAtLog = usrp->get_time_now();
> timeAtLog = usrp->get_time_now();
> std::cerr << " Actual time " << std::endl;
> std::cerr << " " << timeAtLog.get_full_secs() << " / "
> << timeAtLog.get_frac_secs() << std::endl;
> std::cerr << " Expected time " << std::endl;
> std::cerr << " " << expectedTime.get_full_secs() << " / "
> << expectedTime.get_frac_secs() << std::endl;
> }
>
>
> The intent of this insertion is to log the time at which we return from
> tx_stream->send() and the time at which the first sample of that sent data
> should be transmitted -- at approximately once per second when running at 50
> MSPS.
>
> After the first second, I consistently saw the following results:
>
> Actual time
> 1 / 0.10517
> Expected time
> 1 / 0.27253
>
> Actual time
> 1 / 0.105419
> Expected time
> 1 / 0.27255
>
> Which indicates to me that there is a latency of approximately 167ms when
> transmitting data. That is, the send() function is returning 167ms earlier
> than I expect the data to actually be transmitted. If I halve the sample
> rate to 25 MSPS, the latency doubles.
>
> What is the source of the latency when running in a continuous mode?
> Initially, I had thought that this might be related to the send_buffer_size,
> but making changes to send_buffer_size seem to not have an effect. FWIW,
> 167ms at 50 MSPS is suspiciously close to the value for wmem_max (33554432)
> suggested in the x300 system configuration ... but neither changing that
> value or send_buffer_size seems to make a difference.
>
> Is this latency tunable?
>
> Thank you for your help!
>
> Best Regards,
> Doug Blackburn
>
> ___
> USRP-users mailing list
> USRP-users@lists.ettus.com
> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com
___
USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com
