Rob,
I think I found the source of the IQ swap issue you observed. This fixes it
for me:
diff --git a/host/lib/rfnoc/mgmt_portal.cpp b/host/lib/rfnoc/mgmt_portal.cpp
index a54efaaf7..7b09c540b 100644
--- a/host/lib/rfnoc/mgmt_portal.cpp
+++ b/host/lib/rfnoc/mgmt_portal.cpp
@@ -617,6 +617,24 @@ public:
_send_recv_mgmt_transaction(xport, cfg_xact);
}
+ // Build a management transaction to configure the destination node
+ {
+ mgmt_payload cfg_xact;
+ cfg_xact.set_header(my_epid, _protover, _chdr_w);
+ _traverse_to_node(cfg_xact, dst_node_addr);
+ mgmt_hop_t cfg_hop;
+ // Configure buffer types
+ cfg_hop.add_op(mgmt_op_t(mgmt_op_t::MGMT_OP_CFG_WR_REQ,
+ mgmt_op_t::cfg_payload(REG_ISTRM_CTRL_STATUS,
+ BUILD_CTRL_STATUS_WORD(false, false, BUFF_U64,
BUFF_U64, false))));
+ // Return the packet back to us
+ cfg_hop.add_op(mgmt_op_t(mgmt_op_t::MGMT_OP_RETURN));
+ // Send the transaction and receive a response.
+ // We don't care about the contents of the response.
+ cfg_xact.add_hop(cfg_hop);
+ _send_recv_mgmt_transaction(xport, cfg_xact);
+ }
+
// Wait for stream configuration to finish on the HW side
_validate_stream_setup(xport, src_node_addr, timeout);
Thanks,
Wade
On Mon, Dec 14, 2020 at 8:32 PM Wade Fife <wade.f...@ettus.com> wrote:
> Hi Rob,
>
> I was able to reproduce the issue, so I'll see if I can get to the bottom
> of it. I'm glad you're able to work around it for now. Thanks for letting
> us know about it!
>
> Wade
>
> On Mon, Dec 14, 2020 at 8:59 AM Rob Kossler <rkoss...@nd.edu> wrote:
>
>> Hi Wade,
>> I tried the command to re-load the FPGA, but I couldn't communicate with
>> UHD nicely until I also added the command "systemctl restart ursp-uhd". I
>> am now able to reset the N310 to proper behavior when it gets in this state.
>>
>> Regarding this issue of real/imag swapping, do you want me to create a
>> support request through supp...@ettus.com? Also, do you need me to
>> provide any more info or an example program / FPGA image? Note that now
>> that I know how to duplicate the issue, I believe I also know how to avoid
>> it. So, it is probably not a critical issue for me presently.
>>
>> Rob
>>
>> On Sat, Dec 12, 2020 at 9:47 AM Wade Fife <wade.f...@ettus.com> wrote:
>>
>>> Thanks for the updates. If you just want to reload the FPGA, try running
>>> "overlay rm n310 && overlay add n310" on the N310. This should simply
>>> reload the FPGA using the bistream already in the flash.
>>>
>>> Wade
>>>
>>> On Fri, Dec 11, 2020 at 6:45 PM Rob Kossler <rkoss...@nd.edu> wrote:
>>>
>>>> Wade,
>>>> The following also fails using just 2 blocks and 2 attempts:
>>>> host_tx => Switchboard#0 => Switchboard#1 => host_rx // SUCCESS
>>>> host_tx => Switchboard#1 => Switchboard#0 => host_rx // FAILURE (RX
>>>> samples equal swapped I/Q of TX samples)
>>>>
>>>> In addition to wanting to get this issue fixed, I also have a question
>>>> about the best way to "reboot" the N310 which is what I need to do to fix
>>>> the issue after it occurs. One way is to give the "reboot now" linux
>>>> command. But, this takes a long time. Another way is to reprogram the
>>>> FPGA image via uhd_image_loader. This is appreciably faster, but I'm
>>>> thinking that this is not a great idea if the flash memory has a limited
>>>> number of write cycles before failure. Is there another way to achieve a
>>>> reboot other than these two?
>>>> Rob
>>>>
>>>> On Fri, Dec 11, 2020 at 7:34 PM Rob Kossler <rkoss...@nd.edu> wrote:
>>>>
>>>>> AHA! I duplicated the issue with the Switchboard image. The way to
>>>>> duplicate the issue is the run the following series of graphs:
>>>>> host_tx => Switchboard#0 => Switchboard#1 => host_rx // SUCCESS
>>>>> host_tx => Switchboard#2 => Switchboard#3 => host_rx // SUCCESS
>>>>> host_tx => Switchboard#0 => Switchboard#2 => host_rx // FAILURE (RX
>>>>> samples equal swapped I/Q of TX samples)
>>>>> Each of these 3 runs consists of running my application (similar to
>>>>> one of the examples such as rfnoc_rx_to_file) such that the UHD object is
>>>>> re-created each time. My guess is that you could use gnuradio to do it
>>>>> instead.
>>>>>
>>>>> My working theory is that the problem is caused by the fact that the
>>>>> Switchboard#2 input port was changed from being connected to the host in
>>>>> test case 2 to then be connected to another RFNoC block in test case 3.
>>>>> Or, I suppose it could be the output port on this block (same logic).
>>>>>
>>>>> If you want me to send you my FPGA image with the 4 Switchboard
>>>>> blocks, let me know.
>>>>> Rob
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Dec 11, 2020 at 7:09 PM Rob Kossler <rkoss...@nd.edu> wrote:
>>>>>
>>>>>> Hi Wade,
>>>>>> After thinking about it a bit, I would ignore the "negation" issue
>>>>>> for now. This may be a byproduct of I/Q swapping related to my custom
>>>>>> block.
>>>>>> Rob
>>>>>>
>>>>>> On Fri, Dec 11, 2020 at 6:34 PM Rob Kossler <rkoss...@nd.edu> wrote:
>>>>>>
>>>>>>> Hi Wade,
>>>>>>> Thanks for the info. I still do not know what's going on, but here
>>>>>>> are a few updates:
>>>>>>>
>>>>>>> - I built a new N310 image with 4 switchboards (1x1) and 1
>>>>>>> splitstream (1x2) blocks in addition to the default blocks. All of
>>>>>>> the
>>>>>>> additional blocks are connected to SEPs for dynamic linking. I
>>>>>>> tried my
>>>>>>> best to get bad behavior using a chain of 1 to 4 switchboards, but I
>>>>>>> could
>>>>>>> not duplicate any I/Q swapping or negation issues.
>>>>>>> - I went back to my custom image (that I described below) and
>>>>>>> found different behavior today (sometimes things worked OK). So,
>>>>>>> this got
>>>>>>> me to thinking that I am somehow getting the FPGA in a bad state
>>>>>>> such that
>>>>>>> a reboot (or FPGA re-flashing) fixes things. I have had some
>>>>>>> success with
>>>>>>> re-booting and things working properly. But, I still do not have a
>>>>>>> true
>>>>>>> handle on things and cannot adequately predict when things are going
>>>>>>> to
>>>>>>> succeed or fail.
>>>>>>> - One thing that has been constant is that I have never seen bad
>>>>>>> behavior when I only have 1 block in my graph (no matter which block
>>>>>>> I
>>>>>>> choose). Note that for all of my tests, the graph looks like this:
>>>>>>> host_tx
>>>>>>> => block_chain => host_rx, where block_chain is a sequential chain
>>>>>>> of 1 or
>>>>>>> more rfnoc blocks.
>>>>>>>
>>>>>>> I will send a follow up once I figure things out.
>>>>>>> Rob
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Dec 11, 2020 at 6:22 PM Wade Fife <wade.f...@ettus.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Rob,
>>>>>>>>
>>>>>>>> The SEPs do have the ability to swap I and Q. This is because on
>>>>>>>> the host computer, I is usually in the lower bits and Q is in the upper
>>>>>>>> bits of each 32-bit word, but in RFNoC, for historical reasons, I goes
>>>>>>>> in
>>>>>>>> the upper bits and Q in the lower bits. The software programs the IQ
>>>>>>>> swapping when it sets up the graph.
>>>>>>>>
>>>>>>>> I assume you're using dynamic connections (through the crossbar) to
>>>>>>>> control the number of FFTs the data is passed through, and not static
>>>>>>>> connections? If that's the case then I wonder if software configures IQ
>>>>>>>> swapping incorrectly in some configurations. I'll see if I can do some
>>>>>>>> testing next week to confirm if it's working correctly.
>>>>>>>>
>>>>>>>> As for negation, I'm not aware of anywhere we do that off the top
>>>>>>>> of my head. Is that behavior block dependent? I'll see if I can find
>>>>>>>> anywhere this happens.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> Wade
>>>>>>>>
>>>>>>>> On Thu, Dec 10, 2020 at 3:54 PM Rob Kossler via USRP-users <
>>>>>>>> usrp-users@lists.ettus.com> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>> I am encountering very strange behavior with a custom FPGA image
>>>>>>>>> (N310). It appears that data streaming through SEPs can get swapped
>>>>>>>>> (I/Q)
>>>>>>>>> and/or negated. Is anyone at Ettus aware of anything that could cause
>>>>>>>>> this? Of course, the issue might be on my end, but I can't think of
>>>>>>>>> what
>>>>>>>>> it might be given that all of my custom blocks work as expected in
>>>>>>>>> isolation (if the block is the only block in graph).
>>>>>>>>>
>>>>>>>>> My custom image is the following:
>>>>>>>>>
>>>>>>>>> - default blocks of Radios, DDCs, DUCs (each 2x2 and
>>>>>>>>> statically connected as in default image)
>>>>>>>>> - custom blocks of two 1x1 windowed-fft blocks, two 1x1
>>>>>>>>> vector-avg blocks, and one 2x2 custom block. Note: each of these
>>>>>>>>> blocks is
>>>>>>>>> connected to its own SEP, so I can connect dynamically in any
>>>>>>>>> fashion.
>>>>>>>>>
>>>>>>>>> My test case is transmitting 8192 random samples from host to FFT
>>>>>>>>> block and then optionally through a 2nd FFT block before back to
>>>>>>>>> host. In
>>>>>>>>> the test case, the radios/DDCs/DUCs are not used.
>>>>>>>>>
>>>>>>>>> Here is what I observed:
>>>>>>>>>
>>>>>>>>> - If I only include 1 FFT block in my RFNoC graph, I get the
>>>>>>>>> expected results (the output from the FPGA matches what I
>>>>>>>>> calculate in
>>>>>>>>> Matlab for the FFT). This is true for either of the two FFT
>>>>>>>>> blocks.
>>>>>>>>> - If I include both FFT blocks in series, I can only match the
>>>>>>>>> FPGA output if I swap the I/Q values in between my Matlab FFTs.
>>>>>>>>> - Note: that this issue is not FFT-related as I can also
>>>>>>>>> duplicate this issue with the other blocks.
>>>>>>>>> - If I use 3 blocks in series (each through SEP), I need to
>>>>>>>>> negate certain data in order to get it to match the FPGA output
>>>>>>>>>
>>>>>>>>> My next step is likely to build a new image with Ettus-developed
>>>>>>>>> FIFOs to prove that the data is getting swapped/negated when 2 or
>>>>>>>>> more are
>>>>>>>>> used in series through SEPs.
>>>>>>>>>
>>>>>>>>> Let me know if you have any suggestions for other things to try.
>>>>>>>>>
>>>>>>>>> Rob
>>>>>>>>> _______________________________________________
>>>>>>>>> USRP-users mailing list
>>>>>>>>> USRP-users@lists.ettus.com
>>>>>>>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com
>>>>>>>>>
>>>>>>>>
_______________________________________________
USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com
