Re: [casper] SNAP FPGA data endianness and networking

2020-08-19 Thread Jack Hickish
On a more serious note, I would generally add that in any FPGA system - in particularly those that output data over Ethernet - IMO there should be some mechanism to replace the stream with known test vectors. Inbuilt test vectors make finding bugs (like endianness issues) down the road so much

Re: [casper] SNAP FPGA data endianness and networking

2020-08-18 Thread Nitish Ragoomundun
:) Thank you all for this small debate. Our sampling rates are in the hundreds of MHz but we have many many dual-polarised antennas, so we will rather go with the pragmatic solution to be on the safe side. All the processing nodes are little-endian and the final products that we might be sharing

Re: [casper] SNAP FPGA data endianness and networking

2020-08-18 Thread David MacMahon
Is it April already? :) :) :) > On Aug 18, 2020, at 10:43, Jack Hickish wrote: > > There is, of course, always the compromise option of using half > network-endianness and half little-endianness. For example, all positive > numbers could be encoded with big-endian and negative numbers could

Re: [casper] SNAP FPGA data endianness and networking

2020-08-18 Thread Jack Hickish
There is, of course, always the compromise option of using half network-endianness and half little-endianness. For example, all positive numbers could be encoded with big-endian and negative numbers could be encoded little-endian. This would incur a similar overhead on both little- and big-endian

Re: [casper] SNAP FPGA data endianness and networking

2020-08-18 Thread James Smith
Hi Dave, Yes of course! Though it makes little sense IMO to do the conversion on the host CPU, as GPUs are pretty well-equipped to do this operation pretty quickly if the need arises. In some cases being pragmatic is important - if your instrument is small, for example, and you don't have any

Re: [casper] SNAP FPGA data endianness and networking

2020-08-18 Thread David MacMahon
I guess I’m going to play angels’s advocate and suggest the pragmatic over the dogmatic. :) Some standards mandate network byte order, aka big endian, but if you’re not constrained in that way and you know that the data will be processed downstream by a little-endian system for the foreseeable

Re: [casper] SNAP FPGA data endianness and networking

2020-08-18 Thread James Smith
Hello Nitish, So I'm going to play devil's advocate and say that while you could do the byte swapping in the FPGA, it would be morally wrong ;-) Ideally, all data that goes out on a network will be network order, and you use the ntohl or htohs functions to get it in host format. That way the

Re: [casper] SNAP FPGA data endianness and networking

2020-08-18 Thread Nitish Ragoomundun
Hi, Thanks a lot Jack. It makes sense. And thank you very much for the note on the 2x32-bit pair. It is exactly how our data is formatted. Ok, we will go with an FPGA correction instead of a CPU byteswap. I am guessing it will be faster this way. Thanks again. Cheers Nitish On Tue, Aug 18,

Re: [casper] SNAP FPGA data endianness and networking

2020-08-18 Thread Jack Hickish
Hi Nitish, To try and answer your first question without adding confusion -- If you send a UFix64_0 value into the 10GbE block, you will need to interpret it on the other end via an appropriate 64-bit byte swap if your CPU is little-endian. If you send a 64-bit input into the 10GbE block where