Is it April already? :) :) :) > On Aug 18, 2020, at 10:43, Jack Hickish <[email protected]> wrote: > > There is, of course, always the compromise option of using half > network-endianness and half little-endianness. For example, all positive > numbers could be encoded with big-endian and negative numbers could be > encoded little-endian. This would incur a similar overhead on both little- > and big-endian CPU platforms, and would also be easily parallelizable on a > GPU decoder. > > Yours, > > Nathan Poe > > On Tue, 18 Aug 2020 at 17:18, James Smith <[email protected] > <mailto:[email protected]>> wrote: > Hi Dave, > > Yes of course! Though it makes little sense IMO to do the conversion on the > host CPU, as GPUs are pretty well-equipped to do this operation pretty > quickly if the need arises. > > In some cases being pragmatic is important - if your instrument is small, for > example, and you don't have any user-supplied equipment. In the MeerKAT case > however, we specifically cater for having third-party computers connecting to > our network, then some sort of standards-compliance comes in very handy. > Though most of our data is 8- (or 10-) bit anyway so byte order makes little > difference. > > Regards, > James > > > On Tue, Aug 18, 2020 at 3:30 PM David MacMahon <[email protected] > <mailto:[email protected]>> wrote: > I guess I’m going to play angels’s advocate and suggest the pragmatic over > the dogmatic. :) > > Some standards mandate network byte order, aka big endian, but if you’re not > constrained in that way and you know that the data will be processed > downstream by a little-endian system for the foreseeable future, then I think > it makes sense to send it out in little-endian form. You can use `le32toh()` > etc in the receiving code to make it host-endian agnostic, but on > little-endian systems that is optimized away to nothing. Sure, that might > only be saving 1 CPU cycle per value, but when you’re dealing with billions > of values per second that can start adding up! > > Of course, the packet format should be documented regardless of which > endianess is used. Future users will thank you. > > Cheers, > Dave > >> On Aug 18, 2020, at 07:21, James Smith <[email protected] >> <mailto:[email protected]>> wrote: >> >> >> Hello Nitish, >> >> So I'm going to play devil's advocate and say that while you could do the >> byte swapping in the FPGA, it would be morally wrong ;-) >> >> Ideally, all data that goes out on a network will be network order, and you >> use the ntohl or htohs functions to get it in host format. That way the code >> stays more portable - if you one day find yourself on a big-endian system, >> it would work without modification. >> (https://en.wikipedia.org/wiki/Endianness#Networking >> <https://en.wikipedia.org/wiki/Endianness#Networking>) >> >> Sometimes for performance reasons you may have to make these kinds of >> compromises, and if you do you should document them well! But most modern >> servers should have no issue with 10Gb/s datarates. You could probably even >> do the swaps in the GPUs using Nvidia's primitives. >> >> Regards, >> James >> >> >> >> >> On Tue, Aug 18, 2020 at 1:28 PM Nitish Ragoomundun >> <[email protected] <mailto:[email protected]>> wrote: >> Hi, >> >> Thanks a lot Jack. It makes sense. >> And thank you very much for the note on the 2x32-bit pair. It is exactly how >> our data is formatted. >> Ok, we will go with an FPGA correction instead of a CPU byteswap. I am >> guessing it will be faster this way. >> >> Thanks again. >> Cheers >> Nitish >> >> >> On Tue, Aug 18, 2020 at 4:47 PM Jack Hickish <[email protected] >> <mailto:[email protected]>> wrote: >> Hi Nitish, >> >> To try and answer your first question without adding confusion -- >> >> If you send a UFix64_0 value into the 10GbE block, you will need to >> interpret it on the other end via an appropriate 64-bit byte swap if your >> CPU is little-endian. >> If you send a 64-bit input into the 10GbE block where the most significant >> 32 bits are the value A, and the least significant bits are value B, you >> should interpret the 64-bits on your little endian CPU as the struct >> >> typedef struct pkt { >> uint32_t A; >> uint32_t B; >> } pkt; >> >> where each of the A and B will need byteswapping before you use them. >> >> To answer your second question -- >> Yes, you can absolutely flip the endianness on the FPGA prior to >> transmission so you don't have to byteswap on your CPU. You can either do >> this with a bus-expand + bus-create blocks, using the first to split your >> words into bytes, and then flipping them before concatenating. The Xilinx >> "bitbasher" block would also be good for this, using the Verilog (for a >> 64-bit input): >> >> out = {in[7:0], in[15:8], in[23:16], in[31:24], in[39:32], in[47:40], >> in[55:48], in[63:48]} >> >> If your 64 bit data streams are not made up of 64-bit integers (eg, they are >> pairs of 32-bit integers) then you should flip the 4 bytes of each value >> individually, but leave the ordering of the two values within the 64 bits >> unchanged. >> >> Hopefully that makes sense.... >> >> Jack >> >> >> On Tue, 18 Aug 2020 at 13:28, Nitish Ragoomundun >> <[email protected] <mailto:[email protected]>> wrote: >> >> Hello, >> >> We are setting up the digital back-end of a low-frequency telescope >> consisting of SNAP boards and GPUs. The SNAP boards packetize the data and >> send to the GPU processing nodes via 10 GbE links. We are currently >> programming the packetizer/depacketizer. >> I have a few questions about the 10gbe yellow blocks and endianness. We >> observed from the tutorials that the data stored in bram is big-endian. I >> would like to know how the data is handled by the 10gbe and in what form is >> it sent over the network. >> Our depacketizers run on Intel processors, which are little-endian. We are >> aware that network byte order is big-endian, but we noticed that integer >> data can be sent from one Intel machine to another via network without ever >> calling ntohl( ) or htonl( ) and the data was preserved. So, we would like >> to know if we need to correct the endianness when receiving the data from >> the SNAP. >> >> If we need to perform this correction, is there a way we could possibly >> correct the endianness on the FPGA itself before input to the 10gbe block? >> >> Thanks, >> Nitish >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "[email protected] <mailto:[email protected]>" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] >> <mailto:[email protected]>. >> To view this discussion on the web visit >> https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CAC6X4cOZhVBUvUfs1phQ2csuRnewowZkQ8PzjjBU62LUa0js%3Dw%40mail.gmail.com >> >> <https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CAC6X4cOZhVBUvUfs1phQ2csuRnewowZkQ8PzjjBU62LUa0js%3Dw%40mail.gmail.com?utm_medium=email&utm_source=footer>. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "[email protected] <mailto:[email protected]>" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] >> <mailto:[email protected]>. >> To view this discussion on the web visit >> https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CAG1GKSkaEebTXH0X6xiDt6DLHiEy7WUW5xb%3D7w%2BYUJHB3GB7-w%40mail.gmail.com >> >> <https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CAG1GKSkaEebTXH0X6xiDt6DLHiEy7WUW5xb%3D7w%2BYUJHB3GB7-w%40mail.gmail.com?utm_medium=email&utm_source=footer>. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "[email protected] <mailto:[email protected]>" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] >> <mailto:[email protected]>. >> To view this discussion on the web visit >> https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CAC6X4cMNM0hC1oOwq3mHT%3DO8teO_MGKy9sjUnbKC2fJH2yNUPg%40mail.gmail.com >> >> <https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CAC6X4cMNM0hC1oOwq3mHT%3DO8teO_MGKy9sjUnbKC2fJH2yNUPg%40mail.gmail.com?utm_medium=email&utm_source=footer>. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "[email protected] <mailto:[email protected]>" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] >> <mailto:[email protected]>. >> To view this discussion on the web visit >> https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CAG67D365VFetwDP%2BX80xn%2BuNmf11BTGfK3jia2%3DEOiX%3Ds1VfBw%40mail.gmail.com >> >> <https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CAG67D365VFetwDP%2BX80xn%2BuNmf11BTGfK3jia2%3DEOiX%3Ds1VfBw%40mail.gmail.com?utm_medium=email&utm_source=footer>. > > > -- > You received this message because you are subscribed to the Google Groups > "[email protected] <mailto:[email protected]>" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To view this discussion on the web visit > https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/DA2983DD-358D-45BC-A1C7-A31E80E7B476%40berkeley.edu > > <https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/DA2983DD-358D-45BC-A1C7-A31E80E7B476%40berkeley.edu?utm_medium=email&utm_source=footer>. > > -- > You received this message because you are subscribed to the Google Groups > "[email protected] <mailto:[email protected]>" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To view this discussion on the web visit > https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CAG67D34HQoV9EiszsXVAF-fsQcETh43Gh83fx8VirjyP1afm0Q%40mail.gmail.com > > <https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CAG67D34HQoV9EiszsXVAF-fsQcETh43Gh83fx8VirjyP1afm0Q%40mail.gmail.com?utm_medium=email&utm_source=footer>. > > -- > You received this message because you are subscribed to the Google Groups > "[email protected]" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To view this discussion on the web visit > https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CAG1GKSmd0D3-_bhPGA_7pgZu1aNwvRQm_eP%3DgZYzZg3nDQaaCQ%40mail.gmail.com > > <https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CAG1GKSmd0D3-_bhPGA_7pgZu1aNwvRQm_eP%3DgZYzZg3nDQaaCQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.
-- You received this message because you are subscribed to the Google Groups "[email protected]" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/9703FDED-88A8-4AF1-945C-3988B173B491%40berkeley.edu.

