Re: Developing KrakenSDR Source

[Jeff Long]

Oh, maybe the confusion is over how many items you need to output at one
time. You can hold on to your buffer - one get_iq_online() worth - until it
is empty, through multiple work() calls. Copy out
min(amount_left_from_fetch, output_items) and return the number of items
you copied (not the max size given to work() by the scheduler). It's OK if
the scheduler says to output 1024 items and you only output 4 (bad example)
if that's what is most efficient, or if you have only 4 items left
internally.

On Tue, Aug 16, 2022 at 10:19 PM Jeff Long  wrote:

> Output buffer size is adjustable - set_min_output_buffer(min_items) will
> give a buffer that is at least num_items in size, but is often larger due
> to alignment requirements. I wouldn't use vectors just to get around buffer
> sizes. Very large buffers may not work due to the way they are allocated.
> Give this a try first.
>
> On Tue, Aug 16, 2022 at 9:24 PM Carl Laufer  wrote:
>
>> Thanks. I think it has to be a vector output.
>>
>> It seems that if I'm using a stream output, and have decimation blocks
>> downstream, output_items in the source is always smaller than cpi_size, and
>> I can't fit the 2^20 array into output_items. I think it expects the source
>> to adjust its output buffer size? I'd have to throw away data as there's no
>> way to tell heimdall at runtime to reconfigure to use a smaller cpi_size.
>>
>> Unless, is there any way to force output_items to always be [5, cpi_size]
>> when using a stream output in the source block?
>>
>> On Wed, Aug 17, 2022 at 2:13 AM Jeff Long  wrote:
>>
>>> Hi Carl,
>>>
>>> Use vectors only if data always needs to be grouped in exact quantities,
>>> e.g., if the GR flowgraph needs to always handle blocks of 2^20 items. In
>>> general, a 5-channel stream would be more flexible. The variation in the
>>> number of items would be due to the output buffer sometimes being empty and
>>> sometimes not. This depends on what is happening in downstream blocks, and
>>> also on random scheduling of threads. Hope that answers some of your
>>> questions.
>>>
>>> On Tue, Aug 16, 2022 at 9:49 AM Carl Laufer  wrote:
>>>
 Hi All,

 I'm currently working on a GNU Radio source block for the KrakenSDR. So
 far my block mostly seems to work as expected, but I'm having some minor
 issues and questions.

 If you didn't know, the KrakenSDR is 5 RTL-SDR receivers, on the same
 clock with a noise source for coherence calibration of the channels. We're
 using it for applications like radio direction finding and passive
 radar, and mostly write our own code in Python. But having a GNU Radio
 source would be useful for others.

 With KrakenSDR there is a DAQ software called "heimdall" which handles
 all the coherent calibration automatically. In my source block, I'm able to
 successfully receive the data in the GNU Radio source block from heimdall
 via a socket connection.

 First so you know, the heimdall DAQ buffers an array of "cpi_size" (cpi
 = coherent processing interval) IQ data per channel, and outputs those
 arrays on the socket when it's filled. By default the cpi_size = 2^20. So
 in my GNU Radio source I'm receiving five, 2^20 long arrays of coherent
 complex IQ data every ~400ms.

 I believe in GNU Radio this is considered a vector? So should I make
 the output of the source block five port vectors, with
 out_sig=[(np.complex64, cpi_size)] * numChannels and set vlen to cpi_size
 in the yaml?

 Or instead should I have it as an output stream out_sig=[np.complex64]
 * numChannels, and be using Stream->Vector blocks when needed, with
 num_items set to cpi_size?

 I've tried both methods, and they both work. But I don't understand why
 when using the vector output implementation, the shape of output_items
 keeps flipping between (5, 2, 1048576) and (5, 1, 1048576)?

 Code is all at https://github.com/krakenrf/gr-krakensdr if anyone
 would care to take a look. Everything in Python. If anyone has any tips or
 comments please let me know. Thanks to anyone for your insights.

 Regards,
 Carl Laufer

>>>

Re: Developing KrakenSDR Source

[Jeff Long]

Output buffer size is adjustable - set_min_output_buffer(min_items) will
give a buffer that is at least num_items in size, but is often larger due
to alignment requirements. I wouldn't use vectors just to get around buffer
sizes. Very large buffers may not work due to the way they are allocated.
Give this a try first.

On Tue, Aug 16, 2022 at 9:24 PM Carl Laufer  wrote:

> Thanks. I think it has to be a vector output.
>
> It seems that if I'm using a stream output, and have decimation blocks
> downstream, output_items in the source is always smaller than cpi_size, and
> I can't fit the 2^20 array into output_items. I think it expects the source
> to adjust its output buffer size? I'd have to throw away data as there's no
> way to tell heimdall at runtime to reconfigure to use a smaller cpi_size.
>
> Unless, is there any way to force output_items to always be [5, cpi_size]
> when using a stream output in the source block?
>
> On Wed, Aug 17, 2022 at 2:13 AM Jeff Long  wrote:
>
>> Hi Carl,
>>
>> Use vectors only if data always needs to be grouped in exact quantities,
>> e.g., if the GR flowgraph needs to always handle blocks of 2^20 items. In
>> general, a 5-channel stream would be more flexible. The variation in the
>> number of items would be due to the output buffer sometimes being empty and
>> sometimes not. This depends on what is happening in downstream blocks, and
>> also on random scheduling of threads. Hope that answers some of your
>> questions.
>>
>> On Tue, Aug 16, 2022 at 9:49 AM Carl Laufer  wrote:
>>
>>> Hi All,
>>>
>>> I'm currently working on a GNU Radio source block for the KrakenSDR. So
>>> far my block mostly seems to work as expected, but I'm having some minor
>>> issues and questions.
>>>
>>> If you didn't know, the KrakenSDR is 5 RTL-SDR receivers, on the same
>>> clock with a noise source for coherence calibration of the channels. We're
>>> using it for applications like radio direction finding and passive
>>> radar, and mostly write our own code in Python. But having a GNU Radio
>>> source would be useful for others.
>>>
>>> With KrakenSDR there is a DAQ software called "heimdall" which handles
>>> all the coherent calibration automatically. In my source block, I'm able to
>>> successfully receive the data in the GNU Radio source block from heimdall
>>> via a socket connection.
>>>
>>> First so you know, the heimdall DAQ buffers an array of "cpi_size" (cpi
>>> = coherent processing interval) IQ data per channel, and outputs those
>>> arrays on the socket when it's filled. By default the cpi_size = 2^20. So
>>> in my GNU Radio source I'm receiving five, 2^20 long arrays of coherent
>>> complex IQ data every ~400ms.
>>>
>>> I believe in GNU Radio this is considered a vector? So should I make the
>>> output of the source block five port vectors, with out_sig=[(np.complex64,
>>> cpi_size)] * numChannels and set vlen to cpi_size in the yaml?
>>>
>>> Or instead should I have it as an output stream out_sig=[np.complex64] *
>>> numChannels, and be using Stream->Vector blocks when needed, with num_items
>>> set to cpi_size?
>>>
>>> I've tried both methods, and they both work. But I don't understand why
>>> when using the vector output implementation, the shape of output_items
>>> keeps flipping between (5, 2, 1048576) and (5, 1, 1048576)?
>>>
>>> Code is all at https://github.com/krakenrf/gr-krakensdr if anyone would
>>> care to take a look. Everything in Python. If anyone has any tips or
>>> comments please let me know. Thanks to anyone for your insights.
>>>
>>> Regards,
>>> Carl Laufer
>>>
>>

Re: Developing KrakenSDR Source

[Carl Laufer]

Thanks. I think it has to be a vector output.

It seems that if I'm using a stream output, and have decimation blocks
downstream, output_items in the source is always smaller than cpi_size, and
I can't fit the 2^20 array into output_items. I think it expects the source
to adjust its output buffer size? I'd have to throw away data as there's no
way to tell heimdall at runtime to reconfigure to use a smaller cpi_size.

Unless, is there any way to force output_items to always be [5, cpi_size]
when using a stream output in the source block?

On Wed, Aug 17, 2022 at 2:13 AM Jeff Long  wrote:

> Hi Carl,
>
> Use vectors only if data always needs to be grouped in exact quantities,
> e.g., if the GR flowgraph needs to always handle blocks of 2^20 items. In
> general, a 5-channel stream would be more flexible. The variation in the
> number of items would be due to the output buffer sometimes being empty and
> sometimes not. This depends on what is happening in downstream blocks, and
> also on random scheduling of threads. Hope that answers some of your
> questions.
>
> On Tue, Aug 16, 2022 at 9:49 AM Carl Laufer  wrote:
>
>> Hi All,
>>
>> I'm currently working on a GNU Radio source block for the KrakenSDR. So
>> far my block mostly seems to work as expected, but I'm having some minor
>> issues and questions.
>>
>> If you didn't know, the KrakenSDR is 5 RTL-SDR receivers, on the same
>> clock with a noise source for coherence calibration of the channels. We're
>> using it for applications like radio direction finding and passive
>> radar, and mostly write our own code in Python. But having a GNU Radio
>> source would be useful for others.
>>
>> With KrakenSDR there is a DAQ software called "heimdall" which handles
>> all the coherent calibration automatically. In my source block, I'm able to
>> successfully receive the data in the GNU Radio source block from heimdall
>> via a socket connection.
>>
>> First so you know, the heimdall DAQ buffers an array of "cpi_size" (cpi =
>> coherent processing interval) IQ data per channel, and outputs those arrays
>> on the socket when it's filled. By default the cpi_size = 2^20. So in my
>> GNU Radio source I'm receiving five, 2^20 long arrays of coherent complex
>> IQ data every ~400ms.
>>
>> I believe in GNU Radio this is considered a vector? So should I make the
>> output of the source block five port vectors, with out_sig=[(np.complex64,
>> cpi_size)] * numChannels and set vlen to cpi_size in the yaml?
>>
>> Or instead should I have it as an output stream out_sig=[np.complex64] *
>> numChannels, and be using Stream->Vector blocks when needed, with num_items
>> set to cpi_size?
>>
>> I've tried both methods, and they both work. But I don't understand why
>> when using the vector output implementation, the shape of output_items
>> keeps flipping between (5, 2, 1048576) and (5, 1, 1048576)?
>>
>> Code is all at https://github.com/krakenrf/gr-krakensdr if anyone would
>> care to take a look. Everything in Python. If anyone has any tips or
>> comments please let me know. Thanks to anyone for your insights.
>>
>> Regards,
>> Carl Laufer
>>
>

re: [GSoC porting SIMD to gr-web] weekly update

[史 皓航]

Hi, everyone!

I am working on enabling gnuradio to run faster on the browser through `simd on 
wasm`.
I will post weekly updates in this thread. The following is from last week:

Short version report:
Has Done:
   1. Building debug mode `CPython`.
2. Compare the `webapp` directory output files. (not fully finish)
3. Use `wasm2wat` to ensure `volk` is really under `wasm-simd`.

Delay and TODO in this week:
2. Compare the `webapp` directory output files.
4. Enabling open and save files for user-selected paths in gr-web.

Full version blog: blog here


Yours,
Yao

发件人: 史 皓航
发送时间: 2022年8月9日 23:34
收件人: discuss-gnuradio@gnu.org
主题: re: [GSoC porting SIMD to gr-web] weekly update

Hi, everyone!

I am working on enabling gnuradio to run faster on the browser through `simd on 
wasm`.
I will post weekly updates in this thread. The following is from last week:

Short version report:
Has Done:
Building with `volk`  `wasm-simd` success, running fail.

TODO:
1. Building debug mode `CPython`.
2. Compare the `webapp` directory output files.
3. Use `wasm2wat` to ensure `volk` is really under `wasm-simd`.
4. Enabling open and save files for user-selected paths in gr-web.

Full version blog: blog here


Yours,
Yao

发件人: 史 皓航
发送时间: 2022年8月2日 23:56
收件人: discuss-gnuradio@gnu.org
主题: re: [GSoC porting SIMD to gr-web] weekly update

Hi, everyone!

I am working on enabling gnuradio run faster on browser.
I will post weekly update in this thread. Following is last week:

Short version report:
 Has Done:
running flow graph again success
TODO:
1. Transplant volk to wasm-simd building
2. Enable edit module in flow graph on gnuradio-web (current not support edit 
module).

Full version blog: blog here


Yours,
Yao




发件人: 史 皓航 
发送时间: Monday, July 25, 2022 10:24:15 PM
收件人: discuss-gnuradio@gnu.org 
主题: 回复: [GSoC porting SIMD to gr-web] weekly update


Hi, all!



I will post weekly this project update in this thread, following is last week:



Short version report:( unfortunaly I need to rebuild gr-web)



 Has Done: 1. using latest gr commit to build gr-web. 2. Fix commit missing 
issue.

 TODO: 1. fix the running fail issue.  2. And push the modification to 
marc’s repo.



Full version report: blog 
here



Yours,

Yao



发件人: 史 皓航
发送时间: 2022年7月18日 22:39
收件人: discuss-gnuradio@gnu.org
主题: re: [GSoC porting SIMD to gr-web] weekly update



Hey, all!



I will post weekly this project update in this thread, following is last week:



Short version report:(due to covid-19 the progress slows down a little)



 Has Done: post a  
blog
 to introduce the dev tool usage(part 1)

 TODO: post another blog and measure the different implemented volk  
performance(time cost first)



Full version report: blog 
here



Yours,

Yao.







发件人: 史 皓航 
发送时间: 2022年7月12日 0:10
收件人: discuss-gnuradio@gnu.org 
主题: [GSoC porting SIMD to gr-web] weekly update



Hey, all!



This is a weekly update email about the GSoC project.



Following is last week progress and next week plan:

Short:

*  Has Done:  building gr-web wholly and play grc.

*  Need to be Done: Intend to run a benchmark to measure different implement to 
`volk`

Full blog:

 Week2 | eat4toast.github.io



Yours,

Yao



发件人: 史 皓航
发送时间: 2022年7月5日 00:06
收件人: discuss-gnuradio@gnu.org
主题: [GSoC porting SIMD to gr-web] weekly update



Hi, everyone!

I've been accepted to work with GnuRadio for this year's Google Summer of Code.



The task I intend to implement is porting SIMD code to gr-web through wasm,

which enables gnuradio more extensible and popular through the browser.



I am really sorry, this post seems a bit late due to some unexpected problems 
with the building gr-web from the source.



Last week's blog:

 https://eat4toast.github.io/2022/07/04/week1.html



I will post a weekly update on the mailing list, well the blog will record more 
latest progress.



Yours,

Yao.

Re: Developing KrakenSDR Source

[Jeff Long]

Once this is all working, you could have get_iq_online() work in a separate
thread so that you don't have to hit the network every time work() is
called. Maybe buffer a small number of frames in a queue.

On Tue, Aug 16, 2022 at 10:13 AM Jeff Long  wrote:

> Hi Carl,
>
> Use vectors only if data always needs to be grouped in exact quantities,
> e.g., if the GR flowgraph needs to always handle blocks of 2^20 items. In
> general, a 5-channel stream would be more flexible. The variation in the
> number of items would be due to the output buffer sometimes being empty and
> sometimes not. This depends on what is happening in downstream blocks, and
> also on random scheduling of threads. Hope that answers some of your
> questions.
>
> On Tue, Aug 16, 2022 at 9:49 AM Carl Laufer  wrote:
>
>> Hi All,
>>
>> I'm currently working on a GNU Radio source block for the KrakenSDR. So
>> far my block mostly seems to work as expected, but I'm having some minor
>> issues and questions.
>>
>> If you didn't know, the KrakenSDR is 5 RTL-SDR receivers, on the same
>> clock with a noise source for coherence calibration of the channels. We're
>> using it for applications like radio direction finding and passive
>> radar, and mostly write our own code in Python. But having a GNU Radio
>> source would be useful for others.
>>
>> With KrakenSDR there is a DAQ software called "heimdall" which handles
>> all the coherent calibration automatically. In my source block, I'm able to
>> successfully receive the data in the GNU Radio source block from heimdall
>> via a socket connection.
>>
>> First so you know, the heimdall DAQ buffers an array of "cpi_size" (cpi =
>> coherent processing interval) IQ data per channel, and outputs those arrays
>> on the socket when it's filled. By default the cpi_size = 2^20. So in my
>> GNU Radio source I'm receiving five, 2^20 long arrays of coherent complex
>> IQ data every ~400ms.
>>
>> I believe in GNU Radio this is considered a vector? So should I make the
>> output of the source block five port vectors, with out_sig=[(np.complex64,
>> cpi_size)] * numChannels and set vlen to cpi_size in the yaml?
>>
>> Or instead should I have it as an output stream out_sig=[np.complex64] *
>> numChannels, and be using Stream->Vector blocks when needed, with num_items
>> set to cpi_size?
>>
>> I've tried both methods, and they both work. But I don't understand why
>> when using the vector output implementation, the shape of output_items
>> keeps flipping between (5, 2, 1048576) and (5, 1, 1048576)?
>>
>> Code is all at https://github.com/krakenrf/gr-krakensdr if anyone would
>> care to take a look. Everything in Python. If anyone has any tips or
>> comments please let me know. Thanks to anyone for your insights.
>>
>> Regards,
>> Carl Laufer
>>
>

Re: Developing KrakenSDR Source

[Jeff Long]

Hi Carl,

Use vectors only if data always needs to be grouped in exact quantities,
e.g., if the GR flowgraph needs to always handle blocks of 2^20 items. In
general, a 5-channel stream would be more flexible. The variation in the
number of items would be due to the output buffer sometimes being empty and
sometimes not. This depends on what is happening in downstream blocks, and
also on random scheduling of threads. Hope that answers some of your
questions.

On Tue, Aug 16, 2022 at 9:49 AM Carl Laufer  wrote:

> Hi All,
>
> I'm currently working on a GNU Radio source block for the KrakenSDR. So
> far my block mostly seems to work as expected, but I'm having some minor
> issues and questions.
>
> If you didn't know, the KrakenSDR is 5 RTL-SDR receivers, on the same
> clock with a noise source for coherence calibration of the channels. We're
> using it for applications like radio direction finding and passive
> radar, and mostly write our own code in Python. But having a GNU Radio
> source would be useful for others.
>
> With KrakenSDR there is a DAQ software called "heimdall" which handles all
> the coherent calibration automatically. In my source block, I'm able to
> successfully receive the data in the GNU Radio source block from heimdall
> via a socket connection.
>
> First so you know, the heimdall DAQ buffers an array of "cpi_size" (cpi =
> coherent processing interval) IQ data per channel, and outputs those arrays
> on the socket when it's filled. By default the cpi_size = 2^20. So in my
> GNU Radio source I'm receiving five, 2^20 long arrays of coherent complex
> IQ data every ~400ms.
>
> I believe in GNU Radio this is considered a vector? So should I make the
> output of the source block five port vectors, with out_sig=[(np.complex64,
> cpi_size)] * numChannels and set vlen to cpi_size in the yaml?
>
> Or instead should I have it as an output stream out_sig=[np.complex64] *
> numChannels, and be using Stream->Vector blocks when needed, with num_items
> set to cpi_size?
>
> I've tried both methods, and they both work. But I don't understand why
> when using the vector output implementation, the shape of output_items
> keeps flipping between (5, 2, 1048576) and (5, 1, 1048576)?
>
> Code is all at https://github.com/krakenrf/gr-krakensdr if anyone would
> care to take a look. Everything in Python. If anyone has any tips or
> comments please let me know. Thanks to anyone for your insights.
>
> Regards,
> Carl Laufer
>

Developing KrakenSDR Source

[Carl Laufer]

Hi All,

I'm currently working on a GNU Radio source block for the KrakenSDR. So far
my block mostly seems to work as expected, but I'm having some minor issues
and questions.

If you didn't know, the KrakenSDR is 5 RTL-SDR receivers, on the same clock
with a noise source for coherence calibration of the channels. We're using
it for applications like radio direction finding and passive radar, and
mostly write our own code in Python. But having a GNU Radio source would be
useful for others.

With KrakenSDR there is a DAQ software called "heimdall" which handles all
the coherent calibration automatically. In my source block, I'm able to
successfully receive the data in the GNU Radio source block from heimdall
via a socket connection.

First so you know, the heimdall DAQ buffers an array of "cpi_size" (cpi =
coherent processing interval) IQ data per channel, and outputs those arrays
on the socket when it's filled. By default the cpi_size = 2^20. So in my
GNU Radio source I'm receiving five, 2^20 long arrays of coherent complex
IQ data every ~400ms.

I believe in GNU Radio this is considered a vector? So should I make the
output of the source block five port vectors, with out_sig=[(np.complex64,
cpi_size)] * numChannels and set vlen to cpi_size in the yaml?

Or instead should I have it as an output stream out_sig=[np.complex64] *
numChannels, and be using Stream->Vector blocks when needed, with num_items
set to cpi_size?

I've tried both methods, and they both work. But I don't understand why
when using the vector output implementation, the shape of output_items
keeps flipping between (5, 2, 1048576) and (5, 1, 1048576)?

Code is all at https://github.com/krakenrf/gr-krakensdr if anyone would
care to take a look. Everything in Python. If anyone has any tips or
comments please let me know. Thanks to anyone for your insights.

Regards,
Carl Laufer

Re: [VOLK] a += b*c ?

[Johannes Demel]

Hi Randall,

in your case,

https://github.com/gnuradio/volk/blob/main/kernels/volk/volk_32f_x2_multiply_32f.h

followed by
https://github.com/gnuradio/volk/blob/main/kernels/volk/volk_32f_x2_add_32f.h

would be the way to go at the moment.
```
volk_32f_x2_multiply_32f(multiply_result, b, c, num_samples);
volk_32f_x2_add_32f(a, a, multiply_result, num_samples);
```

You're welcome to start a new kernel
```
volk_32f_x3_multiply_add_32f(out, a, b, c, num_samples);
```
In fact, it would be a great addition to VOLK.

Cheers
Johannes


On 16.08.22 01:38, Randall Wayth wrote:
Thanks for the suggestions and apologies for not being 100% clear at the 
start.

I'm not looking for a dot product. I'm looking for
a[i] += b[i]*c[i]     specifically for floating point

So it would be the equivalent of IPP's ippsAddProduct_32f.
The application is to apply a window to a set of samples before 
accumulating, to implement a weighted overlap add PFB. In my case the 
samples are real-valued, but I could also see a case for a and b being 
complex, or the case for b being 8 or 16-bit ints with a and c being 
floating point.


Cheers,
Randall


smime.p7s
Description: S/MIME Cryptographic Signature