Re: re-synchronize data transmission when parameter changes

[Marcus Müller]

Because hacktoberfest is upon us: Made this a formal feature request, with 
quite a lot of
instructions on how to implement it

https://github.com/gnuradio/gnuradio/issues/5113

On 29.09.21 19:11, Marcus Müller wrote:
> Hi Wei,
> 
> it sounds to me like a bit of an "esoteric" use to adjust the number of 
> repetitions at
> runtime for a transmission, but that should not stop you!
> 
> Not all things can work, though:
> 
>>  For example, by asking the two repeat blocks always
>> process the same amount of samples,
> 
> No, that's not how GNU Radio works!
> 
>> or each time I change the
>> parameters, the two file sources re-transmit the file from the beginning
>> simultaneously?
> 
> That wouldn't solve the fact that the repetition factor of your two "Repeat" 
> blocks aren't
> set synchronously.
> 
> Instead:
> If you want, you can:
> 
> Stream A -->|Streams to|   |Vector to|--->
> Stream B -->|  Vector  |--->|repeat|-->| Streams |--->
> 
> By far the most elegant way would be if you wrote your own C++ repeat block 
> that has an
> arbitrary number (instead of just 1) input (and the same number of outputs). 
> Then, you
> could avoid the "Streams to vector" and "vector to streams" trickery.
> 
> Best regards,
> Marcus
> 
> On 29/09/2021 18.10, Huang Wei wrote:
>> Hello everyone,
>>
>> I am testing the transmission of the same signal in parallel through two 
>> repeat blocks
>> (see attached flowgraph). The repeat times of both repeat blocks are set by 
>> the same GUI
>> range from 2 to 10.
>> When I start running the flowgraph, the two output signals are exactly the 
>> same as
>> expected. But when I change the repeat times from the GUI range, the two 
>> signals are not
>> synchronized anymore (see attached signal plot).
>>
>> I am wondering, is there a way to keep the two signals always synchronized? 
>> For example,
>> by asking the two repeat blocks always process the same amount of samples, 
>> or each time
>> I change the parameters, the two file sources re-transmit the file from the 
>> beginning
>> simultaneously?
>>
>> I am doing this because I want to transmit the signals simultaneously 
>> through two
>> synchronized USRPs, and I don't want the mis-match of the signals due to 
>> changing the
>> parameter in the repeat blocks every time.
>>
>> Appreciate if someone could help me!
>>
>> Regards,
>> Wei
>>
> 

Re: Completely blank flowgraph takes a minute to generate

[Marcus Müller]

Hi Jameson,

yep, and if it's not IO-bound, it's still a lot of YAML files to parse, 
so it's really a nice mini-benchmark for your filesystem and python's 
yaml implementation. By the way, which version of GNU Radio are you running?
If you want to have a look into what your OS is up to while you run 
grcc, I'd run `sudo perf record -a -g grcc parameters...`


followed by `perf report`; it's not as useful to profile the Python side 
of things, Johannes' method is the right one here, but in case you need 
to know in which libc function or which kernel function you're stuck 
most of the time: great tool!


Best regards,
Marcus

On 29/09/2021 14.04, Jameson Collins wrote:

@Johannes, that's neat.  I'll give that a shot.

@Marcus, I can't run GRC on the target as it doesn't have X.  I'm using 
grcc to generate the block, I should have mentioned that.  When you say 
IO-bound I'm guessing you mean because it's scanning the disk?  I 
haven't benchmarked this hardware yet, but the filesystem is a ram disk 
so I would expect it to be reasonably fast.  But this is something I can 
look into.


Thanks

On Wed, Sep 29, 2021 at 6:55 AM Marcus Müller > wrote:


But before going into too much depth, make sure the duration isn't just
basically identical to clicking on the "reload block library" button,
which is first IO-bound (which *can* take significant amount of CPU
time
on weaker ARMs) and then parsing-limited.

Best Regards,

Marcus

On 9/29/21 11:21 AM, Johannes Demel wrote:
 > Hi,
 >
 > I used:
 > https://docs.python.org/3/library/profile.html#module-cProfile

 > in the past to locate the problematic lines of code.
 >
 > ```
 > import cProfile
 > import pstats
 >
 > with cProfile.Profile() as pr:
 >     run_the_problematic_function_etc()
 > stats = pstats.Stats(pr)
 > stats.sort_stats('cumtime').reverse_order()
 > stats.print_stats()
 > ```
 > You might want to adopt these lines for your use-case. I started at
 > the top-level of my application and then gradually moved the code
to a
 > more fitting function.
 >
 > Cheers
 > Johannes
 >
 > On 28.09.21 23:40, Jameson Collins wrote:
 >> I have a completely blank flowgraph (well just the default
'Options'
 >> block).
 >>
 >> On my 8 core, 2 GHz, Intel based virtual machine this flowgraph
 >> generates in under a second.
 >>
 >> On my 4 core, 1GHz, embedded ARM processor it takes 56 seconds and
 >> completely maxes out 1 core while it's running.
 >>
 >> Why might cause this?  How might I troubleshoot it?
 >





smime.p7s
Description: S/MIME Cryptographic Signature

Re: re-synchronize data transmission when parameter changes

[Marcus Müller]

Hi Wei,

it sounds to me like a bit of an "esoteric" use to adjust the number of 
repetitions at runtime for a transmission, but that should not stop you!


Not all things can work, though:

>  For example, by asking the two repeat blocks always
> process the same amount of samples,

No, that's not how GNU Radio works!

> or each time I change the
> parameters, the two file sources re-transmit the file from the beginning
> simultaneously?

That wouldn't solve the fact that the repetition factor of your two 
"Repeat" blocks aren't set synchronously.


Instead:
If you want, you can:

Stream A -->|Streams to|   |Vector to|--->
Stream B -->|  Vector  |--->|repeat|-->| Streams |--->

By far the most elegant way would be if you wrote your own C++ repeat 
block that has an arbitrary number (instead of just 1) input (and the 
same number of outputs). Then, you could avoid the "Streams to vector" 
and "vector to streams" trickery.


Best regards,
Marcus

On 29/09/2021 18.10, Huang Wei wrote:

Hello everyone,

I am testing the transmission of the same signal in parallel through two 
repeat blocks (see attached flowgraph). The repeat times of both repeat 
blocks are set by the same GUI range from 2 to 10.
When I start running the flowgraph, the two output signals are exactly 
the same as expected. But when I change the repeat times from the GUI 
range, the two signals are not synchronized anymore (see attached signal 
plot).


I am wondering, is there a way to keep the two signals always 
synchronized? For example, by asking the two repeat blocks always 
process the same amount of samples, or each time I change the 
parameters, the two file sources re-transmit the file from the beginning 
simultaneously?


I am doing this because I want to transmit the signals simultaneously 
through two synchronized USRPs, and I don't want the mis-match of the 
signals due to changing the parameter in the repeat blocks every time.


Appreciate if someone could help me!

Regards,
Wei





smime.p7s
Description: S/MIME Cryptographic Signature

re-synchronize data transmission when parameter changes

[Huang Wei]

Hello everyone,

I am testing the transmission of the same signal in parallel through two
repeat blocks (see attached flowgraph). The repeat times of both repeat
blocks are set by the same GUI range from 2 to 10.
When I start running the flowgraph, the two output signals are exactly the
same as expected. But when I change the repeat times from the GUI range,
the two signals are not synchronized anymore (see attached signal plot).

I am wondering, is there a way to keep the two signals always synchronized?
For example, by asking the two repeat blocks always process the same amount
of samples, or each time I change the parameters, the two file sources
re-transmit the file from the beginning simultaneously?

I am doing this because I want to transmit the signals simultaneously
through two synchronized USRPs, and I don't want the mis-match of the
signals due to changing the parameter in the repeat blocks every time.

Appreciate if someone could help me!

Regards,
Wei

Re: Completely blank flowgraph takes a minute to generate

[Jameson Collins]

@Johannes, that's neat.  I'll give that a shot.

@Marcus, I can't run GRC on the target as it doesn't have X.  I'm using
grcc to generate the block, I should have mentioned that.  When you say
IO-bound I'm guessing you mean because it's scanning the disk?  I haven't
benchmarked this hardware yet, but the filesystem is a ram disk so I would
expect it to be reasonably fast.  But this is something I can look into.

Thanks

On Wed, Sep 29, 2021 at 6:55 AM Marcus Müller  wrote:

> But before going into too much depth, make sure the duration isn't just
> basically identical to clicking on the "reload block library" button,
> which is first IO-bound (which *can* take significant amount of CPU time
> on weaker ARMs) and then parsing-limited.
>
> Best Regards,
>
> Marcus
>
> On 9/29/21 11:21 AM, Johannes Demel wrote:
> > Hi,
> >
> > I used:
> > https://docs.python.org/3/library/profile.html#module-cProfile
> > in the past to locate the problematic lines of code.
> >
> > ```
> > import cProfile
> > import pstats
> >
> > with cProfile.Profile() as pr:
> > run_the_problematic_function_etc()
> > stats = pstats.Stats(pr)
> > stats.sort_stats('cumtime').reverse_order()
> > stats.print_stats()
> > ```
> > You might want to adopt these lines for your use-case. I started at
> > the top-level of my application and then gradually moved the code to a
> > more fitting function.
> >
> > Cheers
> > Johannes
> >
> > On 28.09.21 23:40, Jameson Collins wrote:
> >> I have a completely blank flowgraph (well just the default 'Options'
> >> block).
> >>
> >> On my 8 core, 2 GHz, Intel based virtual machine this flowgraph
> >> generates in under a second.
> >>
> >> On my 4 core, 1GHz, embedded ARM processor it takes 56 seconds and
> >> completely maxes out 1 core while it's running.
> >>
> >> Why might cause this?  How might I troubleshoot it?
> >
>
>

Re: Completely blank flowgraph takes a minute to generate

[Marcus Müller]

But before going into too much depth, make sure the duration isn't just 
basically identical to clicking on the "reload block library" button, 
which is first IO-bound (which *can* take significant amount of CPU time 
on weaker ARMs) and then parsing-limited.


Best Regards,

Marcus

On 9/29/21 11:21 AM, Johannes Demel wrote:

Hi,

I used:
https://docs.python.org/3/library/profile.html#module-cProfile
in the past to locate the problematic lines of code.

```
import cProfile
import pstats

with cProfile.Profile() as pr:
    run_the_problematic_function_etc()
stats = pstats.Stats(pr)
stats.sort_stats('cumtime').reverse_order()
stats.print_stats()
```
You might want to adopt these lines for your use-case. I started at 
the top-level of my application and then gradually moved the code to a 
more fitting function.


Cheers
Johannes

On 28.09.21 23:40, Jameson Collins wrote:
I have a completely blank flowgraph (well just the default 'Options' 
block).


On my 8 core, 2 GHz, Intel based virtual machine this flowgraph 
generates in under a second.


On my 4 core, 1GHz, embedded ARM processor it takes 56 seconds and 
completely maxes out 1 core while it's running.


Why might cause this?  How might I troubleshoot it?

Re: Completely blank flowgraph takes a minute to generate

[Johannes Demel]

Hi,

I used:
https://docs.python.org/3/library/profile.html#module-cProfile
in the past to locate the problematic lines of code.

```
import cProfile
import pstats

with cProfile.Profile() as pr:
run_the_problematic_function_etc()
stats = pstats.Stats(pr)
stats.sort_stats('cumtime').reverse_order()
stats.print_stats()
```
You might want to adopt these lines for your use-case. I started at the 
top-level of my application and then gradually moved the code to a more 
fitting function.


Cheers
Johannes

On 28.09.21 23:40, Jameson Collins wrote:

I have a completely blank flowgraph (well just the default 'Options' block).

On my 8 core, 2 GHz, Intel based virtual machine this flowgraph 
generates in under a second.


On my 4 core, 1GHz, embedded ARM processor it takes 56 seconds and 
completely maxes out 1 core while it's running.


Why might cause this?  How might I troubleshoot it?