Re: [Discuss-gnuradio] fastnoise_source is not thread-safe

[CEL]

S, Markus,

Yep, this works.

As for some background: I build my GNU Radio with
CMAKE_BUILD_TYPE=RelWithDebInfo, which boils down to a reasonable '-g
-O2' using clang. I've built my gr-xoroshiro module with the same
parameters.

It stochastically outperforms the fastnoise_source in Gaussian mode by
orders, up to pool sizes that are roughly equivalent to the sample
size. For more benchmarks refer to [1].

So, I'd argue:

* This is already as least as good as the fastnoise source, but uses
less RAM to achieve the same quality, or is higher-quality when
fastnoise RAM usage is benign
* If I don't use a toy implementation of a normal-shaper, this will
probably perform faster than fastnoise source by orders of magnitude
* This is just demo code; the real RNG is header-only in [2], based on
[3,4] (which is public domain).
* We should fix fastnoise_src by replacing rand() with
xoroshiro128p_next(state), which is threadsafe (if state isn't shared)
* If I'm right and a non-naive normal-shaper performs much better than
averaging, then there's little reason to stick with
boost::random::mt19337 in the (non-fast) noise source, and we can just
replace the algorithm under the hood without API changes, but
increasing the performance enough to deprecate fastnoise source.

To get a feeling for the use case that should be optimized for: What's
your application of the fastnoise source? What is your parameterization
of choice?

Best regards,
Marcus

[1] https://github.com/marcusmueller/gr-xoroshiro/blob/master/examples/
Kolmogorov-Smirnov-Test.ipynb
[2] https://github.com/marcusmueller/gr-xoroshiro/blob/master/include/x
oroshiro/xoroshiro-variates.h
[3] http://vigna.di.unimi.it/xorshift/ 
[4] http://vigna.di.unimi.it/xorshift/xoroshiro128plus.c
On Fri, 2018-01-26 at 16:58 +, Müller, Marcus (CEL) wrote:
> Hello Markus,
> 
> just to add to the things said:
> The (non-fast) noise source in GR has experienced a bit of bugfixing
> (but more on a mathematical level: turns out that the built-in random
> function GNU Radio used wasn't all that random if one looked for about
> 10^8 samples)...
> 
> I'm currently reading the fastnoise_source block's code, I'm really not
> all that sure it's *fast*. I mean, neither `rand()` nor `indirectly
> addressing a RAM buffer that's larger than a CPU cache` are really an
> indication for speed. The reason why it's still faster than the noise
> source is that the distribution shaping happens offline (which means
> that probably, our distribution shaping is suboptimal; then again, the
> method used in the generation of normally distributed variables can be
> found all over the internet, probably because it's the method in
> /Numerical Recipes/, and it's a cool method, anyway!).
> 
> That having been said, maybe, instead of just fixing fastnoise source
> (which we must do, I agree!) only, we might simply want to look into
> replacing it by something faster yet threadsafe. I'll whip something up
> for benchmarking purposes; XOROSHIRO128+ has an excellent reputation
> with respect to both speed and statistical properties, and I'd be
> surprised if we couldn't use it to generate both random floats as well
> as random integers. Problem, as usual, is that a good random number
> generator does not make a good source of (pseudo)continuous or even
> just discrete randomness of any given distribution. However, I'm
> willing to test a few reference distributions (mainly, Numpy's, because
> they're easy to use in tests and hopefully well-tested themselves) for
> distributional similarity¹ to the expected distributions, and some
> auto- and crosscorrelation tests.
> 
> So, hm, how do we do this?
> 
> Anyway: Seeing you post on the mailing list lately I'd say it really
> can't hurt letting you sign the FSF CRA; I think you'd greatly benefit
> of having the possibility to quickly upstream a fix, even if it's
> complex. For smaller fixes, a CRA isn't always strictly necessary, but
> your friendly maintainer knows more :)
> 
> Best regards,
> Marcus
> 
> 
> 
> ¹ seriously, I'll have to restrict myself to a Kolmogorov-Smirnov test,
> and maybe Kullback-Leibler divergence considerations. That's all I can
> honestly claim to understand well enough to make qualified statements.
> 
> On Fri, 2018-01-26 at 00:45 +0100, Markus Wirsing wrote:
> > Hello,
> > 
> > I noticed that the fastnoise_source block is not thread safe, as it
> > calls rand(), which keeps global state.
> > This makes it non-reproducable between multiple runs once more than one
> > such source is used.
> > 
> > I also opened an issue about this on github:
> > https://github.com/gnuradio/gnuradio/issues/1542
> > However, there were no reactions to it.
> > 
> > I'm considering to write a fix for it when I find the time.
> > However, I'm not sure about the requirements that need to be met that it
> > can later be included in gnuradio.
> > 
> > Would this already require me to have a CRA?
> > I'm also wondering whether C++11 is allowed in gnuradio.
> > 

Re: [Discuss-gnuradio] fastnoise_source is not thread-safe

[CEL]

Hello Markus,

just to add to the things said:
The (non-fast) noise source in GR has experienced a bit of bugfixing
(but more on a mathematical level: turns out that the built-in random
function GNU Radio used wasn't all that random if one looked for about
10^8 samples)...

I'm currently reading the fastnoise_source block's code, I'm really not
all that sure it's *fast*. I mean, neither `rand()` nor `indirectly
addressing a RAM buffer that's larger than a CPU cache` are really an
indication for speed. The reason why it's still faster than the noise
source is that the distribution shaping happens offline (which means
that probably, our distribution shaping is suboptimal; then again, the
method used in the generation of normally distributed variables can be
found all over the internet, probably because it's the method in
/Numerical Recipes/, and it's a cool method, anyway!).

That having been said, maybe, instead of just fixing fastnoise source
(which we must do, I agree!) only, we might simply want to look into
replacing it by something faster yet threadsafe. I'll whip something up
for benchmarking purposes; XOROSHIRO128+ has an excellent reputation
with respect to both speed and statistical properties, and I'd be
surprised if we couldn't use it to generate both random floats as well
as random integers. Problem, as usual, is that a good random number
generator does not make a good source of (pseudo)continuous or even
just discrete randomness of any given distribution. However, I'm
willing to test a few reference distributions (mainly, Numpy's, because
they're easy to use in tests and hopefully well-tested themselves) for
distributional similarity¹ to the expected distributions, and some
auto- and crosscorrelation tests.

So, hm, how do we do this?

Anyway: Seeing you post on the mailing list lately I'd say it really
can't hurt letting you sign the FSF CRA; I think you'd greatly benefit
of having the possibility to quickly upstream a fix, even if it's
complex. For smaller fixes, a CRA isn't always strictly necessary, but
your friendly maintainer knows more :)

Best regards,
Marcus



¹ seriously, I'll have to restrict myself to a Kolmogorov-Smirnov test,
and maybe Kullback-Leibler divergence considerations. That's all I can
honestly claim to understand well enough to make qualified statements.

On Fri, 2018-01-26 at 00:45 +0100, Markus Wirsing wrote:
> Hello,
> 
> I noticed that the fastnoise_source block is not thread safe, as it
> calls rand(), which keeps global state.
> This makes it non-reproducable between multiple runs once more than one
> such source is used.
> 
> I also opened an issue about this on github:
> https://github.com/gnuradio/gnuradio/issues/1542
> However, there were no reactions to it.
> 
> I'm considering to write a fix for it when I find the time.
> However, I'm not sure about the requirements that need to be met that it
> can later be included in gnuradio.
> 
> Would this already require me to have a CRA?
> I'm also wondering whether C++11 is allowed in gnuradio.
> I guess it is not as I found this message on the mailing list:
> https://lists.gnu.org/archive/html/discuss-gnuradio/2014-06/msg00014.html
> It might be worth mentioning that on the coding guide in the wiki though:
> https://wiki.gnuradio.org/index.php/Coding_guide_impl
> It also might be worth considering to change that rule.
> C++11 is pretty well supported by now. And it does add some nice features.
> 
> I think the easiest fix would be to use random::ran_int() in place of
> rand() or lrand48(). That way it would also be possible to get rid of
> the (slightly) biased method of using a linear congruence to reduce the
> range.
> 
> However, while looking at the code in random.cc
> https://github.com/gnuradio/gnuradio/blob/master/gnuradio-runtime/lib/math/random.cc
> I noticed that it keeps raw pointers to the different random generators.
> I don't see why this is done. If I don't misunderstand anything, it
> would work equally well to have the objects as members in the class.
> With the added benefit of having them correctly deconstructed in the
> case of exceptions. Even though that may not be of much relevance in
> this case.
> But it at least would make sense to use unique pointers instead of raw
> pointers.
> So if I find the time to create a fix for the other problem, would some
> cleanup in this place be appreciated too?
> 
> I have no experience so far with submitting code to open source projects.
> These issues seem to be a good starting point. However, before investing
> time, I want to make sure my work can actually be used.
> 
> Appreciating any feedback on the proposed changes and on the necessary
> formalities,
> Markus
> 
> ___
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

smime.p7s
Description: S/MIME cryptographic signature
___
Discuss-gnuradio 

Re: [Discuss-gnuradio] OFDM channel tap questions

[Edwin Li]

Hi Bob,

Thanks for your detailed explanation. I figured out the causes for non-unity 
channel taps in my case. One is I didn’t consider normalization. Second, it was 
because that the Schmidl-Cox sync has a metric plateau. Imperfect 
synchronization gave me the phase rotation. 

Regards,
Edwin

Sent from Mail for Windows 10

___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] Performance Counters

[Sumit Kumar]

I had almost the same problem, then I used pybombs to install GNU Radio 
and everything. It installed apache-thrift smoothly.


After that I have built apache-thrift many times under pybombs 
environment, and it never gave any more pain(touch wood).


Sumit


On 26/01/2018 16:47, larafe...@web.de wrote:

Hello gnuradio-community,
I'm trying to use performance counters for my flowgraph which consists 
of some gnuradio blocks and some of my own oot blocks. In order to do 
so I need to install 'thrift'.

I have tried everything, but it does not work.
1) I installed thrift accoding to 
"https://wiki.gnuradio.org/index.php/ControlPort;. I have installed 
python-twisted and libssl-dev.

2) 'thrift -version' gives me 0.11.0. So it is installed.
2) Then I enter the build-folder of my gnuradio installation.
3) I do 'cmake ../'.
4) I get an error 'thrift not found'.
Gnuradio or python or both are unable to find thrift. I'm frustrated 
and I have no idea where to start.

My OS: Ubuntu 16.04
Gnuradio: 3.7.11.1
In case I'm not able to solve the problem: Is there a way to query 
performance counters from python with the flowgraph running?

Thank you,
Lara


___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio


___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] Performance Counters

[larafedor]

I'm feel like I'm spending more time reinstalling Gnuradio than actually working with it.

 

And pybombs is adding another layer of complexity: Now I have to learn what a recipe is, a prefix, I have to figure out how to reuse my oot-modules and so on... My lifetime is limited.

 

I thank you for your advice but I refuse to learn pybombs.

 

Gesendet: Freitag, 26. Januar 2018 um 16:59 Uhr
Von: "Sumit Kumar" 
An: larafe...@web.de, discuss-gnuradio@gnu.org
Betreff: Re: [Discuss-gnuradio] Performance Counters



I had almost the same problem, then I used pybombs to install GNU Radio and everything. It installed apache-thrift smoothly.

After that I have built apache-thrift many times under pybombs environment, and it never gave any more pain(touch wood).

Sumit
 

On 26/01/2018 16:47, larafe...@web.de wrote:




Hello gnuradio-community,

 

I'm trying to use performance counters for my flowgraph which consists of some gnuradio blocks and some of my own oot blocks. In order to do so I need to install 'thrift'.

 

I have tried everything, but it does not work.

 

1) I installed thrift accoding to "https://wiki.gnuradio.org/index.php/ControlPort". I have installed python-twisted and libssl-dev.

 

2) 'thrift -version' gives me 0.11.0. So it is installed.

 

2) Then I enter the build-folder of my gnuradio installation.

 

3) I do 'cmake ../'.

 

4) I get an error 'thrift not found'.

 

Gnuradio or python or both are unable to find thrift. I'm frustrated and I have no idea where to start.

 


My OS: Ubuntu 16.04

Gnuradio: 3.7.11.1


 

In case I'm not able to solve the problem: Is there a way to query performance counters from python with the flowgraph running?

 

 

Thank you,

Lara


 

 
 

___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio








___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

[Discuss-gnuradio] Performance Counters

[larafedor]

Hello gnuradio-community,

 

I'm trying to use performance counters for my flowgraph which consists of some gnuradio blocks and some of my own oot blocks. In order to do so I need to install 'thrift'.

 

I have tried everything, but it does not work.

 

1) I installed thrift accoding to "https://wiki.gnuradio.org/index.php/ControlPort". I have installed python-twisted and libssl-dev.

 

2) 'thrift -version' gives me 0.11.0. So it is installed.

 

2) Then I enter the build-folder of my gnuradio installation.

 

3) I do 'cmake ../'.

 

4) I get an error 'thrift not found'.

 

Gnuradio or python or both are unable to find thrift. I'm frustrated and I have no idea where to start.

 


My OS: Ubuntu 16.04

Gnuradio: 3.7.11.1


 

In case I'm not able to solve the problem: Is there a way to query performance counters from python with the flowgraph running?

 

 

Thank you,

Lara


___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] Call for Papers / Contributions: SDRA 2018 Friedrichshafen

[Markus Heller]

Dear List,

apologies, we still had a little problem with the DNS server of our
conference website, which is now resolved. You can find the conference 
information at:

http://2018.sdra.io

and your Yutube channel is reachable through the following forwarder:

http://youtube.sdra.io

we're looking forward to receive your contribution proposals. 

vy73 / BR
Markus
dl8rds

> >
> Call for Papers: SDRA-2018 Friedrichshafen
> 
> HAMRADIO Friedrichshafen Software Defined Radio Academy 2018 (SDRA-
> 2018)
> 
> Date: Saturday 02.06.2018
> 
> Conference Websites:
> 
> http://www.hamradio-friedrichshafen.de
> http://2018.sdra.io
> 
> SDRA-2018 invites radio amateurs and researchers from acadaemia and
> industry to submit papers for oral and poster presentations on recent
> research that addresses theoretical aspects, algorithms,
> applications, hardware and software architectures for applied
> Software Defined Radio systems and resources and other aspects of
> SDR, as well as survey and discussion papers. The invitation
> particularly addresses open source research and projects. We also
> particularly invite specialists giving introductory talks and
> tutorials on SDR technologies.
> SDRA Topics:
> 
> Advances in GNURadio related projects and research
> Algorithms, applications, architectures in SDR systems
> Real Time signal processing
> Innovative applications using modern ADC/DAC environments
> 
> Submission Information
> 
> How to submit: Please send an abstract of approximately 250 words to:
> 
> s...@darc.de
> 
> Please include the following information:
> 
> Paper title
> Author's name (and callsign). Names and callsigns of all authors
> if multiple authors.
> Author's affiliation
> Country
> Email address of the main author
> 
> All accepted papers will be published. Publication details will be
> available at a later point of time.
> 
> We ask authors to keep a limit of 6 pages. If there is a reason why
> the paper should be longer, please contact us.
> 
> We also solicit Posters and Demo papers: Poster/Demo papers describe
> a small focused result, a negative result, or a late-breaking result,
> or a description of a system that can be demonstrated on-site at the
> conference.
> 
> Papers should be formatted using the IEEE A4 templates: http://www.ie
> ee.org/conferences_events/conferences/publishing/templates.html
> Organization
> 
> Prof. Dr. Michael Hartje, DK5HH
> Markus Heller, M.A., DL8RDS
> 
> Senior Programme Committee
> 
> Prof. Dr. Michael Hartje, HS Bremen, DK5HH
> Prof. Dr. Michael Niemetz, OTH Regensburg, DG2RAM
> Prof. Dr. Michael Mächtel, HTWG Konstanz, DL2SBS
> 
> Important Dates:
> 
> Abstract Submission: 28.02.2018
> Acceptance Notification: 15.03.2018
> Presentation Slides: 15.05.2081
> Paper Presentation: 02.06.2018
> Paper Submission: 31.07.2018
> 
> Contact
> 
> For enquiries and paper submission details please do not hesitate to
> contact us:
> 
> Email: s...@darc.de Tel.: +49.89.420956305-0

___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

[Discuss-gnuradio] SDRPlay in gr-osmosdr

[Jeff Long]

Here is a PyBOMBS recipe file that pulls gr-osmosdr from my sdrplay2
branch. If anyone would like to try out this support with a PyBOMBS build,
this would be the easiest way. Best to build everything from scratch, but
you can remove the src/gr-osmosdr directory, do a pybombs fetch/rebuild if
you're comfortable with that.

The SDRplay SDK v2.11 is required, and should be installed before
gr-osmosdr is rebuilt. The ENABLE_NONFREE flag is turned on for this build.

This is just for testing, so I can get some feedback. Hopefully, this will
go into the gr-osmosdr master at some point. It "works for me".

Note that the gains on SDRplay are actually attenuations, so they appear
backward. The LNA attenuation steps on RSPs are frequency dependent, so I
gave up trying to make them look like gains. My main use case is GQRX, and
that way was even more confusing.

There is a GRC sdrplay source that is a little less generic than the
osmocom source.

There are probably ways to repackage this so it can be distributed freely,
but I'll address that if there's more interest. The SDK include file is
redistributable and its terms seem to be compatible with GPL. The shared
object can be dynamically loaded over a placeholder library that does
nothing. Yes, that's hackish and it would be better to have everything free.


gr-osmosdr.lwr
Description: Binary data
___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] fastnoise_source is not thread-safe

[Johannes Demel]

Hi Markus,

the whole gr::random class would be a candidate for a C++11 overhaul.
Though, C++11 is only supported on the next branch aka GR3.8. On GR3.7 
C++11 is not supported. To make matters worse, 3.7 requires to work with 
a fairly old Boost version. Somewhere in the 1.3x range if I recall 
correctly. Again, that makes it more difficult to write code here.


That fact that raw pointers are used in gr::random is due to its history.
I would suggest write a patch which uses unique pointers instead of raw 
pointers. Then issue a pull request and go through the discussion. I 
would expect this fix to be issued against maint. Jonathan is very 
supportive when it comes to PRs. He'll help you.


The same goes for fastnoise source. Write a small patch and create a PR.

Johannes

On 26.01.2018 00:45, Markus Wirsing wrote:

Hello,

I noticed that the fastnoise_source block is not thread safe, as it
calls rand(), which keeps global state.
This makes it non-reproducable between multiple runs once more than one
such source is used.

I also opened an issue about this on github:
https://github.com/gnuradio/gnuradio/issues/1542
However, there were no reactions to it.

I'm considering to write a fix for it when I find the time.
However, I'm not sure about the requirements that need to be met that it
can later be included in gnuradio.

Would this already require me to have a CRA?
I'm also wondering whether C++11 is allowed in gnuradio.
I guess it is not as I found this message on the mailing list:
https://lists.gnu.org/archive/html/discuss-gnuradio/2014-06/msg00014.html
It might be worth mentioning that on the coding guide in the wiki though:
https://wiki.gnuradio.org/index.php/Coding_guide_impl
It also might be worth considering to change that rule.
C++11 is pretty well supported by now. And it does add some nice features.

I think the easiest fix would be to use random::ran_int() in place of
rand() or lrand48(). That way it would also be possible to get rid of
the (slightly) biased method of using a linear congruence to reduce the
range.

However, while looking at the code in random.cc
https://github.com/gnuradio/gnuradio/blob/master/gnuradio-runtime/lib/math/random.cc
I noticed that it keeps raw pointers to the different random generators.
I don't see why this is done. If I don't misunderstand anything, it
would work equally well to have the objects as members in the class.
With the added benefit of having them correctly deconstructed in the
case of exceptions. Even though that may not be of much relevance in
this case.
But it at least would make sense to use unique pointers instead of raw
pointers.
So if I find the time to create a fix for the other problem, would some
cleanup in this place be appreciated too?

I have no experience so far with submitting code to open source projects.
These issues seem to be a good starting point. However, before investing
time, I want to make sure my work can actually be used.

Appreciating any feedback on the proposed changes and on the necessary
formalities,
Markus

___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio



___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio