Re: GnuRadio 3.10 OOT python modules not found - destination path error '/usr/local/local/lib64'
Thanks Jeff and Cinaed -- good info on volk and uhd. At this time, I am trying to use just the Fedora packages installed through dnf. It simplifies the install process. If I need to install from source this is a good heads up. Criss Swaim csw...@tpginc.net cell: 505.301.5701 On 6/1/2022 5:09 AM, Jeff Long wrote: It looks like volk and uhd also place files in /local on the newer platforms, so users will need to add the extra path even though we think we've fixed it for GNU Radio. On Tue, May 31, 2022 at 6:54 PM Cinaed Simson wrote: Hi Criss - I remember having the same issue. I rolled with the punch by adding the python path - then I forgot about it. I upgraded the OS but did a clean install of gnuradio 10. I just updated volk and gnuradio and rebuilt volk the problem is still there for me - at least after rebuilding volk I was going to do a dist-upgrade to get rid of all the previous OS packages and update them but I haven't had chance. -- Cinaed On 5/31/22 11:39, Criss Swaim wrote: > Upgraded to GnuRadio 3.10 from 3.9 and gnuradio-companion aborts with > a module not found trying to import my OOT python blocks. The modules > are being placed in /usr/local/local/lib64/python3.10/site-packages - > note the local/local. the destination should be > /usr/local/lib64/python3.10/site-packages. > > The fix is after running the cmake ../ command in the build directory, > edit build/python/cmake_install.cmake and > build/python/bindings/cmake_install.cmake and change > > file(INSTALL DESTINATION > "$(CMAKE_INSTALL_PREFIX)/local/lib64/python3.10/site-packages. > to > > file(INSTALL DESTINATION > "$(CMAKE_INSTALL_PREFIX)/lib64/python3.10/site-packages. (remove > /local) > > there are 2 changes in build/python/cmake_install.cmake and 8 changes > in build/python/bindings/cmake_install.cmake > > Make these changes before running the make/sudo make install. > > I am unable to determine if this is a problem in the gr_modtool or in > the GR_PYTHON_DIR constant or in some setting in my custom OOT project. > > My environment is Fedora 36, Gnuradio 3.10 and Python 3.10. >
GnuRadio 3.10 OOT python modules not found - destination path error '/usr/local/local/lib64'
Upgraded to GnuRadio 3.10 from 3.9 and gnuradio-companion aborts with a module not found trying to import my OOT python blocks. The modules are being placed in /usr/local/local/lib64/python3.10/site-packages - note the local/local. the destination should be /usr/local/lib64/python3.10/site-packages. The fix is after running the cmake ../ command in the build directory, edit build/python/cmake_install.cmake and build/python/bindings/cmake_install.cmake and change file(INSTALL DESTINATION "$(CMAKE_INSTALL_PREFIX)/local/lib64/python3.10/site-packages. to file(INSTALL DESTINATION "$(CMAKE_INSTALL_PREFIX)/lib64/python3.10/site-packages. (remove /local) there are 2 changes in build/python/cmake_install.cmake and 8 changes in build/python/bindings/cmake_install.cmake Make these changes before running the make/sudo make install. I am unable to determine if this is a problem in the gr_modtool or in the GR_PYTHON_DIR constant or in some setting in my custom OOT project. My environment is Fedora 36, Gnuradio 3.10 and Python 3.10. -- Criss Swaim csw...@tpginc.net cell: 505.301.5701
Re: (gnuradio3.8) module 'gnuradio.blocks' has no attribute 'message_sink'
Here is
a solution using an asynchronous probe:
In this
application, there is a need to periodically send signals to a
background task for non-realtime calculations.
# connect the
queue to the input signal
msgq = gr.msg_queue(2)
msg_sink =
blocks.message_sink(gr.sizeof_float*num_bins, msgq, True) #
dont block
self.connect(self, msg_sink)
# start the update thread
update_thread(
num_bins=num_bins,
update_timeout=update_timeout,
msgq=msgq,
)
then in the background thread, the queue is read
and processed
# read from
the queue and resize to have arrays of size num bins
raw_samps =
numpy.fromstring(self._msgq.delete_head().to_string(),
numpy.float32)
raw_samps.resize(len(raw_samps) / self._num_bins,
self._num_bins)
With the message_sink block no longer supported,
the probe_signal_vf block works for me:
# setup the
probe and connect to input signal
probe = blocks.probe_signal_vf(num_bins)
self.connect(self, probe)
# start the update thread
update_thread(
num_bins=num_bins,
update_timeout=update_timeout,
probe = probe,
..
)
and in the background thread:
# read from the
probe and resize to have arrays of size num bins
raw_samps = numpy.array(self._probe.level())
raw_samps.resize(1, self._num_bins)
This will work if your application does not need
every signal passed to the background task.
Criss Swaim
csw...@tpginc.net
cell: 505.301.5701
On 1/15/2021 2:09 AM, Marcus Müller
wrote:
So, the old msg_queue infrastructure of GNU Radio has been removed, yes.
A suitable replacement are the asynchronous message passing blocks.
Depending on your use case, the conversion of your code should be
relatively straightforward - the message debug sink might even do what
you want.
Best regards,
Marcus
On 07.01.21 04:46, Ting Wu wrote:
Hi!
I recently upgraded to gnuradio 3.8 and my old code threw the following
error:
module 'gnuradio.blocks' has no attribute 'message_sink'
Is the 'message_sink' has been deprecated in the recent version of
gnuradio? If so, what should I use now for the message source and sink?
I really do not want to rewrite all the codes, so I would be really
grateful if there is a simple way to make the following code work with
the recent version of gnuradio. The code simply gets data stream from a
USRP N200 (LFRX daughterboard).
=
class my_block(gr.top_block):
def __init__(self):
gr.top_block.__init__(self)
self.source = uhd.usrp_source(device_addr="",
stream_args=uhd.stream_args('sc16', 'sc16', args="scalar=1024"))
self.source.set_samp_rate(samp_rate)
self.source.set_gain(gain)
self.source.set_center_freq(0)
self.queue = gr.msg_queue()
self.sink = blocks.message_sink(gr.sizeof_short*2, self.queue,
False) #This line throws the error
self.connect(self.source, self.sink)
===
Thanks in advance!
Ting Wu
Re: Migration 3.7 to 3.8 - GRC OOT module fails to import multi_usrp
Resolved: the 3.7 to 3.8 migration guide shows components must be added to the CMake: I am using the UHD component, so added as such > 3.7.2 > set(GR_REQUIRED_COMPONENTS RUNTIME PMT UHD) > find_package(Gnuradio "3.7.2" REQUIRED) > > 3.8.2 > find_package(Gnuradio "3.8" REQUIRED COMPONENTS UHD) > include(GrVersion) Also, the CMake in the lib directory must be updated, adding uhd > target_link_libraries(gnuradio-vandevender gnuradio::gnuradio-runtime uhd) Criss Swaim csw...@tpginc.net cell: 505.301.5701 On 1/28/2021 2:55 PM, Criss Swaim wrote: > I am running Fedora 33, GnuRadio 3.8.2 and UHD 3.15. GnuRadio/UHD was > installed from the dnf package with the following command: > >> dnf install -y gnuradio3.8 uhd-devel gnuradio-devel uhd-tools > Following the OOT tutorial I have updated the cmake as follows: > >> 3.7.2 >> set(GR_REQUIRED_COMPONENTS RUNTIME PMT UHD) >> find_package(Gnuradio "3.7.2" REQUIRED) >> >> 3.8.2 >> find_package(Gnuradio "3.8" REQUIRED COMPONENTS UHD) >> include(GrVersion) > After compiling my OOT module, the compile is clean, but running the > block in grc I receive an error. > >> ImportError: >> /home/sensor/qn382/gr-vandevender/build/lib/libgnuradio-vandevender.so.1.0.0git: >> undefined symbol: _ZN3uhd4usrp10multi_usrp9ALL_CHANSE > in the OOTcmodule, there is this line of code: > >> usrp = uhd::usrp::multi_usrp::make(ip_addr); > To debug, I opened ipython and imported the swig file and am receiving > the following error (thank you Martin Braun for this debugging tip): > >> In [1]: import swig._vandevender_swig >> --- >> ImportError Traceback (most recent call >> last) >> in >> > 1 import swig._vandevender_swig >> >> ImportError: >> /home/sensor/qn382/gr-vandevender/build/lib/libgnuradio-vandevender.so.1.0.0git: >> undefined symbol: _ZN3uhd4usrp10multi_usrp9ALL_CHANSE > In ipython, I have imported uhd.usrp and dir(uhd.usrp.lib.usrp) finds > the multi_ursp function is found, > > So I suspect I need something, somewhere in the CMake file is needed to > point swig to the correct python module uhd.usrp.lib.usrp. But I am > lost. Is this even the correct area? Are there other areas I need to > look into? This is my first exposure to the 'modern CMake', so if the > problem is in it, some specific guidance would be appreciated. >
Migration 3.7 to 3.8 - GRC OOT module fails to import multi_usrp
I am running Fedora 33, GnuRadio 3.8.2 and UHD 3.15. GnuRadio/UHD was installed from the dnf package with the following command: > dnf install -y gnuradio3.8 uhd-devel gnuradio-devel uhd-tools Following the OOT tutorial I have updated the cmake as follows: > 3.7.2 > set(GR_REQUIRED_COMPONENTS RUNTIME PMT UHD) > find_package(Gnuradio "3.7.2" REQUIRED) > > 3.8.2 > find_package(Gnuradio "3.8" REQUIRED COMPONENTS UHD) > include(GrVersion) After compiling my OOT module, the compile is clean, but running the block in grc I receive an error. > ImportError: > /home/sensor/qn382/gr-vandevender/build/lib/libgnuradio-vandevender.so.1.0.0git: > undefined symbol: _ZN3uhd4usrp10multi_usrp9ALL_CHANSE in the OOTcmodule, there is this line of code: > usrp = uhd::usrp::multi_usrp::make(ip_addr); To debug, I opened ipython and imported the swig file and am receiving the following error (thank you Martin Braun for this debugging tip): > In [1]: import swig._vandevender_swig > --- > ImportError Traceback (most recent call > last) > in > > 1 import swig._vandevender_swig > > ImportError: > /home/sensor/qn382/gr-vandevender/build/lib/libgnuradio-vandevender.so.1.0.0git: > undefined symbol: _ZN3uhd4usrp10multi_usrp9ALL_CHANSE In ipython, I have imported uhd.usrp and dir(uhd.usrp.lib.usrp) finds the multi_ursp function is found, So I suspect I need something, somewhere in the CMake file is needed to point swig to the correct python module uhd.usrp.lib.usrp. But I am lost. Is this even the correct area? Are there other areas I need to look into? This is my first exposure to the 'modern CMake', so if the problem is in it, some specific guidance would be appreciated. -- Criss Swaim csw...@tpginc.net cell: 505.301.5701
Re: Migration from 3.7 to 3.8 OOT - module 'howto' has no attribute 'square_ff'
Martin: Thanks for the debugging hint. I went back through the tutorial and with the help of this import I was able to determine the problems( code error and I needed to run ldconfig I went to my 'real' OOT module and import the swig file and receive this message: ImportError: /home/sensor/qn382/gr-vandevender/build/lib/libgnuradio-vandevender.so.1.0.0git: undefined symbol: _ZN3uhd4usrp10multi_usrp9ALL_CHANSE This seems to be point to the uhd:usrp:multi_usrp - so I am researching. I wonder if you have any suggestions on where to look for this error. Again, thanks for the assist. Criss Swaim csw...@tpginc.net cell: 505.301.5701 On 1/25/2021 2:48 AM, Martin Braun wrote: Criss, a good debugging method is to go into the build dir, where you should find a DLL called _howto_swig.so (or something like that). Try importing that: >>> import _howto_swig It will probably fail, and will likely give you a better clue of what's going on. --M On Sat, Jan 23, 2021 at 12:51 AM Criss Swaim <csw...@tpginc.net> wrote: I am upgrading from gnuradio 3.7.2 to 3.8.2 and have run into a couple of issues with my OOT modules not showing up in the python module. So I went back and walked through the HowTo tutorial to see it the tutorial would work. It fails also. OS: Fedora 33 > QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to > '/tmp/runtime-sensor' > Traceback (most recent call last): > File "/home/sensor/gr-howto/HowTo.py", line 175, in > main() > File "/home/sensor/gr-howto/HowTo.py", line 151, in main > tb = top_block_cls() > File "/home/sensor/gr-howto/HowTo.py", line 112, in __init__ > self.howto_square_ff_0 = howto.square_ff() > AttributeError: module 'howto' has no attribute 'square_ff' The libgnuradio-howto.so lib is in /usr/local/lib64 I have added the /usr/local/lib64 directory to ld.so.conf and ran the sudo ldconfig to reload the lib paths, followed by make/make install. When the python howto module is created it does not have the square_ff entry point: > dir(howto) > ['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', > '__name__', '__package__', '__path__', '__spec__', 'unicode_literals'] Any ideas on where to look next? I am not sure if this is a path-ing issue or is something wrong in the building of the python module. -- Criss Swaim csw...@tpginc.net cell: 505.301.5701
Re: 3.9 Setup for OOT development
I have not used 3.9, but I have seen this behavior in 3.8 when the gr_modtool makeyaml blockname step was skipped. I would suggest running this step then re-make/make install. After running this you should see the input/output parameters defined in the yaml file Criss Swaim csw...@tpginc.net cell: 505.301.5701 On 1/22/2021 4:07 PM, Gavin Jacobs wrote: You may recall that I was asking about gr_modtool a week ago. I've made some progress and I think I'm very close to making my block work. I'm using Ubuntu Desktop 20.04 in a Virtual Machine, with GNU Radio 3.9.0.0git~master~14059~focal. Here is what I've learned so far: packages needed: git doxygen clang-format cmake pybind11-dev python3-pygccxml liborc-0.4-dev workflow: cd ~/modules gr_modtool newmod modulename cd gr-modulename gr_modtool add blockname < answer questions about blockname > < edit ./lib/blockname_impl.h ./lib/blockname_impl.cc ./include/modulename/blockname.h > gr_modtool makeyaml < edit ./grc/modulename_blockname.block.yml gr_modtool bind blockname mkdir build cd build cmake ../ < check for errors; ignore '-- Could NOT find MPIR ...' make sudo make install sudo ldconfig (needed once on ubuntu, after initial 'sudo make install') < check gnuradio-companion for modulename->blockname; test blockname > My block is a trivial block that copies a float input to a float output. There are no parameters. I've gone through the above steps without errors, and I get to the very end, where I can see the block in the gnuradio-companion list, BUT when I try to add it to a flowgraph, it won't show up on the canvas. I've tried double-click and drag/drop, but neither puts the block in the flowgraph. I would like to hear from anyone who has built an OOT block from scratch, using 3.9.0 gr_modtool. Is the above procedure correct? What might cause the block to not load? Assuming I get it working, is there a page planned on the wiki where I can document the procedures, tips, and pitfalls(I've found a few)? Thanks, Gavin
Migration from 3.7 to 3.8 OOT - module 'howto' has no attribute 'square_ff'
I am upgrading from gnuradio 3.7.2 to 3.8.2 and have run into a couple of issues with my OOT modules not showing up in the python module. So I went back and walked through the HowTo tutorial to see it the tutorial would work. It fails also. OS: Fedora 33 > QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to > '/tmp/runtime-sensor' > Traceback (most recent call last): > File "/home/sensor/gr-howto/HowTo.py", line 175, in > main() > File "/home/sensor/gr-howto/HowTo.py", line 151, in main > tb = top_block_cls() > File "/home/sensor/gr-howto/HowTo.py", line 112, in __init__ > self.howto_square_ff_0 = howto.square_ff() > AttributeError: module 'howto' has no attribute 'square_ff' The libgnuradio-howto.so lib is in /usr/local/lib64 I have added the /usr/local/lib64 directory to ld.so.conf and ran the sudo ldconfig to reload the lib paths, followed by make/make install. When the python howto module is created it does not have the square_ff entry point: > dir(howto) > ['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', > '__name__', '__package__', '__path__', '__spec__', 'unicode_literals'] Any ideas on where to look next? I am not sure if this is a path-ing issue or is something wrong in the building of the python module. -- Criss Swaim csw...@tpginc.net cell: 505.301.5701
Re: (gnuradio3.8) module 'gnuradio.blocks' has no attribute 'message_sink'
Hi
Ting:
I have
been working on this same issue for a couple of day and have
not found a solution. Did you find a solution to the missing
"message_sink"?
Criss Swaim
csw...@tpginc.net
cell: 505.301.5701
On 1/6/2021 8:46 PM, Ting Wu wrote:
Hi!
I recently upgraded to gnuradio 3.8 and my old code threw the
following error:
module 'gnuradio.blocks' has no attribute 'message_sink'
Is the 'message_sink' has been deprecated in the recent version of
gnuradio? If so, what should I use now for the message source and
sink? I really do not want to rewrite all the codes, so I would be
really grateful if there is a simple way to make the following
code work with the recent version of gnuradio. The code simply
gets data stream from a USRP N200 (LFRX daughterboard).
=
class my_block(gr.top_block):
def __init__(self):
gr.top_block.__init__(self)
self.source = uhd.usrp_source(device_addr="",
stream_args=uhd.stream_args('sc16', 'sc16', args="scalar=1024"))
self.source.set_samp_rate(samp_rate)
self.source.set_gain(gain)
self.source.set_center_freq(0)
self.queue = gr.msg_queue()
self.sink = blocks.message_sink(gr.sizeof_short*2,
self.queue, False) #This line throws the error
self.connect(self.source, self.sink)
===
Thanks in advance!
Ting Wu
Re: Maximum Number of Bins
Thank you Marcus & Marcus - your insights are greatly appreciated. I am looking at the suggestions, exp the fft conversion and we are considering upgrading, but need to see if the system will scale, as is. BTW, I am maintaining the code and not the original developer, so I am not familiar with all the pieces, esp. the ones that have been working. I look at this as an opportunity to dig deeper into GnuRadio. 1) for clarification, we have been running the 3.7 code for 4 -5 years - not sure when we upgraded to 3.7.9. The system runs with 2 million fft bins, but at 3 or 4 million, it fails. M. Leach has demonstrated that without our custom block, GnuRadio can process the high bin levels. I have run various configurations of our model (without the bin_sub_avg python block) but I still receive the error. 2) Both of you have mentioned we are using old message queues...Can you point me to some documentation that explains this. We are using the blocks.add_const_vff and connect functions to remove background constant (a numpy array) from the signal stream. What would be a better approach? I have not looked at this for several years, so I need to refresh and this would be a good time to look at alternate options. It is bit of a black box for me and I would like to research alternate approaches as I dig into this process. 3) M. Leach: you indicated that the conversions from a stream->string->numpy array is very inefficient. Can you point to another approach to convert a stream to numpy array? This is done once every 60 minutes, but still if it could be improved, that would help. 4) Finally, I have also been looking for a change log for the 3.7 to 3.8 system. Moving from 3.6 to 3.7 was a significant change and was wondering if 3.7 to 3.8 is the same level of effort for custom blocks. Also, is there a timeline for 3.9? Again, thanks for any guidance. Criss Swaim csw...@tpginc.net cell: 505.301.5701 On 10/31/2020 9:55 AM, Marcus Müller wrote: > Hi Craig, hi Marcus, > > Also, just because I need to point that out: > > GNU Radio 3.7 is really a legacy series of releases by now. You should > avoid using it for new developments - it's getting harder and harder to > even build it on modern versions of Linux. In fact, a lot of its > dependencies simply don't exist for modern systems anymore. > Developing for 3.7 is hence dangerous in terms of lifetime. That's among > the chief reasons why we released 3.8. Took us long enough! > > 3.7.9.2 is positively ancient. A 3.7.13.4 or later should be the oldest > version of GNU Radio you work with, even when maintaining old code. > > Other than that: > >> Oct 29 10:45:07 tf kernel: analysis_sink_1[369]: segfault at >> 7f9c5a7fd000 ip 7f9dd9361d43 sp 7f9c5a48a638 error 6 in >> libgnuradio-vandevender.so[7f9dd9336000+4d000] > This really looks like a bug in your code! > These happen easily with the older style msgq that you seem to be using > (we've basically all but removed these in current development versions > of GNU Radio), especially if directly interfacing with Python land, > which has different ideas of object lifetime than your C++ code might > have... > I think a slight reconsideration of your software architecture might > help here, but I've not seen your overall code. > >> With an FFT size of 2**22 bins. This took about 20 seconds for the >> FFTW3 "planner" to crunch on, but after that, worked >> just fine within the flow-graph. >> > Not quite 20s for me, but yes, single-threaded FFT performance was about > 14 transforms of that size per second, 2 threads allowed for ~23 > transforms a second, 4 threads for about 28. Knowing GNU Radio, I'd > recommend you rather stick with a single thread per transform, because > other block also have CPU requirements (if you really want to increase > throughput, deinterleave vectors and have multiple single-threaded FFTs > run in parallel, then recombine after). > > Seeing that you you only need 20 MS/s, and 14 transform are 14 · 2²² = > 7·2²³ samples a second and that would be roughly 56 MS/s, I think you > are fine. If you're not, get a faster PC, honestly! > > Best regards, > Marcus M (the younger Marcus) > > On 29.10.20 23:53, Marcus D. Leech wrote: >> On 10/29/2020 06:03 PM, Criss Swaim wrote: >>> we are running version 3.7.9.2 >>> >> I constructed a simple flow-graph in GR 3.7.13.5 >> >> osmosdr_source--->stream-to-vector-->fft-->null-sink >> >> With an FFT size of 2**22 bins. This took about 20 seconds for the >> FFTW3 "planner" to crunch on, but after that, worked >> just fine within the flow-graph. >> >> You should really keep your FFT sizes to a power-of-2, particularly at >> this
Re: Maximum Number of Bins
we are running version 3.7.9.2 Criss Swaim csw...@tpginc.net cell: 505.301.5701 On 10/29/2020 11:37 AM, Marcus D. Leech wrote: On 10/29/2020 01:17 PM, Criss Swaim wrote: I have attached a png of the flow graph and the error msgs from the system log are below. These error messages are the only messages. Oct 29 10:45:26 tf abrt-hook-ccpp[378]: /var/spool/abrt is 23611049718 bytes (more than 1279MiB), deleting 'ccpp-2020-10-27-15:30:43-28474' Oct 29 10:45:07 tf abrt-hook-ccpp[378]: Process 329 (python2.7) of user 1000 killed by SIGSEGV - dumping core Oct 29 10:45:07 tf audit[370]: ANOM_ABEND auid=1000 uid=1000 gid=1000 ses=8656 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 pid=370 comm="copy11" exe="/usr/bin/Oct 29 10:45:07 tf audit[369]: ANOM_ABEND auid=1000 uid=1000 gid=1000 ses=8656 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 pid=369 comm="analysis_sink_1" exe="Oct 29 10:45:07 tf kernel: traps: copy11[370] general protection ip:7f9e0acfdee0 sp:7f9c5a7fb590 error:0 in libpthread-2.22.so[7f9e0acf1000+18000] Oct 29 10:45:07 tf kernel: analysis_sink_1[369]: segfault at 7f9c5a7fd000 ip 7f9dd9361d43 sp 7f9c5a48a638 error 6 in libgnuradio-vandevender.so[7f9dd9336000+4d000] Flow is USRP -> stream to vector -> fft -> complex to mag -> bin_sub_avg -> analysis_sinkf bin_sub_avg (python) & analysis_sinkf (c/c++) are custom blocks. the function of Bin Sub Avg, which is written in Python, is to start a background task which periodically (in this case hourly) samples the input signal, calculates the background noise and subtracts it from the signal that is passed the the Analysis_sinkf module. Analys_sinkf monitors each bin and only when specific thresholds for the bin are met (ie duration, strength) is the signal written out to a signal file. Signals not passing the criteria are dropped. This code base has been running for over 3 years, with the original system implementation about 8/9 years ago. I have traced the problem to the input signal into bin_sub_avg when the number of fft bins is 3 million (2 million works). At 3 million bins, any reference to the result of the delete_head() function in the python code causes a failure. The python code just fails without a traceback, then the invalid data stream is passed to the analysis_sinkf module which is C/C++ and it causes the segment fault. Thus my suspicion is there is a limit in the fft block on the number of bins it can handle and some variable is overflowing, but this is a guess at this point. There may be a restriction in the gr.signature_io module, but that seems unlikely. What version of Gnu Radio is this?
Re: Maximum Number of Bins
I have
attached a png of the flow graph and the error msgs from the
system log are below. These error messages are the only
messages.
Oct 29 10:45:26 tf
abrt-hook-ccpp[378]: /var/spool/abrt is 23611049718 bytes
(more than 1279MiB), deleting
'ccpp-2020-10-27-15:30:43-28474'
Oct 29 10:45:07 tf abrt-hook-ccpp[378]: Process 329
(python2.7) of user 1000 killed by SIGSEGV - dumping core
Oct 29 10:45:07 tf audit[370]: ANOM_ABEND auid=1000 uid=1000
gid=1000 ses=8656
subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
pid=370 comm="copy11" exe="/usr/bin/Oct 29 10:45:07 tf
audit[369]: ANOM_ABEND auid=1000 uid=1000 gid=1000 ses=8656
subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
pid=369 comm="analysis_sink_1" exe="Oct 29 10:45:07 tf
kernel: traps: copy11[370] general protection
ip:7f9e0acfdee0 sp:7f9c5a7fb590 error:0 in
libpthread-2.22.so[7f9e0acf1000+18000]
Oct 29 10:45:07 tf kernel: analysis_sink_1[369]: segfault at
7f9c5a7fd000 ip 7f9dd9361d43 sp 7f9c5a48a638 error 6
in libgnuradio-vandevender.so[7f9dd9336000+4d000]
Flow is
USRP -> stream to vector -> fft -> complex to mag
-> bin_sub_avg -> analysis_sinkf
bin_sub_avg
(python) & analysis_sinkf (c/c++) are custom blocks.
the
function of Bin Sub Avg, which is written in Python, is to
start a background task which periodically (in this case
hourly) samples the input signal, calculates the background
noise and subtracts it from the signal that is passed the the
Analysis_sinkf module.
Analys_sinkf
monitors each bin and only when specific thresholds for the
bin are met (ie duration, strength) is the signal written out
to a signal file. Signals not passing the criteria are
dropped.
This
code base has been running for over 3 years, with the original
system implementation about 8/9 years ago.
I have
traced the problem to the input signal into bin_sub_avg when
the number of fft bins is 3 million (2 million works). At 3
million bins, any reference to the result of the delete_head()
function in the python code causes a failure. The python code
just fails without a traceback, then the invalid data stream
is passed to the analysis_sinkf module which is C/C++ and it
causes the segment fault.
Thus my
suspicion is there is a limit in the fft block on the number
of bins it can handle and some variable is overflowing, but
this is a guess at this point. There may be a restriction in
the gr.signature_io module, but that seems unlikely.
Criss Swaim
csw...@tpginc.net
cell: 505.301.5701
On 10/28/2020 5:34 PM, Marcus D Leech
wrote:
Sharing your flow-graph. The exact error messages and more context would be good
Presumably you’re talking about FFT bins but it’s not clear.
Also why are your samples being conveyed as strings ? That’s wildly inefficient.
Sent from my iPhone
On Oct 28, 2020, at 7:24 PM, Criss Swaim wrote:
I am working on a new application of gnuradio that pushes the
limits--satellite-based detection of RF from rotating
magnetized-quark-nugget dark matter transiting through the
magnetosphere--and need as many bins as possible to reduce the
background noise per frequency channel.
I have successfully run with 2 million bins, but when I jump to 3
million bins, the application abends with a segment fault. I have
deconstructed the following python line in a custom python block that is
failing:
raw_samps = numpy.fromstring(self._msgq.delete_head().to_string(), numpy.float32)
and the failure is occurring while trying to convert the results from
the delete_head() to a string (to_string()). Any reference to the
result of the delete_head() functions results in an error.
the _msgq is defined as _msgq = gr.msg_queue(MSGQ_LIMIT) where the
MSGQ_LIMIT = 2
Here is the refactored code:
# refactor raw_samps line test_str = self._msgq.delete_head()
print("bin_sub_avg::got msg "
sys.stdout.flush()
print(test_str)
sys.stdout.flush() test_string = test_str.to_string()
print("bin_sub_avg::converted msg to string")
sys.stdout.flush()
raw_samps = numpy.fromstring(test_string, numpy.float32)
print("bin_sub_avg::converted from string to numpy array")
sys.stdout.flush()
The output is:
bin_sub_avg::got msg
"the object for the shared pointer - test_str" (I did not sav
Maximum Number of Bins
I am working on a new application of gnuradio that pushes the
limits--satellite-based detection of RF from rotating
magnetized-quark-nugget dark matter transiting through the
magnetosphere--and need as many bins as possible to reduce the
background noise per frequency channel.
I have successfully run with 2 million bins, but when I jump to 3
million bins, the application abends with a segment fault. I have
deconstructed the following python line in a custom python block that is
failing:
> raw_samps = numpy.fromstring(self._msgq.delete_head().to_string(),
> numpy.float32)
and the failure is occurring while trying to convert the results from
the delete_head() to a string (to_string()). Any reference to the
result of the delete_head() functions results in an error.
the _msgq is defined as _msgq = gr.msg_queue(MSGQ_LIMIT) where the
MSGQ_LIMIT = 2
Here is the refactored code:
> # refactor raw_samps line test_str = self._msgq.delete_head()
> print("bin_sub_avg::got msg "
> sys.stdout.flush()
> print(test_str)
> sys.stdout.flush() test_string = test_str.to_string()
> print("bin_sub_avg::converted msg to string")
> sys.stdout.flush()
> raw_samps = numpy.fromstring(test_string, numpy.float32)
> print("bin_sub_avg::converted from string to numpy array")
> sys.stdout.flush()
The output is:
bin_sub_avg::got msg
"the object for the shared pointer - test_str" (I did not save the exact
message)
Then the application aborts.
Is there a limit on the number of bins gnuradio can handle?
Any thoughts on how to find the cause or limit?
--
Criss Swaim
csw...@tpginc.net
cell: 505.301.5701
Re: [Discuss-gnuradio] firmware problem causing problem with find_usrps?
On 7/1/2010 12:20 PM, ra...@csail.mit.edu wrote: > Hi, > > I am trying to get USRP2 with RFX2400 daughterboard working on Ubuntu > 9.10 (karmic). I have followed the instructions here > (http://gnuradio.org/redmine/wiki/1/UbuntuInstall). > > I installed the firmware and FPGA code on the SD card from > http://code.ettus.com/redmine/ettus/projects/public/wiki/U2binaries. I > specifically used the "Raw Ethernet" FPGA image (rel-20100603) and the > "Raw Ethernet, No WBX or XCVR" Firmware image (Jun 08, 2010). > > When I power up the USRP2, all the lights first glow, followed by only > D and F staying lit. The green light on the ethernet is also lit. > However, find_usrps returns with "No USRP2 found". > > I ran it with wireshark. The command does send a broadcast ping packet > from the machine but receives no response. The orange light briefly > flashes at the time the packet is sent, both on the PC and on the USRP. > > I tried the command with both the ethernet cards on the machine after > running ethtool to ensure that the problem might not be due to some > autonegotiation settings etc. On one of the cards, it says the command > is not supported, but on the other, it says "rx unmodified, ignoring" > and "no pause parameters changed, aborting". > > I have tried the command with two different USRPs to try and eliminate > the possibility that it might be a USRP issue too. > > In summary, I have tried all combos of 2 different USRP2s with 2 > different ethernet cards with and without the daughter card connected. > But find_usrps does not work in any of the combos. > > At this point, I am not sure what to do. Maybe the problem is a > firmware/FPGA mismatch with the installed gnuradio software, but I am > not sure, and I don't know what versions to use if that is indeed the > problem. > > (The sketch of my problem seems similar to the one outlined by another > person here: > http://www.mail-archive.com/discuss-gnuradio@gnu.org/msg24002.html > except that I am _not_ using the WBX daughterboard. I also tried all > the above with and without my RFX2400 daughterboard connected in case > the problem was with the RFX2400). > > Any suggestions about how to fix the problem? > > Thanks, > Rahul > > > (re-sent from 9 pm, 6/30) > > > ___ > Discuss-gnuradio mailing list > Discuss-gnuradio@gnu.org > http://lists.gnu.org/mailman/listinfo/discuss-gnuradio > Assuming the usual suspects have been resolved (1) ip address on nic is set to 192.168.10.1, (2) no firewall issues and (3) nic is active Then I suspect the newest release of uhd and gnuradio may have not been applied. Replacing just the firmware on my system resulted in no device found. To install the new release of software got to http://ettus-apps.sourcerepo.com/redmine/ettus/projects/uhd/wiki and get the uhd source and the newest gnuradio. hope that helps Criss Swaim ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org http://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: [Discuss-gnuradio] UHD CMake error Boost not found
On 6/28/2010 3:42 PM, Josh Blum wrote: > On 06/28/2010 02:14 PM, Criss Swaim wrote: >> I am trying to upgrade the USRP2 firmware/fpga and UHD code to the most >> current releases. >> Fedora 11, 64-bit >> >> After upgrading the firmware/fpga (and before installing the new UHD >> code) I can ping the USRP2, but the uhd_find_devices returns 'No USRP >> found' >> >> I pulled the newest UHD code from the repository and when I run the >> cmake ../ - receive an error that the boost libraries cannot be found. >> After comparing the CMakelist.txt files between the old and new, the >> only difference around the find boost was the new required boost 1.36 >> and the old required 1.37. Even after changing the required version to >> 1.37, the new cmake fails. > > When you run cmake try: > > cmake -DBOOST_LIBRARYDIR=/usr/lib64 ../ > > Ive seen this on a few fedora 64 bit machines, but the problem doesnt > seem consistent... Anyway, let me know if that fixes it for you. > > -Josh That corrected the problem. System compiles and can access the USRPthanks > > > ___ > Discuss-gnuradio mailing list > Discuss-gnuradio@gnu.org > http://lists.gnu.org/mailman/listinfo/discuss-gnuradio > ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org http://lists.gnu.org/mailman/listinfo/discuss-gnuradio
[Discuss-gnuradio] UHD CMake error Boost not found
I am trying to upgrade the USRP2 firmware/fpga and UHD code to the most current releases. Fedora 11, 64-bit After upgrading the firmware/fpga (and before installing the new UHD code) I can ping the USRP2, but the uhd_find_devices returns 'No USRP found' I pulled the newest UHD code from the repository and when I run the cmake ../ - receive an error that the boost libraries cannot be found. After comparing the CMakelist.txt files between the old and new, the only difference around the find boost was the new required boost 1.36 and the old required 1.37. Even after changing the required version to 1.37, the new cmake fails. Any suggestions would be appreciated. -- Criss Swaim csw...@tpginc.net cell: 505.301.5701 ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org http://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: [Discuss-gnuradio] UHD accessing metadata
> gnuradio does not offer any kind of sample decoration. So, the > metadata is present in the simple uhd source block, and dropped on its > way out of the block. > > You would need to make some kind of c++ app in gnuradio or maybe > standalone to deal with the metatdata. I dont know another way. > > Also, the uhd comes with an example app called rx_timed_samples.cpp, > you may want to take a look at this to see how the api deals with the > metadata. I will expand this example to include clock configuration > when i get the chance as well. > > I hope that helps, > -Josh > > On 05/06/2010 12:35 PM, Criss Swaim wrote: > > I have been working with the uhd version of gnuradio and have attached a > gps unit to the usrp2 to set the pps. So far, I have not figured out > how to extract the time (secs and nsecs) metadata for a signal sample. > This is mostly due to my limited C++ coding skills. Has anyone > successfully extracted the time metadata and would you be willing to > share the C++ code to show me how to capture and extract the time > metadata. > > > Josh: for discussion, consider a model: uhd-source-block ---> custom filter block ->custom sink (sample file) For clarification, when you say the metadata is present in the uhd source block but dropped on the way out, this implies that if I want the time associated with a sample, I need to modify the uhd source block with the code to pull the secs/nsecs. Is this correct? Currently, I have a custom filter block attached to the USRP source block, but your comment indicates that the metadata is not present when the sample is passed to my custom filter block. If that understanding is correct, then I must capture the time data in a new custom uhd block, push it to an array with some identifier and then in my existing custom block, re-attach the metadata with the sample based on the key or position in the array. Am I understanding this correctly? The example is quite helpful and I have spent much of the weekend working through it. thanks -- Criss Swaim csw...@tpginc.net ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org http://lists.gnu.org/mailman/listinfo/discuss-gnuradio
[Discuss-gnuradio] UHD accessing metadata
I have been working with the uhd version of gnuradio and have attached a gps unit to the usrp2 to set the pps. So far, I have not figured out how to extract the time (secs and nsecs) metadata for a signal sample. This is mostly due to my limited C++ coding skills. Has anyone successfully extracted the time metadata and would you be willing to share the C++ code to show me how to capture and extract the time metadata. -- Criss Swaim csw...@tpginc.net cell: 505.301.5701 ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org http://lists.gnu.org/mailman/listinfo/discuss-gnuradio
