Re: [Discuss-gnuradio] crashes, memory errors and valgrind
Tom Rondeau wrote on 2012-06-06 01:12: > On Tue, Jun 5, 2012 at 10:06 AM, Patrick Strasser > wrote: >> Tom Rondeau wrote on 2012-06-04 14:18: >>> On Sun, Jun 3, 2012 at 2:22 PM, Patrick Strasser >>> wrote: Full valgrind log in http://pastebin.com/7GCs3bWy >> What seems strange to me is the allocation of a buffer of a non by 8 >> dividable size, that is accessed in blocks of 8 bytes. So the last read >> always either does not touch the end of the buffer, or it reads beyond >> the end. > > Sorry, Patrick, I think there was a typo or something in your first > sentence, and I'm not sure I understood. But from the gist of it, yes, > if the items used in a block have size 8, then the buffer allocation > should be sized enough to handle an integer number of items. This is > true because the scheduler sends the block noutput_items, which are > the number of items to process (so the buffer size would be > 8*noutput_items). To make it more clear: >From the valgrind log: ==19688== Invalid read of size 8 [...] ==19688== Address 0x1a200600 is 1 bytes after a block of size 447 alloc'd [...] ==19688==by 0x4F7AF67: gr_fir_fff_simd::set_taps(std::vector > const&) (gr_fir_fff_simd.cc:88) [...] ==19688==by 0x4F8DF06: gr_make_rational_resampler_base_fff(unsigned int, unsigned int, std::vector > const&) (gr_rational_resampler_base_fff.cc:45) I do not completely understand where this number is calculated, but a buffer of size 447 is allocated, and later accessed in steps of 8. Btw even if the buffer was a multiple of 8, the read would be out of bounds. Anyway, reading out of bounds may trigger a segfault if you are lucky, if you are not, it goes unnoticed and you get wrong/weird results. I wonder if this stems from an error in the application. Is it possible to mis-use the API to trigger something like this that would not happen if everything was used rigth? I'm sorry if my mails are not always completely clear and understandable, still trying to improve my English on technical topics that are new to me ;-) Regards Patrick -- Engineers motto: cheap, good, fast: choose any two Patrick Strasser Student of Telematics, Graz Univ. of Technology, Austria ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: [Discuss-gnuradio] crashes, memory errors and valgrind
On Tue, Jun 5, 2012 at 10:06 AM, Patrick Strasser wrote: > Tom Rondeau wrote on 2012-06-04 14:18: >> On Sun, Jun 3, 2012 at 2:22 PM, Patrick Strasser >> wrote: >>> Hi Tom, >>> >>> Tom Rondeau wrote on 2012-06-03 17:12: On Sat, Jun 2, 2012 at 5:50 AM, Patrick Strasser Patrick, It looks like you're problem is in the rational_resampler code. I wonder if there's something about the resampling rate being used that's causing something to go out of bounds here. Can you dig into the code and figure out what interpolation and decimation rates are being used? >>> >>> Interpolation is 1, decimation is 2. > >>> Full valgrind log in >>> http://pastebin.com/7GCs3bWy >> >> I was hoping you'd say the interpolation and/or decimation were some >> ridiculously large numbers. Since the block is only actually >> decimating, could you replace it in the code with an fir_filter_fff >> (or fft_filter_fff) just for testing purposes? That'll help us see if >> it's the rational resampler itself or something more general. > > What seems strange to me is the allocation of a buffer of a non by 8 > dividable size, that is accessed in blocks of 8 bytes. So the last read > always either does not touch the end of the buffer, or it reads beyond > the end. Sorry, Patrick, I think there was a typo or something in your first sentence, and I'm not sure I understood. But from the gist of it, yes, if the items used in a block have size 8, then the buffer allocation should be sized enough to handle an integer number of items. This is true because the scheduler sends the block noutput_items, which are the number of items to process (so the buffer size would be 8*noutput_items). > I'm a little sparse at time until Sunday evening. I tried to write some > mini program starting from dial_tone.cc which uses a filter, have to > convince cmake to compile it... learning ;-) > > Regards > > Patrick Thanks. Let us know how it goes. Tom ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: [Discuss-gnuradio] crashes, memory errors and valgrind
Tom Rondeau wrote on 2012-06-04 14:18: > On Sun, Jun 3, 2012 at 2:22 PM, Patrick Strasser > wrote: >> Hi Tom, >> >> Tom Rondeau wrote on 2012-06-03 17:12: >>> On Sat, Jun 2, 2012 at 5:50 AM, Patrick Strasser >>> Patrick, >>> >>> It looks like you're problem is in the rational_resampler code. I >>> wonder if there's something about the resampling rate being used >>> that's causing something to go out of bounds here. Can you dig into >>> the code and figure out what interpolation and decimation rates are >>> being used? >> >> Interpolation is 1, decimation is 2. >> Full valgrind log in >> http://pastebin.com/7GCs3bWy > > I was hoping you'd say the interpolation and/or decimation were some > ridiculously large numbers. Since the block is only actually > decimating, could you replace it in the code with an fir_filter_fff > (or fft_filter_fff) just for testing purposes? That'll help us see if > it's the rational resampler itself or something more general. What seems strange to me is the allocation of a buffer of a non by 8 dividable size, that is accessed in blocks of 8 bytes. So the last read always either does not touch the end of the buffer, or it reads beyond the end. I'm a little sparse at time until Sunday evening. I tried to write some mini program starting from dial_tone.cc which uses a filter, have to convince cmake to compile it... learning ;-) Regards Patrick -- Engineers motto: cheap, good, fast: choose any two Patrick Strasser Student of Telematics, Graz Univ. of Technology, Austria ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: [Discuss-gnuradio] crashes, memory errors and valgrind
On Mon, Jun 4, 2012 at 2:18 PM, Tom Rondeau wrote: > On Sun, Jun 3, 2012 at 2:22 PM, Patrick Strasser > wrote: >> Hi Tom, >> >> Tom Rondeau wrote on 2012-06-03 17:12: >>> On Sat, Jun 2, 2012 at 5:50 AM, Patrick Strasser >>> Patrick, >>> >>> It looks like you're problem is in the rational_resampler code. I >>> wonder if there's something about the resampling rate being used >>> that's causing something to go out of bounds here. Can you dig into >>> the code and figure out what interpolation and decimation rates are >>> being used? >> >> Interpolation is 1, decimation is 2. >> >> I compiled GNU Radio branch v3.6.0 with >> $ cmake -DCMAKE_BUILD_TYPE=Debug .. >> which resulted in a compile with flags -g -O2 . >> I tried to track what's happening in gdb, in gr_fir_fff_simd::set_taps, >> but -O2 was not helping, a lot optimized out, loops unrolled. >> >> Full valgrind log in >> http://pastebin.com/7GCs3bWy >> >> regards >> >> Patrick > > I was hoping you'd say the interpolation and/or decimation were some > ridiculously large numbers. Since the block is only actually > decimating, could you replace it in the code with an fir_filter_fff > (or fft_filter_fff) just for testing purposes? That'll help us see if > it's the rational resampler itself or something more general. Patrick: You can now try my own osmosdr branch from https://github.com/csete/gqrx I have made many changes since I told you to stay away few days ago and I think it is slightly usable now. This code adds proper multirate support by using the arbitrary PFB resamplers on both the quadrature and the audio end. It also takes advantage of the device discovery API in gr-osmosdr so you really just plug in your devices and go :-) It still works very well with the Funcube Dongle and is supposed to work with UHD devices but I really haven't had time to debug that part yet. Gr-osmosdr source works though with UHD devices in GRC, so it's a gqrx issue if it doesn't work. Concerning the original topic of this thread: I can't dismiss that it's a rarely seen bug in the GR code, but keep in mind that the fork was made when much of my code assumed 96ksps input rate and 48ksps audio rate. Alex ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: [Discuss-gnuradio] crashes, memory errors and valgrind
On Sun, Jun 3, 2012 at 2:22 PM, Patrick Strasser wrote: > Hi Tom, > > Tom Rondeau wrote on 2012-06-03 17:12: >> On Sat, Jun 2, 2012 at 5:50 AM, Patrick Strasser >> Patrick, >> >> It looks like you're problem is in the rational_resampler code. I >> wonder if there's something about the resampling rate being used >> that's causing something to go out of bounds here. Can you dig into >> the code and figure out what interpolation and decimation rates are >> being used? > > Interpolation is 1, decimation is 2. > > I compiled GNU Radio branch v3.6.0 with > $ cmake -DCMAKE_BUILD_TYPE=Debug .. > which resulted in a compile with flags -g -O2 . > I tried to track what's happening in gdb, in gr_fir_fff_simd::set_taps, > but -O2 was not helping, a lot optimized out, loops unrolled. > > Full valgrind log in > http://pastebin.com/7GCs3bWy > > regards > > Patrick I was hoping you'd say the interpolation and/or decimation were some ridiculously large numbers. Since the block is only actually decimating, could you replace it in the code with an fir_filter_fff (or fft_filter_fff) just for testing purposes? That'll help us see if it's the rational resampler itself or something more general. Tom ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: [Discuss-gnuradio] crashes, memory errors and valgrind
Patrick Strasser wrote on 2012-06-03 20:22: > Hi Tom, > > Tom Rondeau wrote on 2012-06-03 17:12: >> On Sat, Jun 2, 2012 at 5:50 AM, Patrick Strasser >> Patrick, >> >> It looks like you're problem is in the rational_resampler code. I >> wonder if there's something about the resampling rate being used >> that's causing something to go out of bounds here. I'm looking for some test program, that uses C++ and filters/decimators. The only examples in C++ I found are either dialtone, which does no processing, or programs that require some hardware that I do not have (fcd_nfm_rx.cc, Does anyone know of an small GNU Radio C++ application that does some filtering/resampling? Regards Patrick -- Engineers motto: cheap, good, fast: choose any two Patrick Strasser Student of Telematics, Graz Univ. of Technology, Austria ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: [Discuss-gnuradio] crashes, memory errors and valgrind
Hi Tom, Tom Rondeau wrote on 2012-06-03 17:12: > On Sat, Jun 2, 2012 at 5:50 AM, Patrick Strasser > Patrick, > > It looks like you're problem is in the rational_resampler code. I > wonder if there's something about the resampling rate being used > that's causing something to go out of bounds here. Can you dig into > the code and figure out what interpolation and decimation rates are > being used? Interpolation is 1, decimation is 2. I compiled GNU Radio branch v3.6.0 with $ cmake -DCMAKE_BUILD_TYPE=Debug .. which resulted in a compile with flags -g -O2 . I tried to track what's happening in gdb, in gr_fir_fff_simd::set_taps, but -O2 was not helping, a lot optimized out, loops unrolled. Full valgrind log in http://pastebin.com/7GCs3bWy regards Patrick -- Engineers motto: cheap, good, fast: choose any two Patrick Strasser Student of Telematics, Graz Univ. of Technology, Austria ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: [Discuss-gnuradio] crashes, memory errors and valgrind
On Sat, Jun 2, 2012 at 5:50 AM, Patrick Strasser wrote: > Hello list, > > I'm playing around with gqrx, an excellent example which great > standalone software can be built with gnuradio. > > I'm running phirsch's rtlsdr fork, do not have a Funcube Dongle or USRP > at hand, but rather a Terratex NOXON DAB stick - 25€ part does the trick > fine enough for me ;-) > > I'm running a self-compiled gnuradio rev3.6.0 on my Debian testing amd64 > machine. > > Sometime it crashes with messages like "glibc detected *** gqrx: > corrupted double-linked list". Firing up valgrind on it I find -besides > several uninitialized variables- messages of the kind: > > ==31294== Invalid read of size 8 > ==31294== at 0x4F782AC: float_dotprod_sse (in > /usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0) > [deleted some lines...] > ==31294== Address 0x1a211ba8 is 440 bytes inside a block of size 447 > alloc'd > ==31294== at 0x402894D: malloc (in > /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) > ==31294== by 0x4FB3C5E: malloc16Align (in > /usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0) > ==31294== by 0x4FB3CA9: calloc16Align (in > /usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0) > ==31294== by 0x4F75E27: gr_fir_fff_simd::set_taps(std::vector std::allocator > const&) (in > /usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0) > [deleted some lines...] > > Full trace of that error below, I can provide a full trace of a run if > needed. > > Usually the program runs stable, but occasionally it crashes. I played > around with the internal bandwidth to get better FM reception - 75kHz is > quite narrow for FM reception. This change seems to trigger the problems > much faster. I hardly can change any parameters while running without a > crash. > > I'm no expert, but it seems to me there is not enough memory allocated. > Maybe an off-by-one error? Did anyone see things like that or used > valgrind recently on gnuradio programs? What could I do to find out more > about this problem? > > Regards > > Patrick > > - > > ==31294== Invalid read of size 8 > ==31294== at 0x4F782AC: float_dotprod_sse (in > /usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0) > ==31294== by 0x4F8A27F: > gr_rational_resampler_base_fff::general_work(int, std::vectorhttps://lists.gnu.org/mailman/listinfo/discuss-gnuradio
[Discuss-gnuradio] crashes, memory errors and valgrind
Hello list, I'm playing around with gqrx, an excellent example which great standalone software can be built with gnuradio. I'm running phirsch's rtlsdr fork, do not have a Funcube Dongle or USRP at hand, but rather a Terratex NOXON DAB stick - 25€ part does the trick fine enough for me ;-) I'm running a self-compiled gnuradio rev3.6.0 on my Debian testing amd64 machine. Sometime it crashes with messages like "glibc detected *** gqrx: corrupted double-linked list". Firing up valgrind on it I find -besides several uninitialized variables- messages of the kind: ==31294== Invalid read of size 8 ==31294==at 0x4F782AC: float_dotprod_sse (in /usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0) [deleted some lines...] ==31294== Address 0x1a211ba8 is 440 bytes inside a block of size 447 alloc'd ==31294==at 0x402894D: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==31294==by 0x4FB3C5E: malloc16Align (in /usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0) ==31294==by 0x4FB3CA9: calloc16Align (in /usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0) ==31294==by 0x4F75E27: gr_fir_fff_simd::set_taps(std::vector > const&) (in /usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0) [deleted some lines...] Full trace of that error below, I can provide a full trace of a run if needed. Usually the program runs stable, but occasionally it crashes. I played around with the internal bandwidth to get better FM reception - 75kHz is quite narrow for FM reception. This change seems to trigger the problems much faster. I hardly can change any parameters while running without a crash. I'm no expert, but it seems to me there is not enough memory allocated. Maybe an off-by-one error? Did anyone see things like that or used valgrind recently on gnuradio programs? What could I do to find out more about this problem? Regards Patrick - ==31294== Invalid read of size 8 ==31294==at 0x4F782AC: float_dotprod_sse (in /usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0) ==31294==by 0x4F8A27F: gr_rational_resampler_base_fff::general_work(int, std::vector >&, std::vector >&, std::vector >&) (in /usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0) ==31294==by 0x4F45CBD: gr_block_executor::run_one_iteration() (in /usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0) ==31294==by 0x4F65AA8: gr_tpb_thread_body::gr_tpb_thread_body(boost::shared_ptr, int) (in /usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0) ==31294==by 0x4F60645: boost::detail::function::void_function_obj_invoker0, void>::invoke(boost::detail::function::function_buffer&) (in /usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0) ==31294==by 0x52BBCCD: boost::detail::thread_data >::run() (in /usr/local/lib/libgruel-3.6.0.so.0.0.0) ==31294==by 0x81EA928: ??? (in /usr/lib/libboost_thread.so.1.49.0) ==31294==by 0x6C08B4F: start_thread (pthread_create.c:304) ==31294==by 0x76959DC: clone (clone.S:112) ==31294== Address 0x1a211ba8 is 440 bytes inside a block of size 447 alloc'd ==31294==at 0x402894D: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==31294==by 0x4FB3C5E: malloc16Align (in /usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0) ==31294==by 0x4FB3CA9: calloc16Align (in /usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0) ==31294==by 0x4F75E27: gr_fir_fff_simd::set_taps(std::vector > const&) (in /usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0) ==31294==by 0x4F8A144: gr_rational_resampler_base_fff::install_taps(std::vector > const&) (in /usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0) ==31294==by 0x4F8A5B6: gr_rational_resampler_base_fff::gr_rational_resampler_base_fff(unsigned int, unsigned int, std::vector > const&) (in /usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0) ==31294==by 0x4F8A826: gr_make_rational_resampler_base_fff(unsigned int, unsigned int, std::vector > const&) (in /usr/local/lib/libgnuradio-core-3.6.0.so.0.0.0) ==31294==by 0x44A41D: resampler_ff::resampler_ff(unsigned int, unsigned int) (resampler_ff.cpp:71) ==31294==by 0x44A025: make_resampler_ff(unsigned int, unsigned int) (resampler_ff.cpp:33) ==31294==by 0x4141C9: receiver::receiver(std::string, std::string) (receiver.cpp:80) ==31294==by 0x41DBD9: MainWindow::MainWindow(QString, QWidget*) (mainwindow.cpp:64) ==31294==by 0x41D4D7: main (main.cpp:52) -- Engineers motto: cheap, good, fast: choose any two Patrick Strasser Student of Telematics, Graz Univ. of Technology, Austria ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
