Re: [Discuss-gnuradio] Crash of gqrx in gr block deallocator when building gnuradio w/o log4cpp on Ubuntu 16-17
Alrighty, thanks for another data point. If I end up trying to debug further it'll be good to know. On Jul 14, 2017 2:54 PM, "Geof Nieboer" wrote: > Doing this from memory, but as I recall the corruption happens because > iqbal/gqrx libraries believe sizeof(block) is different than what > gnuradio's libraries believe. So one library malloc's a different amount > of memory than the other library uses and frees. If during debugging you > call sizeof() from different parts of the call stack, it should become > clear. > > The problem I found is that if logger.h is included without ENABLE_GR_LOG, > it defines logger_ptr as a void*, whereas with it enabled and without > log4cpp it defines it as a std::string... and those are not guaranteed to > be the same size. > > Now having said that, this was on Windows, so perhaps on GCC > sizeof(std::string) == sizeof(void*) and you are facing a completely > different issue. But it sounds likely it's the same. Like yourself, I > didn't have time to dig any further as to what code change caused this > issue to surface at the time. > > Geof > > > On Fri, Jul 14, 2017 at 11:07 AM, Anon Lister > wrote: > >> Hmm, so in my last test I completely skipped gr-iqbal. I had gnuradio, >> osmosdr with only file playback and python modules enabled, and gqrx. >> You'll still get a crash if you start file playback, then goto demod and >> change the type to nbfm, then wbfm. Believe it hits some cout in gnuradio >> like "Resampling audio from 96000 -> 48000." >> >> Given that line is just a cout, I'm geussing whatever logging facility is >> hijacking cout and leaves it in a bad state after some destructor is >> called. >> >> Also, if you build with -fsanitize=address, you do see that there's a >> memory corruption way before that destructor gets called, but I'm not sure >> if it's related. >> >> I gotta submit a PR for something in gr-dtv this weekend, so I might dig >> in more while I have things setup. >> >> >> On Jul 14, 2017 9:50 AM, "Geof Nieboer" wrote: >> >> Don, >> >> I ran into the same exact issue while updating the windows build scripts. >> >> >> Another fix is to manually define ENABLE_GR_LOG during build of iqbal and >> gqrx to work around the issue without installing log4cpp. It appears to >> just affect those two so far. The windows build does not currently include >> log4cpp on 3.7.x builds. >> >> Geof >> >> >> On Thu, Jul 13, 2017 at 10:55 AM, Anon Lister >> wrote: >> >>> Hey all, just an FYI, >>> >>> I don't have the time to go figure out what exactly is causing it, but >>> if you build from source with the log4cpp dep missing after a merged PR in >>> April/May(that fixed some log4cpp headers), then in certain circumstances, >>> you will get a segfault, due to a heap buffer overflow. >>> >>> There are a few code paths that crash. If gr-iqbal is installed, it will >>> crash right on opening gqrx. If not it will crash when switching demod >>> type. >>> >>> I believe both cases a call to a destructor on at least 1 block, >>> followed by some kind of printed message is the culprit. >>> >>> Took me a few days to find the "fix", as I thought it was related to >>> some new hardware I had, but then upon pulling I was able to reproduce on >>> my normal workstation and it went away when I installed log4cpp and >>> recompiled GR. Just though I'd throw this out Incase anyone else was having >>> the issue. >>> >>> I'll try to file a bug report, but ideally I would like to include more >>> information and I don't have time for that atm, and from what I understand >>> it might be OBE when 3.8 comes out as that's getting added as a required >>> dependency. >>> >>> I have not reproduced the crash in anything besides gqrx, but it seems >>> pretty solid that it's a bug in GR. I bet other apps that reconfigure >>> flowgraphs like shinysdr would trigger it as well. >>> >>> -Don >>> >>> ___ >>> Discuss-gnuradio mailing list >>> Discuss-gnuradio@gnu.org >>> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio >>> >>> >> >> > ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: [Discuss-gnuradio] Crash of gqrx in gr block deallocator when building gnuradio w/o log4cpp on Ubuntu 16-17
Doing this from memory, but as I recall the corruption happens because iqbal/gqrx libraries believe sizeof(block) is different than what gnuradio's libraries believe. So one library malloc's a different amount of memory than the other library uses and frees. If during debugging you call sizeof() from different parts of the call stack, it should become clear. The problem I found is that if logger.h is included without ENABLE_GR_LOG, it defines logger_ptr as a void*, whereas with it enabled and without log4cpp it defines it as a std::string... and those are not guaranteed to be the same size. Now having said that, this was on Windows, so perhaps on GCC sizeof(std::string) == sizeof(void*) and you are facing a completely different issue. But it sounds likely it's the same. Like yourself, I didn't have time to dig any further as to what code change caused this issue to surface at the time. Geof On Fri, Jul 14, 2017 at 11:07 AM, Anon Lister wrote: > Hmm, so in my last test I completely skipped gr-iqbal. I had gnuradio, > osmosdr with only file playback and python modules enabled, and gqrx. > You'll still get a crash if you start file playback, then goto demod and > change the type to nbfm, then wbfm. Believe it hits some cout in gnuradio > like "Resampling audio from 96000 -> 48000." > > Given that line is just a cout, I'm geussing whatever logging facility is > hijacking cout and leaves it in a bad state after some destructor is > called. > > Also, if you build with -fsanitize=address, you do see that there's a > memory corruption way before that destructor gets called, but I'm not sure > if it's related. > > I gotta submit a PR for something in gr-dtv this weekend, so I might dig > in more while I have things setup. > > > On Jul 14, 2017 9:50 AM, "Geof Nieboer" wrote: > > Don, > > I ran into the same exact issue while updating the windows build scripts. > > Another fix is to manually define ENABLE_GR_LOG during build of iqbal and > gqrx to work around the issue without installing log4cpp. It appears to > just affect those two so far. The windows build does not currently include > log4cpp on 3.7.x builds. > > Geof > > > On Thu, Jul 13, 2017 at 10:55 AM, Anon Lister > wrote: > >> Hey all, just an FYI, >> >> I don't have the time to go figure out what exactly is causing it, but if >> you build from source with the log4cpp dep missing after a merged PR in >> April/May(that fixed some log4cpp headers), then in certain circumstances, >> you will get a segfault, due to a heap buffer overflow. >> >> There are a few code paths that crash. If gr-iqbal is installed, it will >> crash right on opening gqrx. If not it will crash when switching demod >> type. >> >> I believe both cases a call to a destructor on at least 1 block, followed >> by some kind of printed message is the culprit. >> >> Took me a few days to find the "fix", as I thought it was related to some >> new hardware I had, but then upon pulling I was able to reproduce on my >> normal workstation and it went away when I installed log4cpp and recompiled >> GR. Just though I'd throw this out Incase anyone else was having the issue. >> >> I'll try to file a bug report, but ideally I would like to include more >> information and I don't have time for that atm, and from what I understand >> it might be OBE when 3.8 comes out as that's getting added as a required >> dependency. >> >> I have not reproduced the crash in anything besides gqrx, but it seems >> pretty solid that it's a bug in GR. I bet other apps that reconfigure >> flowgraphs like shinysdr would trigger it as well. >> >> -Don >> >> ___ >> Discuss-gnuradio mailing list >> Discuss-gnuradio@gnu.org >> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio >> >> > > ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: [Discuss-gnuradio] Crash of gqrx in gr block deallocator when building gnuradio w/o log4cpp on Ubuntu 16-17
Hmm, so in my last test I completely skipped gr-iqbal. I had gnuradio, osmosdr with only file playback and python modules enabled, and gqrx. You'll still get a crash if you start file playback, then goto demod and change the type to nbfm, then wbfm. Believe it hits some cout in gnuradio like "Resampling audio from 96000 -> 48000." Given that line is just a cout, I'm geussing whatever logging facility is hijacking cout and leaves it in a bad state after some destructor is called. Also, if you build with -fsanitize=address, you do see that there's a memory corruption way before that destructor gets called, but I'm not sure if it's related. I gotta submit a PR for something in gr-dtv this weekend, so I might dig in more while I have things setup. On Jul 14, 2017 9:50 AM, "Geof Nieboer" wrote: Don, I ran into the same exact issue while updating the windows build scripts. Another fix is to manually define ENABLE_GR_LOG during build of iqbal and gqrx to work around the issue without installing log4cpp. It appears to just affect those two so far. The windows build does not currently include log4cpp on 3.7.x builds. Geof On Thu, Jul 13, 2017 at 10:55 AM, Anon Lister wrote: > Hey all, just an FYI, > > I don't have the time to go figure out what exactly is causing it, but if > you build from source with the log4cpp dep missing after a merged PR in > April/May(that fixed some log4cpp headers), then in certain circumstances, > you will get a segfault, due to a heap buffer overflow. > > There are a few code paths that crash. If gr-iqbal is installed, it will > crash right on opening gqrx. If not it will crash when switching demod > type. > > I believe both cases a call to a destructor on at least 1 block, followed > by some kind of printed message is the culprit. > > Took me a few days to find the "fix", as I thought it was related to some > new hardware I had, but then upon pulling I was able to reproduce on my > normal workstation and it went away when I installed log4cpp and recompiled > GR. Just though I'd throw this out Incase anyone else was having the issue. > > I'll try to file a bug report, but ideally I would like to include more > information and I don't have time for that atm, and from what I understand > it might be OBE when 3.8 comes out as that's getting added as a required > dependency. > > I have not reproduced the crash in anything besides gqrx, but it seems > pretty solid that it's a bug in GR. I bet other apps that reconfigure > flowgraphs like shinysdr would trigger it as well. > > -Don > > ___ > Discuss-gnuradio mailing list > Discuss-gnuradio@gnu.org > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio > > ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: [Discuss-gnuradio] Crash of gqrx in gr block deallocator when building gnuradio w/o log4cpp on Ubuntu 16-17
Don, I ran into the same exact issue while updating the windows build scripts. Another fix is to manually define ENABLE_GR_LOG during build of iqbal and gqrx to work around the issue without installing log4cpp. It appears to just affect those two so far. The windows build does not currently include log4cpp on 3.7.x builds. Geof On Thu, Jul 13, 2017 at 10:55 AM, Anon Lister wrote: > Hey all, just an FYI, > > I don't have the time to go figure out what exactly is causing it, but if > you build from source with the log4cpp dep missing after a merged PR in > April/May(that fixed some log4cpp headers), then in certain circumstances, > you will get a segfault, due to a heap buffer overflow. > > There are a few code paths that crash. If gr-iqbal is installed, it will > crash right on opening gqrx. If not it will crash when switching demod > type. > > I believe both cases a call to a destructor on at least 1 block, followed > by some kind of printed message is the culprit. > > Took me a few days to find the "fix", as I thought it was related to some > new hardware I had, but then upon pulling I was able to reproduce on my > normal workstation and it went away when I installed log4cpp and recompiled > GR. Just though I'd throw this out Incase anyone else was having the issue. > > I'll try to file a bug report, but ideally I would like to include more > information and I don't have time for that atm, and from what I understand > it might be OBE when 3.8 comes out as that's getting added as a required > dependency. > > I have not reproduced the crash in anything besides gqrx, but it seems > pretty solid that it's a bug in GR. I bet other apps that reconfigure > flowgraphs like shinysdr would trigger it as well. > > -Don > > ___ > Discuss-gnuradio mailing list > Discuss-gnuradio@gnu.org > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio > > ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
[Discuss-gnuradio] Crash of gqrx in gr block deallocator when building gnuradio w/o log4cpp on Ubuntu 16-17
Hey all, just an FYI, I don't have the time to go figure out what exactly is causing it, but if you build from source with the log4cpp dep missing after a merged PR in April/May(that fixed some log4cpp headers), then in certain circumstances, you will get a segfault, due to a heap buffer overflow. There are a few code paths that crash. If gr-iqbal is installed, it will crash right on opening gqrx. If not it will crash when switching demod type. I believe both cases a call to a destructor on at least 1 block, followed by some kind of printed message is the culprit. Took me a few days to find the "fix", as I thought it was related to some new hardware I had, but then upon pulling I was able to reproduce on my normal workstation and it went away when I installed log4cpp and recompiled GR. Just though I'd throw this out Incase anyone else was having the issue. I'll try to file a bug report, but ideally I would like to include more information and I don't have time for that atm, and from what I understand it might be OBE when 3.8 comes out as that's getting added as a required dependency. I have not reproduced the crash in anything besides gqrx, but it seems pretty solid that it's a bug in GR. I bet other apps that reconfigure flowgraphs like shinysdr would trigger it as well. -Don ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
