Hi Dirk, Well, if you have a let's say 2 kHz uncertainty in the frequency of your pulse, I'd really start very simple. Looking at your plots, I think we can sufficiently suppress the noise simply by low-pass filtering:
* Use a Low Pass filter (real taps) with a cutoff of 1 kHz – that will let everything from -1 kHz to +1 kHz around your center frequency through. You can use the "freq xlating FIR" block to first shift by a given frequency and then apply the filter; might make things easier. * Detect power by putting your complex samples through a complex mag^2 block. If the output of that looks somewhat OK, then you could e.g. low-pass filter the result to get rid of noise pulses that are too short, for example :) By the way, it might be cool to put up a short recording (ie. a file generated using the GNU Radio file sink) somewhere (zipped, e.g. on Google Drive). I haven't worked with that myself, but maybe putting it on SigIDwiki[1] might be beneficial for people encountering the wildlife tracker signal later on. I actually think your project is pretty cool – short warning: if this works out, I'll try to convince you to write a short article for the GNU Radio blog! I'd imagine it being about how you approached the problem and what you've found, with emphasis on all the problems you had because of GNU Radio and maybe also DSP. We surely could use a few more "oh, GNU Radio is awesome because it works for everyone" articles (like [2]), but what we far more severely lack is stories of people that actually made progress, but also struggled, because they didn't know GR for years before writing a blog post. One of the problems that we face (and that was a very prominent theme at the Panel at FOSDEM [3]) is that we don't really know how to help people that are just approaching GNU Radio, and aren't already familiar with a lot of the stuff that we've gotten used to. Happy tracking, Marcus [1] http://www.sigidwiki.com/wiki/Adding_a_Signal_Entry [2] http://gnuradio.org/blog/filtering-time-series-data__elemental-building-blocks/ [3] https://fosdem.org/2017/schedule/event/sdr_panel/ ; don't know – maybe we'll have recordings of that. On 02/06/2017 09:39 PM, Dirk Gorissen wrote: > Thanks Marcus & Martin for the responses. > > To clarify, Im working on a wildlife tracking problem but from the air > (drone). Im purely interested in finding out if the pulse (which gets > transmitted at a fixed interval of 1500ms) occurred or not. If it did, > I know Im within some range of the animal of interest. I have no > control over the transmitter and no further technical details > (unfortunately). I did take some screenshots to give you an idea what > it looks like: > > https://goo.gl/photos/Y3Ea6kJo1ewrcDYM8 (taken with tx at close range > to static antenna) > > I expect quite a bit of (fairly uniform) background noise from other > electronics to deal with and there are hardware things that can be > improved. But for the purposes of this thread I want to ensure Im > doing the right things on the DSP side. > > And yes, Im pretty much a DSP novice but happy to learn. So be gentle :) > > >So, my understanding is that your SDR device first downconverts your > 150.22 > >MHz signal to complex baseband, is that right? > > Yes, so think something like an rtlsdr or airspy. > > About the FFT question, thanks I got a bit confused. I now understand > that if I want FFT over a longer timeframe I should just take a larger > size and I can vary the sampling rate (e.g., via decimation) to get > the resolution (Hz per bin) I require. > > Im now wondering if there is a founded way to pick an optimal FFT size > given my pulse is only 10ms long. Im guessing it should not be much > longer than the 10ms equivalent but maybe there is a noise tradeoff (?). > > In any case I agree this is a time based phenomenon so a time domain > approach should likely be the main one. > > So it seems I should at least be trying an FIR filter to do cross > correlation. Not quite clear how I would actually do that in practise > with the gr block though. How would I provide the second input (the > synthetic pulse)? > > I have come across matched filters before but so far failed to bridge > the gap from theory to code. This is partly the reason I came to gnuradio. > > >You can implement a limited-length autocorrelation/crosscorrelation > relatively > >easily; we can talk about how that would look like, but my gut feeling is > >that this > >might be something that might not make too much sense in your specific > use > >case. > > Not too sure myself tbh, but happy to take your lead. > > >it's possible cyclostationary estimation methods might be helpful here. > > So I googled this and poked around a bit > on https://cyclostationary.blog. Looks really interesting but out of > my depth here.. > > Thanks for the tip on gr-fosphor. > > Cheers > Dirk > > On 5 February 2017 at 12:06, Marcus Müller <marcus.muel...@ettus.com > <mailto:marcus.muel...@ettus.com>> wrote: > > Hi Dirk, > > nice to have you around, welcome to GNU Radio! I don't know your > level of DSP knowledge, so please excuse if I either throw too > many high-level concepts at you or assume you could want to read > up on something that you already know. If something in my reply is > unclear, please don't hesitate to ask for clarification. > > So, my understanding is that your SDR device first downconverts > your 150.22 MHz signal to complex baseband, is that right? > > So, what is the objective here? You say you need to /detect/ the > pulse, but for what purpose? Is it just about detecting the > presence of the pulse, or is it used for eg. time detection? > > Your filter->decimate approach sounds very reasonable; I'm not > 100% convinced by using the FFT for something that is concentrated > in *time* domain, but that might depend on the purpose mentioned > above. > >> 2) Im somewhat confused about the FFT block if I just pipe the SDR >> straight into it. The FFT size is set to 1024 and the window is set to >> "window.blackmanharris(1024)". So Im assuming the FFT just applies one >> window (?) and outputs 1024 bins. However, how many samples are >> accumulated before the FFT is run? I would have assumed I can control >> that too. And if so, should I best be doing this every 50ms, 500ms, >> 2000ms, ..? > I'm not sure I understand the question. An FFT is simply an DFT. > It's a mapping of N-dimensional vector to N-dimensional vector, > $\text{DFT}_N: \mathbb C^N \mapsto \mathbb C^N$. The window is > multiplied point-wise with the input vector prior to the DFT to > avoid spectral leakage. > > There's no accumulation involved anywhere. > >> 3) I can use the rational resampling block to bring the sample rate >> down to 48khz so I can use the audio sink. From that I can still hear >> the pulse even if it is not visible in the spectrum (gui sink). Im >> assuming this is just because the plotting cannot keep up? > Maybe, or maybe the spectrum sink really isn't the right tool to > visualize a pulse! Again, we might want to discuss what this pulse > is and what it's used for. > >> 4) In the time domain I guess I can generate a synthetic pulse of the >> same length / frequency and then cross correlate. > Hm, but correlating a signal with a known fixed sequence is, > mathematically, a convolution. > > That is identical to doing FIR filtering – in fact, if we consider > the shape of your pulse as what is often referred to as /pulse > shape filter/, then that filter in the receiver would simply be > the /matched filter/ to that. > >> Not obvious to me >> how to generate the required pulse in gnuradio though (would a >> continuous signal work?). > "Continuous" would mean you'd do an infinite-length correlation, > so that's not 100% possible. >> I also notice there are no built in >> (auto)correlation blocks? > Hm, but an autocorrelation would take a complete signal, shift it > by all possible shifts and calculating the dot product between the > shifted version and the unshifted, right? That would require to > have the complete signal at once. > > But GNU Radio is a streaming architecture, so that can't work. > > You can implement a limited-length > autocorrelation/crosscorrelation relatively easily; we can talk > about how that would look like, but my gut feeling is that this > might be something that might not make too much sense in your > specific use case. > >> I found the "correlation estimator" but not >> clear how to use it. As for dealing with the frequency uncertainty >> problem. Does one just try correlating with different freuencies and >> pick the best one? Or what is the good thing to do here given I may >> also have to deal with quite a bit of noise. > As a gut feeling: you don't really care about whether the pulse is > *exactly* at a certain frequency (it's absolutely not normal for > wireless receivers to know the exact frequency a priori), but when > it happens. So we might want to discuss the kind of pulse, and > kind of noise we're talking about. > > As a further gut feeling: I think your autocorrelation question > indicates you might be on a very good track – it's possible > cyclostationary estimation methods might be helpful here. > > > Best regards, > > Marcus > > > > On 02/04/2017 11:39 PM, Dirk Gorissen wrote: >> Fist of all, while Im a newbie to (gnu)radio, congrats to the dev team >> for a great piece of software. >> >> My question is about the need to detect a weak, noisy, short (10ms) >> pulse that occurs every 1.5 seconds. It is transmitted at a particular >> frequency (e.g., 150.22 MHz) but in practise I have found this can >> vary by as much as +/- 500Hz. There is no modulation, it is simply an >> on/off keyed pulse. >> >> Say I have an SDR generating data at 2.5 MSPS. I have so far been >> experimenting with standard scipy/numpy routines to collect a batch of >> samples, perform a FFT (freq domain) and do a cross correlation (time >> domain). However, Im by no means a dsp guy and would like to leverage >> gnuradio as much as I can. I have been poking a bit but have some >> basic questions. >> >> 1) 2.5 Msps gives me way more bandwidth than I neeed. Assuming, for >> now, I only care about a single pulse frequency I really only need >> ~1khz bandwidth. In the frequency domain I can directly decimate down >> (with a big factor) to the 1-2 khz range using the low pass filter >> block, do an fft, and look for peaks. Is that the right approach? >> >> 2) Im somewhat confused about the FFT block if I just pipe the SDR >> straight into it. The FFT size is set to 1024 and the window is set to >> "window.blackmanharris(1024)". So Im assuming the FFT just applies one >> window (?) and outputs 1024 bins. However, how many samples are >> accumulated before the FFT is run? I would have assumed I can control >> that too. And if so, should I best be doing this every 50ms, 500ms, >> 2000ms, ..? >> >> 3) I can use the rational resampling block to bring the sample rate >> down to 48khz so I can use the audio sink. From that I can still hear >> the pulse even if it is not visible in the spectrum (gui sink). Im >> assuming this is just because the plotting cannot keep up? >> >> 4) In the time domain I guess I can generate a synthetic pulse of the >> same length / frequency and then cross correlate. Not obvious to me >> how to generate the required pulse in gnuradio though (would a >> continuous signal work?). I also notice there are no built in >> (auto)correlation blocks? I found the "correlation estimator" but not >> clear how to use it. As for dealing with the frequency uncertainty >> problem. Does one just try correlating with different freuencies and >> pick the best one? Or what is the good thing to do here given I may >> also have to deal with quite a bit of noise. >> >> Any guidance appreciated. >> >> Many thanks, >> >> Dirk >> >> _______________________________________________ >> Discuss-gnuradio mailing list >> Discuss-gnuradio@gnu.org <mailto:Discuss-gnuradio@gnu.org> >> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio >> <https://lists.gnu.org/mailman/listinfo/discuss-gnuradio> > _______________________________________________ Discuss-gnuradio > mailing list Discuss-gnuradio@gnu.org > <mailto:Discuss-gnuradio@gnu.org> > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio > <https://lists.gnu.org/mailman/listinfo/discuss-gnuradio> > > -- > _________________________________________ Dr. Dirk Gorissen Mob: > +44-7763-806-809 Web: dirkgorissen.com <http://dirkgorissen.com> > Skype: dirk.gorissen Twitter : twitter.com/dirkgor > <http://twitter.com/dirkgor> LinkedIn: linkedin.com/in/dirkgorissen > <http://linkedin.com/in/dirkgorissen>
_______________________________________________ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
