Hi Dirk, in my experience SDR is a nice stuff but sometimes analog tech can help: try to lower total NF using a tuned LNA in front of you receiver... RTL dongles or other sampler are a bit deaf and noisy, and your signal seems very feeble.
... my 2 eurocent... ! Victor 2017-02-10 0:01 GMT+01:00 Dirk Gorissen <dgoris...@gmail.com>: > Hi Marcus, > > Thanks again. And yes, Id be happy to do a writeup if I get things > working with GnuRadio. I did a writeup of the first version I did of > my project (*) and happy to do a part two of the improved version when > finished (asap). Replies inline. > > >Well, if you have a let's say 2 kHz uncertainty in the frequency of your > pulse, I'd really start very simple. Looking at your plots, I think we can > sufficiently suppress the noise simply by low-pass filtering: > > Just note that the plots are posted are with the transmitter pretty > much sitting next to the antenna so this is a *very* strong signal, in > reality its much weaker. > > >* Use a Low Pass filter (real taps) with a cutoff of 1 kHz – that will > let everything from -1 kHz to +1 kHz around your center frequency through. > You can use the "freq xlating FIR" block to first shift by a given > frequency and >then apply the filter; might make things easier. > >* Detect power by putting your complex samples through a complex mag^2 > block. > > Thanks. I did this and the following observations > - I went SDR -> Freq Xlating FIR filter with 4x decimation -> low pass > filter block with 128x decimation and 1khz cutoff -> complex to mag2 > > - The Freq Xlating block was giving me an error unless I filled in > something for the Tap, some googleing led me to fill in > firdes.low_pass_2 and I just used the same filter settings as the > filter block. That made it happy but to be honest Im not sure why and > if that was the right thing to do. > > - I did the decimation somewhat arbitrarily in two steps as I read > somewhere that is better computationally, perhaps it should be more > even though > > - With the above I could see the peaks (yay!) but it gets harder as > the signal gets weaker. I plugged in the peak detector block (peak > detector 2 seems broken btw) but found its usage very unintuitive and > I didnt get it to mark the peaks in the way I wanted. Even though they > were pretty big it wouldn't catch many of them, but perhaps its a > plotting refresh rate thing. Its not clear to me over what window the > average is taken (I was expecting that as an option). > > - Also, Im not 100% sure what sample rate I should be setting (or how > to count number of samples (e.g., peak look ahead)) in blocks that are > downstream from decimation. If the source sample rate is 1msps, and I > have decimated by 4x, say, that means I have 4x less samples coming > through per second so I should be setting 0.250msps as the sample rate > in downstream blocks (filters, display, fft, ...) right? > > >If the output of that looks somewhat OK, then you could e.g. low-pass > filter the result to get rid of noise pulses that are too short, for > example :) > > Ok, so low-pass filter the real valued mag^2 values. I tried this but > was not obvious how to choose the right cutoff frequency & transition > width other than just trial and error. Also, as the signal gets weaker > the peaks get narrower. > > I still feel like I should be cross correlating with the known signal, > or some kind of matched filter setup, or exploiting the regularity of > the pulse somehow to give me more sensitivity, but unsure how best to > proceed there. I stumbled across a post about the correlate_and_sync > block. That seemed somewhat similar to my problem (though I guess I > have no preamble) but raised more questions than it solved. > > >By the way, it might be cool to put up a short recording (ie. a file > generated using the GNU Radio file sink) somewhere (zipped, e.g. on Google > Drive). I haven't worked with that myself, but maybe putting it on > >SigIDwiki[1] might be beneficial for people encountering the wildlife > tracker signal later on. > > Good idea. I quickly captured a file with raw IQ samples (File sink) > in case anybody is interested. It starts with the transmitter very > close to antenna, moves progressively further until out of range and > then back. Its only about a minute or two tops but at 1msps it ended > up as 813 MB though. > > https://drive.google.com/drive/folders/0B5dKo9igl8W4dWpiMmlEYWx3b0k? > usp=sharing > > >We surely could use a few more "oh, GNU Radio is awesome because it works > for everyone" articles (like [2]), but what we far more severely lack is > stories of people that actually made progress, but also struggled, > >because they didn't know GR for years before writing a blog post. One of > the problems that we face (and that was a very prominent theme at the Panel > at FOSDEM [3]) is that we don't really know how to help people > >that are just approaching GNU Radio, and aren't already familiar with a > lot of the stuff that we've gotten used to. > > Happy to do that. Generally I have found companion very robust and > works well. I love the python export/integration. Im just slightly > concerned about performance on a low spec machine but Ill cross that > bridge when I get to it. Main thing for me is that I find many > explanations for blocks very concise and without enough radio > background you feel a bit lost as to what exactly to use when or what > the options mean. Also, silly question but I am still confused as to > what a "tap" is. I googled around the docs hoping to find a page "what > is a tap" but failed. > > Thanks for the links. > > Many thanks again, > > Dirk > > > (*) https://dirkgorissen.com/2016/04/19/wheres-susi-airborne- > orangutan-tracking-with-python-and-react-js/ > > On 7 February 2017 at 14:22, Marcus Müller <marcus.muel...@ettus.com> > wrote: > > Hi Dirk, > > > > Well, if you have a let's say 2 kHz uncertainty in the frequency of your > > pulse, I'd really start very simple. Looking at your plots, I think we > can > > sufficiently suppress the noise simply by low-pass filtering: > > > > * Use a Low Pass filter (real taps) with a cutoff of 1 kHz – that will > let > > everything from -1 kHz to +1 kHz around your center frequency through. > You > > can use the "freq xlating FIR" block to first shift by a given frequency > and > > then apply the filter; might make things easier. > > * Detect power by putting your complex samples through a complex mag^2 > > block. > > > > If the output of that looks somewhat OK, then you could e.g. low-pass > filter > > the result to get rid of noise pulses that are too short, for example :) > > > > By the way, it might be cool to put up a short recording (ie. a file > > generated using the GNU Radio file sink) somewhere (zipped, e.g. on > Google > > Drive). I haven't worked with that myself, but maybe putting it on > > SigIDwiki[1] might be beneficial for people encountering the wildlife > > tracker signal later on. > > > > I actually think your project is pretty cool – short warning: if this > works > > out, I'll try to convince you to write a short article for the GNU Radio > > blog! > > I'd imagine it being about how you approached the problem and what you've > > found, with emphasis on all the problems you had because of GNU Radio and > > maybe also DSP. We surely could use a few more "oh, GNU Radio is awesome > > because it works for everyone" articles (like [2]), but what we far more > > severely lack is stories of people that actually made progress, but also > > struggled, because they didn't know GR for years before writing a blog > post. > > One of the problems that we face (and that was a very prominent theme at > the > > Panel at FOSDEM [3]) is that we don't really know how to help people that > > are just approaching GNU Radio, and aren't already familiar with a lot of > > the stuff that we've gotten used to. > > > > Happy tracking, > > > > Marcus > > > > [1] http://www.sigidwiki.com/wiki/Adding_a_Signal_Entry > > [2] > > http://gnuradio.org/blog/filtering-time-series-data__ > elemental-building-blocks/ > > [3] https://fosdem.org/2017/schedule/event/sdr_panel/ ; don't know – > maybe > > we'll have recordings of that. > > > > > > On 02/06/2017 09:39 PM, Dirk Gorissen wrote: > > > > Thanks Marcus & Martin for the responses. > > > > To clarify, Im working on a wildlife tracking problem but from the air > > (drone). Im purely interested in finding out if the pulse (which gets > > transmitted at a fixed interval of 1500ms) occurred or not. If it did, I > > know Im within some range of the animal of interest. I have no control > over > > the transmitter and no further technical details (unfortunately). I did > take > > some screenshots to give you an idea what it looks like: > > > > https://goo.gl/photos/Y3Ea6kJo1ewrcDYM8 (taken with tx at close range > to > > static antenna) > > > > I expect quite a bit of (fairly uniform) background noise from other > > electronics to deal with and there are hardware things that can be > improved. > > But for the purposes of this thread I want to ensure Im doing the right > > things on the DSP side. > > > > And yes, Im pretty much a DSP novice but happy to learn. So be gentle :) > > > >>So, my understanding is that your SDR device first downconverts your > 150.22 > >>MHz signal to complex baseband, is that right? > > > > Yes, so think something like an rtlsdr or airspy. > > > > About the FFT question, thanks I got a bit confused. I now understand > that > > if I want FFT over a longer timeframe I should just take a larger size > and I > > can vary the sampling rate (e.g., via decimation) to get the resolution > (Hz > > per bin) I require. > > > > Im now wondering if there is a founded way to pick an optimal FFT size > given > > my pulse is only 10ms long. Im guessing it should not be much longer than > > the 10ms equivalent but maybe there is a noise tradeoff (?). > > > > In any case I agree this is a time based phenomenon so a time domain > > approach should likely be the main one. > > > > So it seems I should at least be trying an FIR filter to do cross > > correlation. Not quite clear how I would actually do that in practise > with > > the gr block though. How would I provide the second input (the synthetic > > pulse)? > > > > I have come across matched filters before but so far failed to bridge the > > gap from theory to code. This is partly the reason I came to gnuradio. > > > >>You can implement a limited-length autocorrelation/crosscorrelation > >> relatively > >>easily; we can talk about how that would look like, but my gut feeling is > >> that this > >>might be something that might not make too much sense in your specific > use > >>case. > > > > Not too sure myself tbh, but happy to take your lead. > > > >>it's possible cyclostationary estimation methods might be helpful here. > > > > So I googled this and poked around a bit on https://cyclostationary.blog > . > > Looks really interesting but out of my depth here.. > > > > Thanks for the tip on gr-fosphor. > > > > Cheers > > Dirk > > > > On 5 February 2017 at 12:06, Marcus Müller <marcus.muel...@ettus.com> > wrote: > >> > >> Hi Dirk, > >> > >> nice to have you around, welcome to GNU Radio! I don't know your level > of > >> DSP knowledge, so please excuse if I either throw too many high-level > >> concepts at you or assume you could want to read up on something that > you > >> already know. If something in my reply is unclear, please don't > hesitate to > >> ask for clarification. > >> > >> So, my understanding is that your SDR device first downconverts your > >> 150.22 MHz signal to complex baseband, is that right? > >> > >> So, what is the objective here? You say you need to /detect/ the pulse, > >> but for what purpose? Is it just about detecting the presence of the > pulse, > >> or is it used for eg. time detection? > >> > >> Your filter->decimate approach sounds very reasonable; I'm not 100% > >> convinced by using the FFT for something that is concentrated in *time* > >> domain, but that might depend on the purpose mentioned above. > >> > >> 2) Im somewhat confused about the FFT block if I just pipe the SDR > >> straight into it. The FFT size is set to 1024 and the window is set to > >> "window.blackmanharris(1024)". So Im assuming the FFT just applies one > >> window (?) and outputs 1024 bins. However, how many samples are > >> accumulated before the FFT is run? I would have assumed I can control > >> that too. And if so, should I best be doing this every 50ms, 500ms, > >> 2000ms, ..? > >> > >> I'm not sure I understand the question. An FFT is simply an DFT. It's a > >> mapping of N-dimensional vector to N-dimensional vector, . The window is > >> multiplied point-wise with the input vector prior to the DFT to avoid > >> spectral leakage. > >> > >> There's no accumulation involved anywhere. > >> > >> 3) I can use the rational resampling block to bring the sample rate > >> down to 48khz so I can use the audio sink. From that I can still hear > >> the pulse even if it is not visible in the spectrum (gui sink). Im > >> assuming this is just because the plotting cannot keep up? > >> > >> Maybe, or maybe the spectrum sink really isn't the right tool to > visualize > >> a pulse! Again, we might want to discuss what this pulse is and what > it's > >> used for. > >> > >> 4) In the time domain I guess I can generate a synthetic pulse of the > >> same length / frequency and then cross correlate. > >> > >> Hm, but correlating a signal with a known fixed sequence is, > >> mathematically, a convolution. > >> > >> That is identical to doing FIR filtering – in fact, if we consider the > >> shape of your pulse as what is often referred to as pulse shape filter, > then > >> that filter in the receiver would simply be the matched filter to that. > >> > >> Not obvious to me > >> how to generate the required pulse in gnuradio though (would a > >> continuous signal work?). > >> > >> "Continuous" would mean you'd do an infinite-length correlation, so > that's > >> not 100% possible. > >> > >> I also notice there are no built in > >> (auto)correlation blocks? > >> > >> Hm, but an autocorrelation would take a complete signal, shift it by all > >> possible shifts and calculating the dot product between the shifted > version > >> and the unshifted, right? That would require to have the complete > signal at > >> once. > >> > >> But GNU Radio is a streaming architecture, so that can't work. > >> > >> You can implement a limited-length autocorrelation/crosscorrelation > >> relatively easily; we can talk about how that would look like, but my > gut > >> feeling is that this might be something that might not make too much > sense > >> in your specific use case. > >> > >> I found the "correlation estimator" but not > >> clear how to use it. As for dealing with the frequency uncertainty > >> problem. Does one just try correlating with different freuencies and > >> pick the best one? Or what is the good thing to do here given I may > >> also have to deal with quite a bit of noise. > >> > >> As a gut feeling: you don't really care about whether the pulse is > exactly > >> at a certain frequency (it's absolutely not normal for wireless > receivers to > >> know the exact frequency a priori), but when it happens. So we might > want to > >> discuss the kind of pulse, and kind of noise we're talking about. > >> > >> As a further gut feeling: I think your autocorrelation question > indicates > >> you might be on a very good track – it's possible cyclostationary > estimation > >> methods might be helpful here. > >> > >> > >> Best regards, > >> > >> Marcus > >> > >> > >> > >> On 02/04/2017 11:39 PM, Dirk Gorissen wrote: > >> > >> Fist of all, while Im a newbie to (gnu)radio, congrats to the dev team > >> for a great piece of software. > >> > >> My question is about the need to detect a weak, noisy, short (10ms) > >> pulse that occurs every 1.5 seconds. It is transmitted at a particular > >> frequency (e.g., 150.22 MHz) but in practise I have found this can > >> vary by as much as +/- 500Hz. There is no modulation, it is simply an > >> on/off keyed pulse. > >> > >> Say I have an SDR generating data at 2.5 MSPS. I have so far been > >> experimenting with standard scipy/numpy routines to collect a batch of > >> samples, perform a FFT (freq domain) and do a cross correlation (time > >> domain). However, Im by no means a dsp guy and would like to leverage > >> gnuradio as much as I can. I have been poking a bit but have some > >> basic questions. > >> > >> 1) 2.5 Msps gives me way more bandwidth than I neeed. Assuming, for > >> now, I only care about a single pulse frequency I really only need > >> ~1khz bandwidth. In the frequency domain I can directly decimate down > >> (with a big factor) to the 1-2 khz range using the low pass filter > >> block, do an fft, and look for peaks. Is that the right approach? > >> > >> 2) Im somewhat confused about the FFT block if I just pipe the SDR > >> straight into it. The FFT size is set to 1024 and the window is set to > >> "window.blackmanharris(1024)". So Im assuming the FFT just applies one > >> window (?) and outputs 1024 bins. However, how many samples are > >> accumulated before the FFT is run? I would have assumed I can control > >> that too. And if so, should I best be doing this every 50ms, 500ms, > >> 2000ms, ..? > >> > >> 3) I can use the rational resampling block to bring the sample rate > >> down to 48khz so I can use the audio sink. From that I can still hear > >> the pulse even if it is not visible in the spectrum (gui sink). Im > >> assuming this is just because the plotting cannot keep up? > >> > >> 4) In the time domain I guess I can generate a synthetic pulse of the > >> same length / frequency and then cross correlate. Not obvious to me > >> how to generate the required pulse in gnuradio though (would a > >> continuous signal work?). I also notice there are no built in > >> (auto)correlation blocks? I found the "correlation estimator" but not > >> clear how to use it. As for dealing with the frequency uncertainty > >> problem. Does one just try correlating with different freuencies and > >> pick the best one? Or what is the good thing to do here given I may > >> also have to deal with quite a bit of noise. > >> > >> Any guidance appreciated. > >> > >> Many thanks, > >> > >> Dirk > >> > >> _______________________________________________ > >> Discuss-gnuradio mailing list > >> Discuss-gnuradio@gnu.org > >> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio > >> > >> _______________________________________________ Discuss-gnuradio > mailing > >> list Discuss-gnuradio@gnu.org > >> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio > > > > -- > > _________________________________________ Dr. Dirk Gorissen Mob: > > +44-7763-806-809 Web: dirkgorissen.com Skype: dirk.gorissen Twitter : > > twitter.com/dirkgor LinkedIn: linkedin.com/in/dirkgorissen > > > > -- > _________________________________________ > Dr. Dirk Gorissen > Mob: +44-7763-806-809 > Web: dirkgorissen.com > Skype: dirk.gorissen > Twitter : twitter.com/dirkgor > LinkedIn: linkedin.com/in/dirkgorissen > > _______________________________________________ > Discuss-gnuradio mailing list > Discuss-gnuradio@gnu.org > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio >
_______________________________________________ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
