Re: [Discuss-gnuradio] Handling of IQ files

[Derek Kozel]

Hi Henry,

Take a look at the fftw library. It's very well regarded and looks to have
C# wrappers.

Regards,
Derek

On Wed, Mar 23, 2016 at 3:30 PM, Henry Barton  wrote:

> I'm planning to write my program from scratch in C#. I understand the
> basics of the FFT now; I had always wondered how transient signals would
> show up in a window if they were significantly shorter than the time
> covered by an FFT window, and I see now that there is a tradeoff: more
> samples = higher frequency precision but fewer FFT windows can be drawn.
> I've observed this in HDSDR where RBW can be either frequency-precise or do
> fast windowing. But as everyone says, the FFT is tremendously complex, so I
> will probably use a 3rd-party library. I'm currently working out how to use
> the AForge .NET library.
>
>
> > Date: Tue, 22 Mar 2016 09:20:41 +0100
> > From: vitt...@gmail.com
> > To: discuss-gnuradio@gnu.org
> > Subject: Re: [Discuss-gnuradio] Handling of IQ files
>
> >
> > Happy to read you replay Henry!
> > Feel free to rip/change/upgrade my work, but let me know about ur
> progress, pse!
> > Remember that it's fairly resource hungry ( file larger that 250 Mb
> > crashes the app also on my I7/16Gb), but it's a starting point.
> > I think that with GNURADIO it's possible to refine this task... I've a
> > lot to study!
> >
> > Victor I3VFJ
> >
> > 2016-03-21 20:45 GMT+01:00 Henry Barton :
> > > I like the concept of your program. It looks just like what I’m trying
> to
> > > write.
> > >
> > > Sent from Windows Mail
> > >
> > > From: Vitt Benv
> > > Sent: ‎Sunday‎, ‎March‎ ‎20‎, ‎2016 ‎4‎:‎16‎ ‎PM
> > > To: discuss-gnuradio@gnu.org
> > >
> > > Hi,
> > > this [ https://sourceforge.net/projects/automodrecog/ ] is my little
> > > effort about handling IQ files.
> > > The input IQ file is recorded with HDSDR, very nice piece of sw, that
> > > as a good recording scheduler. By the way the file provided can be
> > > played with it. I do also some tests with IQ file produced by R&S
> > > EM100 receiver.
> > > I wrote this [horrible] application for personal use and it's very raw
> > > IMO
> > > I'm interested on HF monitoring and the main goal is to find SSB
> > > emission along time, not quite simple task.
> > >
> > > In brief:
> > > - read params from input file
> > > - split it in smaller chunk and save FFT for each chunk
> > > - sum (maxhold) or avg (average) all the FFTs
> > > - find relevant ( over threshold) carrier and try to "pack" them to
> > > find "bandwidth" associated
> > > - build a report as .html page with a .png file that represent the
> result.
> > >
> > > The most difficult part is to estimate the best "threshold", and at
> > > the moment I'm almost stuck there and moreover RL reclaims my
> > > presence :-)
> > >
> > > ... my euro "cent" on the subject.
> > >
> > > Victor I3VFJ
> > >
> > > ___
> > > Discuss-gnuradio mailing list
> > > Discuss-gnuradio@gnu.org
> > > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
> > >
> > > ___
> > > Discuss-gnuradio mailing list
> > > Discuss-gnuradio@gnu.org
> > > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
> > >
> >
> > ___
> > Discuss-gnuradio mailing list
> > Discuss-gnuradio@gnu.org
> > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>
> ___
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>
>
___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] Handling of IQ files

[Henry Barton]

I'm planning to write my program from scratch in C#. I understand the basics of 
the FFT now; I had always wondered how transient signals would show up in a 
window if they were significantly shorter than the time covered by an FFT 
window, and I see now that there is a tradeoff: more samples = higher frequency 
precision but fewer FFT windows can be drawn. I've observed this in HDSDR where 
RBW can be either frequency-precise or do fast windowing. But as everyone says, 
the FFT is tremendously complex, so I will probably use a 3rd-party library. 
I'm currently working out how to use the AForge .NET library.> Date: Tue, 22 
Mar 2016 09:20:41 +0100
> From: vitt...@gmail.com
> To: discuss-gnuradio@gnu.org
> Subject: Re: [Discuss-gnuradio] Handling of IQ files
> 
> Happy to read you replay Henry!
> Feel free to rip/change/upgrade my work, but let me know about ur progress, 
> pse!
> Remember that it's fairly resource hungry ( file larger that 250 Mb
> crashes the app also on my I7/16Gb), but it's a starting point.
> I think that with GNURADIO it's possible to refine this task... I've a
> lot to study!
> 
> Victor I3VFJ
> 
> 2016-03-21 20:45 GMT+01:00 Henry Barton :
> > I like the concept of your program. It looks just like what I’m trying to
> > write.
> >
> > Sent from Windows Mail
> >
> > From: Vitt Benv
> > Sent: ‎Sunday‎, ‎March‎ ‎20‎, ‎2016 ‎4‎:‎16‎ ‎PM
> > To: discuss-gnuradio@gnu.org
> >
> > Hi,
> > this [ https://sourceforge.net/projects/automodrecog/ ] is my little
> > effort about handling IQ files.
> > The input IQ file is recorded with HDSDR, very nice piece of sw, that
> > as a good recording scheduler. By the way the file provided can be
> > played with it. I do also some tests with IQ file produced by R&S
> > EM100 receiver.
> > I wrote this [horrible] application for personal use and it's very raw
> > IMO
> > I'm interested on HF monitoring and the main goal is to find SSB
> > emission along time, not quite simple task.
> >
> > In brief:
> > - read params from input file
> > - split it in smaller chunk and save FFT for each chunk
> > - sum (maxhold) or avg (average) all the FFTs
> > - find relevant ( over threshold) carrier and try to "pack" them to
> > find "bandwidth" associated
> > - build a report as .html page with a .png file that represent the result.
> >
> > The most difficult part is to estimate the best "threshold", and at
> > the moment I'm almost stuck there and moreover RL reclaims my
> > presence :-)
> >
> > ... my euro "cent" on the subject.
> >
> > Victor I3VFJ
> >
> > ___
> > Discuss-gnuradio mailing list
> > Discuss-gnuradio@gnu.org
> > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
> >
> > ___
> > Discuss-gnuradio mailing list
> > Discuss-gnuradio@gnu.org
> > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
> >
> 
> ___
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

  ___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] Handling of IQ files

[Vitt Benv]

Happy to read you replay Henry!
Feel free to rip/change/upgrade my work, but let me know about ur progress, pse!
Remember that it's fairly resource hungry ( file larger that 250 Mb
crashes the app also on my I7/16Gb), but it's a starting point.
I think that with GNURADIO it's possible to refine this task... I've a
lot to study!

Victor I3VFJ

2016-03-21 20:45 GMT+01:00 Henry Barton :
> I like the concept of your program. It looks just like what I’m trying to
> write.
>
> Sent from Windows Mail
>
> From: Vitt Benv
> Sent: ‎Sunday‎, ‎March‎ ‎20‎, ‎2016 ‎4‎:‎16‎ ‎PM
> To: discuss-gnuradio@gnu.org
>
> Hi,
> this [ https://sourceforge.net/projects/automodrecog/ ] is my little
> effort about handling IQ files.
> The input IQ file is recorded with HDSDR, very nice piece of sw, that
> as a good recording scheduler. By the way the file provided can be
> played with it. I do also some tests with IQ file produced by R&S
> EM100 receiver.
> I wrote this [horrible] application for personal use and it's very raw
> IMO
> I'm interested on HF monitoring and the main goal is to find SSB
> emission along time, not quite simple task.
>
> In brief:
> - read params from input file
> - split it in smaller chunk and save FFT for each chunk
> - sum (maxhold) or avg (average) all the FFTs
> - find relevant ( over threshold) carrier and try to "pack" them to
> find "bandwidth" associated
> - build a report as .html page with a .png file that represent the result.
>
> The most difficult part is to estimate the best "threshold", and at
> the moment I'm almost stuck there and moreover RL reclaims my
> presence :-)
>
> ... my euro "cent" on the subject.
>
> Victor I3VFJ
>
> ___
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>
> ___
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>

___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] Handling of IQ files

[Henry Barton]

I like the concept of your program. It looks just like what I’m trying to write.






Sent from Windows Mail





From: Vitt Benv
Sent: ‎Sunday‎, ‎March‎ ‎20‎, ‎2016 ‎4‎:‎16‎ ‎PM
To: discuss-gnuradio@gnu.org





Hi,
this [ https://sourceforge.net/projects/automodrecog/ ] is my little
effort about handling IQ files.
The input IQ file is recorded with HDSDR, very nice piece of sw, that
as a good recording scheduler. By the way the file provided can be
played with it. I do also some tests with IQ file produced by R&S
EM100 receiver.
I wrote this [horrible] application for personal use and it's very raw IMO
I'm interested on HF monitoring and the main goal is to find SSB
emission along time, not quite simple task.

In brief:
- read params from input file
- split it in smaller chunk and save FFT for each chunk
- sum (maxhold) or avg (average) all the FFTs
- find relevant ( over threshold) carrier and try to "pack" them to
find "bandwidth" associated
- build a report as .html page with a .png file that represent the result.

The most difficult part is to estimate the best "threshold", and at
the moment I'm almost stuck there and moreover RL reclaims my
presence :-)

... my euro "cent" on the subject.

Victor I3VFJ

___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] Handling of IQ files

[Vitt Benv]

Hi,
this [ https://sourceforge.net/projects/automodrecog/ ] is my little
effort about handling IQ files.
The input IQ file is recorded with HDSDR, very nice piece of sw, that
as a good recording scheduler. By the way the file provided can be
played with it. I do also some tests with IQ file produced by R&S
EM100 receiver.
I wrote this [horrible] application for personal use and it's very raw IMO
I'm interested on HF monitoring and the main goal is to find SSB
emission along time, not quite simple task.

In brief:
- read params from input file
- split it in smaller chunk and save FFT for each chunk
- sum (maxhold) or avg (average) all the FFTs
- find relevant ( over threshold) carrier and try to "pack" them to
find "bandwidth" associated
- build a report as .html page with a .png file that represent the result.

The most difficult part is to estimate the best "threshold", and at
the moment I'm almost stuck there and moreover RL reclaims my
presence :-)

... my euro "cent" on the subject.

Victor I3VFJ

___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] Handling of IQ files

[Marcus Müller]

Ah, sorry, I sent that piece of email a bit too early.


On 20.03.2016 19:07, Henry Barton wrote:
> So every element of the resulting vector will hold the intensity of a
> 1Hz bit of spectrum?
No, not at all. The N-Point DFT gives you N evenly spaced frequencies
that make up the whole Nyquist bandwidth (i.e. the sampling rate $f_s$
in the complex case).
> Also, you mentioned what would be in X, but what should vector W contain?
$W$ is the DFT Matrix. Look it up. The wikipedia Article is very
illustrative with great visualizations!

Best regards,
Marcus

___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] Handling of IQ files

[Marcus Müller]

Hi Henry,

> So correlating is digitally mixing something with a predetermined
> sequence?
um, no. Not in the sense that I'd use "correlating".
Mixing in this context is simple point-wise multiplication.
> I guess, then, if you have a PSK waveform you might multiply it by
> something, starting at various offsets in the signal, to try and
> decode it?
A correlation of two digital signals is pretty much that, you take the
first signal, multiply it with the second, _sum that up_, and then shift
either signal, rinse, repeat. You get a vector of /correlation
coefficients/ which can be interpreted as "how similar is the first
signal to the second being shifted by a different sample delays".

Generally, if you have a known pulse shape (i.e. your transmitter takes
the transmit symbol, in case of BPSK that would be either +1 or -1, and
multiplies it with the pulse shape) you can correlate against that pulse
shape, and you'll get the factor with which the pulse was multiplied
(which is your transmit symbol combined with whatever your channel does
to the signal).

Because we were talking about the DFT/FFT:

Look up the Discrete Fourier Transform in its Matrix form: You take a
vector of samples $x$ and multiply it with a matrix $W$ to get the
discrete Fourier transform of $x$, which I shall denote $X$:

$X=Wx$, with

$W \in \mathbb N ^{N\times N}$ (i.e. a square matrix, representing the
N-point DFT)
$x,X \in \mathbb N ^N$ (i.e. column vectors).

Remember how you do matrix-vector multiplication:
To get the result's element in its $n$-th result row and the $m$-th
result column, you take row $n$ from the left factor (i.e. a row of $W$,
which has $N$ rows, so $n\in\{1,\dots,N\}$), and multiply it point-wise
with $m$-th column of the right factor ($x$, which only has 1 column, so
$m\equiv 1$) and calculate the sum.
Now, point-wise multiplication & summation is what is defined as /dot
product/, so what happens here is that you take your sample vector $x$
and calculate the dot products with each row of the DFT Matrix $W$. The
rows of the DFT matrices happen to be nothing else but samples of
complex sinusoids.

Regarding Lou's:
> The FFT is different (I actually don't know how it works, other than
> it operates on 2^n samples), but the output is the same.
Exactly, an FFT is just a specific implementation of the DFT (because
you really don't want to do vector-matrix multiplication if you can
avoid it, because you'd end up doing $N^2$ multiplications and
summations), and is mathematically equivalent. In a first step, the fact
that the DFT matrix is pretty regular and all entries appear twice is
used to save calculations. Modern FFTs do all kind of cool optimizations
that I don't understand the least, but what I learnt is that today, the
Radix-2 algorithm isn't the only FFT algorithm in existence, and so it
turns out that for example the FFTW library, which GNU Radio uses to do
its FFTs, can deal with any DFT that has a prime factorization with
"nearly no" prime factors >7 [1].

So, yeah, DSP often gets a lot easier if you just consider it an
exercise in linear algebra :)

Cheers,
Marcus

[1] http://www.fftw.org/doc/Complex-DFTs.html#Complex-DFTs



On 20.03.2016 17:45, Henry Barton wrote:
> So correlating is digitally mixing something with a predetermined
> sequence? I’ve been struggling to figure out what correlation means
> and this seems right. I guess, then, if you have a PSK waveform you
> might multiply it by something, starting at various offsets in the
> signal, to try and decode it?
>
> Sent from Windows Mail
>
> *From:* madengr 
> *Sent:* ‎Sunday‎, ‎March‎ ‎20‎, ‎2016 ‎12‎:‎22‎ ‎PM
> *To:* discuss-gnuradio@gnu.org 
>
> Yes, pretty much.  With the DFT (and the continuous one) you are
> correlating
> the input waveform with harmonically related, complex sinusoids;
> essentially
> for each harmonic you mix it down to DC then sum (integrate).  The FFT is
> different (I actually don't know how it works, other than it operates
> on 2^n
> samples), but the output is the same.
> Lou
>
>
> Henry Barton wrote
> > I’ve read up on the FFT and DSP and I must say I’m impressed that
> > multiplying two waveforms is the digital equivalent of heterodyning.
> Am I
> > right in my understanding that finding frequency components (FFT-ing) is
> > simply multiplying a series of known sine waves by your input waveform?
>
>
>
>
>
> --
> View this message in context:
> http://gnuradio.4.n7.nabble.com/Handling-of-IQ-files-tp58915p58935.html
> Sent from the GnuRadio mailing list archive at Nabble.com.
>
> ___
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>
>
> ___
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

___
Discu

Re: [Discuss-gnuradio] Handling of IQ files

[Henry Barton]

So every element of the resulting vector will hold the intensity of a 1Hz bit 
of spectrum? Also, you mentioned what would be in X, but what should vector W 
contain? 






Sent from Windows Mail





From: Marcus Müller
Sent: ‎Sunday‎, ‎March‎ ‎20‎, ‎2016 ‎1‎:‎57‎ ‎PM
To: discuss-gnuradio@gnu.org




Hi Henry,

Look up the Discrete Fourier Transform in its Matrix form: You take a vector of 
samples  $x$ and multiply it with a matrix  $W$ to get the discrete Fourier 
transform of  $x$ , which I shall denote  $X$ :
 $X=Wx$ , with

 $W \in \mathbb N ^{N\times N}$ (i.e. a square matrix, representing the N-point 
DFT)
 $x,X \in \mathbb N ^N$ (i.e. column vectors).

Remember how you do matrix-vector multiplication:
To get the result's element in its $n$-th result row and the $m$-th result 
column, you take row $n$ from the left factor (i.e. a row of $W$, which has $N$ 
rows, so $n\in\{1,\dots,N\}$), and multiply it point-wise with $m$-th column of 
the right factor ($x$, which only has 1 column, so $m\equiv 1$) and calculate 
the sum.


On 20.03.2016 17:45, Henry Barton wrote:



So correlating is digitally mixing something with a predetermined sequence? 
I’ve been struggling to figure out what correlation means and this seems right. 
I guess, then, if you have a PSK waveform you might multiply it by something, 
starting at various offsets in the signal, to try and decode it?






Sent from Windows Mail





From: madengr
Sent: ‎Sunday‎, ‎March‎ ‎20‎, ‎2016 ‎12‎:‎22‎ ‎PM
To: discuss-gnuradio@gnu.org





Yes, pretty much.  With the DFT (and the continuous one) you are correlating
the input waveform with harmonically related, complex sinusoids; essentially
for each harmonic you mix it down to DC then sum (integrate).  The FFT is
different (I actually don't know how it works, other than it operates on 2^n
samples), but the output is the same.
Lou


Henry Barton wrote
> I’ve read up on the FFT and DSP and I must say I’m impressed that
> multiplying two waveforms is the digital equivalent of heterodyning. Am I
> right in my understanding that finding frequency components (FFT-ing) is
> simply multiplying a series of known sine waves by your input waveform?





--
View this message in context: 
http://gnuradio.4.n7.nabble.com/Handling-of-IQ-files-tp58915p58935.html
Sent from the GnuRadio mailing list archive at Nabble.com.

___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio




___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] Handling of IQ files

[Marcus Müller]

Hi Henry,

Look up the Discrete Fourier Transform in its Matrix form: You take a
vector of samples $x$ and multiply it with a matrix $W$ to get the
discrete Fourier transform of $x$, which I shall denote $X$:
$X=Wx$, with

$W \in \mathbb N ^{N\times N}$ (i.e. a square matrix, representing the
N-point DFT)
$x,X \in \mathbb N ^N$ (i.e. column vectors).

Remember how you do matrix-vector multiplication:
To get the result's element in its $n$-th result row and the $m$-th
result column, you take row $n$ from the left factor (i.e. a row of $W$,
which has $N$ rows, so $n\in\{1,\dots,N\}$), and multiply it point-wise
with $m$-th column of the right factor ($x$, which only has 1 column, so
$m\equiv 1$) and calculate the sum.

On 20.03.2016 17:45, Henry Barton wrote:
> So correlating is digitally mixing something with a predetermined
> sequence? I’ve been struggling to figure out what correlation means
> and this seems right. I guess, then, if you have a PSK waveform you
> might multiply it by something, starting at various offsets in the
> signal, to try and decode it?
>
> Sent from Windows Mail
>
> *From:* madengr 
> *Sent:* ‎Sunday‎, ‎March‎ ‎20‎, ‎2016 ‎12‎:‎22‎ ‎PM
> *To:* discuss-gnuradio@gnu.org 
>
> Yes, pretty much.  With the DFT (and the continuous one) you are
> correlating
> the input waveform with harmonically related, complex sinusoids;
> essentially
> for each harmonic you mix it down to DC then sum (integrate).  The FFT is
> different (I actually don't know how it works, other than it operates
> on 2^n
> samples), but the output is the same.
> Lou
>
>
> Henry Barton wrote
> > I’ve read up on the FFT and DSP and I must say I’m impressed that
> > multiplying two waveforms is the digital equivalent of heterodyning.
> Am I
> > right in my understanding that finding frequency components (FFT-ing) is
> > simply multiplying a series of known sine waves by your input waveform?
>
>
>
>
>
> --
> View this message in context:
> http://gnuradio.4.n7.nabble.com/Handling-of-IQ-files-tp58915p58935.html
> Sent from the GnuRadio mailing list archive at Nabble.com.
>
> ___
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>
>
> ___
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] Handling of IQ files

[Henry Barton]

So correlating is digitally mixing something with a predetermined sequence? 
I’ve been struggling to figure out what correlation means and this seems right. 
I guess, then, if you have a PSK waveform you might multiply it by something, 
starting at various offsets in the signal, to try and decode it?






Sent from Windows Mail





From: madengr
Sent: ‎Sunday‎, ‎March‎ ‎20‎, ‎2016 ‎12‎:‎22‎ ‎PM
To: discuss-gnuradio@gnu.org





Yes, pretty much.  With the DFT (and the continuous one) you are correlating
the input waveform with harmonically related, complex sinusoids; essentially
for each harmonic you mix it down to DC then sum (integrate).  The FFT is
different (I actually don't know how it works, other than it operates on 2^n
samples), but the output is the same.
Lou


Henry Barton wrote
> I’ve read up on the FFT and DSP and I must say I’m impressed that
> multiplying two waveforms is the digital equivalent of heterodyning. Am I
> right in my understanding that finding frequency components (FFT-ing) is
> simply multiplying a series of known sine waves by your input waveform?





--
View this message in context: 
http://gnuradio.4.n7.nabble.com/Handling-of-IQ-files-tp58915p58935.html
Sent from the GnuRadio mailing list archive at Nabble.com.

___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] Handling of IQ files

[madengr]

Yes, pretty much.  With the DFT (and the continuous one) you are correlating
the input waveform with harmonically related, complex sinusoids; essentially
for each harmonic you mix it down to DC then sum (integrate).  The FFT is
different (I actually don't know how it works, other than it operates on 2^n
samples), but the output is the same.
Lou


Henry Barton wrote
> I’ve read up on the FFT and DSP and I must say I’m impressed that
> multiplying two waveforms is the digital equivalent of heterodyning. Am I
> right in my understanding that finding frequency components (FFT-ing) is
> simply multiplying a series of known sine waves by your input waveform?





--
View this message in context: 
http://gnuradio.4.n7.nabble.com/Handling-of-IQ-files-tp58915p58935.html
Sent from the GnuRadio mailing list archive at Nabble.com.

___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] Handling of IQ files

[Henry Barton]

I’ve read up on the FFT and DSP and I must say I’m impressed that multiplying 
two waveforms is the digital equivalent of heterodyning. Am I right in my 
understanding that finding frequency components (FFT-ing) is simply multiplying 
a series of known sine waves by your input waveform?






Sent from Windows Mail





From: Nate Temple
Sent: ‎Saturday‎, ‎March‎ ‎19‎, ‎2016 ‎2‎:‎51‎ ‎PM
To: Henry Barton
Cc: discuss-gnuradio@gnu.org





Hi Henry,

Here are a few open source applications you may find useful to reference to 
build your tool.

rtl_power + heatmap.py (c/python) - Hard coded to use the RTL-SDRs
https://github.com/keenerd/rtl-sdr/blob/master/src/rtl_power.c
http://kmkeen.com/rtl-power/

rtl_power port that uses FFTW - https://github.com/AD-Vega/rtl-power-fftw

inspectrum (c++) - https://github.com/miek/inspectrum

Such Samples by Tim O'Shea - GR Based Sample Data File Visualization
https://oshearesearch.com/2015/05/22/such-samples-a-gnu-radio-tool-for-sample-data-visualization/
https://oshearesearch.com/2015/12/08/such-samples-2/

- Nate


> On Mar 19, 2016, at 11:05 AM, Henry Barton  wrote:
> 
> So there’s no “read x samples, divide by y, do such-and-such, and you have a 
> frequency-domain array” that I can average over time?
> 
> Sent from Windows Mail
> 
> From: Nikos Balkanas
> Sent: ‎Saturday‎, ‎March‎ ‎19‎, ‎2016 ‎1‎:‎31‎ ‎PM
> To: James Humphries
> Cc: Henry Barton, discuss-gnuradio@gnu.org
> 
> Hi,
> 
> I missed your second part. gr-fosphor is realtime, so It will follow whatever 
> frequencies you have. Frequency hops show as frequency bands in a frequency 
> spectrum.
> The frequency spread of a single plot, is your sampling frequency.
> 
> HTH,
> Nikos​
> 
> On Sat, Mar 19, 2016 at 7:22 PM, James Humphries  
> wrote:
> Hi Henry,
> 
> There is a script, read_complex_binary.m, that is included with gnuradio. You 
> can use that with Octave or Matlab to read the I/Q recordings from a file as 
> a time vector.
> 
> -Trip
> 
> On Sat, Mar 19, 2016 at 12:43 PM, Henry Barton  wrote:
> Is there any simple formula for plotting spectrum (finding the intensity of 
> each frequency component, Hertz by Hertz) from IQ recordings? Specifically I 
> need to know how to read an IQ file and somehow dissect clusters of samples. 
> I’ve written programs that deal with large amounts of data from files, so I 
> think this shouldn't be too hard. I want to write my program so that it takes 
> in a multi-hour IQ file and averages it like the 24-hour band averaging on 
> the University of Twente WebSDR site. This would allow users to average an IQ 
> file over time and see the most active frequencies and times. There’s no 
> utility for this yet, and I’d like to write it and release it on my blog.
> 
> On a side note: is it possible to go “frame-by-frame” in an IQ file? For 
> example, to follow the hops of a 900-MHz FHSS device.
> 
> ___
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
> 
> 
> 
> ___
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
> 
> 
> ___
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] Handling of IQ files

[Nate Temple]

Hi Henry,

Here are a few open source applications you may find useful to reference to 
build your tool.

rtl_power + heatmap.py (c/python) - Hard coded to use the RTL-SDRs
https://github.com/keenerd/rtl-sdr/blob/master/src/rtl_power.c
http://kmkeen.com/rtl-power/

rtl_power port that uses FFTW - https://github.com/AD-Vega/rtl-power-fftw

inspectrum (c++) - https://github.com/miek/inspectrum

Such Samples by Tim O'Shea - GR Based Sample Data File Visualization
https://oshearesearch.com/2015/05/22/such-samples-a-gnu-radio-tool-for-sample-data-visualization/
https://oshearesearch.com/2015/12/08/such-samples-2/

- Nate


> On Mar 19, 2016, at 11:05 AM, Henry Barton  wrote:
> 
> So there’s no “read x samples, divide by y, do such-and-such, and you have a 
> frequency-domain array” that I can average over time?
> 
> Sent from Windows Mail
> 
> From: Nikos Balkanas
> Sent: ‎Saturday‎, ‎March‎ ‎19‎, ‎2016 ‎1‎:‎31‎ ‎PM
> To: James Humphries
> Cc: Henry Barton, discuss-gnuradio@gnu.org
> 
> Hi,
> 
> I missed your second part. gr-fosphor is realtime, so It will follow whatever 
> frequencies you have. Frequency hops show as frequency bands in a frequency 
> spectrum.
> The frequency spread of a single plot, is your sampling frequency.
> 
> HTH,
> Nikos​
> 
> On Sat, Mar 19, 2016 at 7:22 PM, James Humphries  
> wrote:
> Hi Henry,
> 
> There is a script, read_complex_binary.m, that is included with gnuradio. You 
> can use that with Octave or Matlab to read the I/Q recordings from a file as 
> a time vector.
> 
> -Trip
> 
> On Sat, Mar 19, 2016 at 12:43 PM, Henry Barton  wrote:
> Is there any simple formula for plotting spectrum (finding the intensity of 
> each frequency component, Hertz by Hertz) from IQ recordings? Specifically I 
> need to know how to read an IQ file and somehow dissect clusters of samples. 
> I’ve written programs that deal with large amounts of data from files, so I 
> think this shouldn't be too hard. I want to write my program so that it takes 
> in a multi-hour IQ file and averages it like the 24-hour band averaging on 
> the University of Twente WebSDR site. This would allow users to average an IQ 
> file over time and see the most active frequencies and times. There’s no 
> utility for this yet, and I’d like to write it and release it on my blog.
> 
> On a side note: is it possible to go “frame-by-frame” in an IQ file? For 
> example, to follow the hops of a 900-MHz FHSS device.
> 
> ___
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
> 
> 
> 
> ___
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
> 
> 
> ___
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio


___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] Handling of IQ files

[Nikos Balkanas]

Hi,

With gr-fosphor you just playback the file and it loops until you stop it.
Plot supports averages, max and mins. Since I/Q files do not contain the
actual frequency, you'll have to calibrate the plot, which is 1 sample
frequency wide.

gr-fosphor runs on opencl.

HTH
Nikos
​

On Sat, Mar 19, 2016 at 8:05 PM, Henry Barton  wrote:

> So there’s no “read x samples, divide by y, do such-and-such, and you have
> a frequency-domain array” that I can average over time?
>
> Sent from Windows Mail
>
> *From:* Nikos Balkanas 
> *Sent:* ‎Saturday‎, ‎March‎ ‎19‎, ‎2016 ‎1‎:‎31‎ ‎PM
> *To:* James Humphries 
> *Cc:* Henry Barton , discuss-gnuradio@gnu.org
>
> Hi,
>
> I missed your second part. gr-fosphor is realtime, so It will follow
> whatever frequencies you have. Frequency hops show as frequency bands in a
> frequency spectrum.
> The frequency spread of a single plot, is your sampling frequency.
>
> HTH,
> Nikos​
>
> On Sat, Mar 19, 2016 at 7:22 PM, James Humphries <
> james.humphr...@ettus.com> wrote:
>
>> Hi Henry,
>>
>> There is a script, read_complex_binary.m, that is included with gnuradio.
>> You can use that with Octave or Matlab to read the I/Q recordings from a
>> file as a time vector.
>>
>> -Trip
>>
>> On Sat, Mar 19, 2016 at 12:43 PM, Henry Barton  wrote:
>>
>>> Is there any simple formula for plotting spectrum (finding the intensity
>>> of each frequency component, Hertz by Hertz) from IQ recordings?
>>> Specifically I need to know how to read an IQ file and somehow dissect
>>> clusters of samples. I’ve written programs that deal with large amounts of
>>> data from files, so I think this shouldn't be too hard. I want to write my
>>> program so that it takes in a multi-hour IQ file and averages it like the
>>> 24-hour band averaging on the University of Twente WebSDR site. This would
>>> allow users to average an IQ file over time and see the most active
>>> frequencies and times. There’s no utility for this yet, and I’d like to
>>> write it and release it on my blog.
>>>
>>> On a side note: is it possible to go “frame-by-frame” in an IQ file? For
>>> example, to follow the hops of a 900-MHz FHSS device.
>>>
>>> ___
>>> Discuss-gnuradio mailing list
>>> Discuss-gnuradio@gnu.org
>>> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>>>
>>>
>>
>> ___
>> Discuss-gnuradio mailing list
>> Discuss-gnuradio@gnu.org
>> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>>
>>
>
> ___
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>
>
___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] Handling of IQ files

[Marcus D. Leech]

On 03/19/2016 02:05 PM, Henry Barton wrote:
So there’s no “read x samples, divide by y, do such-and-such, and you 
have a frequency-domain array” that I can average over time?


Sent from Windows Mail

Gnu Radio has various types of FFT blocks, filters, decimators, etc, 
etc.But there is no big, blue 'Do that thing that Henry Barton wants 
to do'
  button.  It's a development framework for developing DSP 
applications, largely in the radio space.  One could *use* Gnu Radio to
  *build* a "signal analysis workbench", but Gnu Radio isn't in and of 
itself, "a signal analysis workbench".


Just as GCC isn't "that cool spreadsheet app I'd love to have", but it 
can certainly be *used* to *build* "that cool spreadsheet app I'd love

  to have".



*From:* Nikos Balkanas 
*Sent:* ‎Saturday‎, ‎March‎ ‎19‎, ‎2016 ‎1‎:‎31‎ ‎PM
*To:* James Humphries 
*Cc:* Henry Barton , 
discuss-gnuradio@gnu.org 


Hi,

I missed your second part. gr-fosphor is realtime, so It will follow 
whatever frequencies you have. Frequency hops show as frequency bands 
in a frequency spectrum.

The frequency spread of a single plot, is your sampling frequency.

HTH,
Nikos​

On Sat, Mar 19, 2016 at 7:22 PM, James Humphries 
mailto:james.humphr...@ettus.com>> wrote:


Hi Henry,

There is a script, read_complex_binary.m, that is included with
gnuradio. You can use that with Octave or Matlab to read the I/Q
recordings from a file as a time vector.

-Trip

On Sat, Mar 19, 2016 at 12:43 PM, Henry Barton mailto:kw...@outlook.com>> wrote:

Is there any simple formula for plotting spectrum (finding the
intensity of each frequency component, Hertz by Hertz) from IQ
recordings? Specifically I need to know how to read an IQ file
and somehow dissect clusters of samples. I’ve written programs
that deal with large amounts of data from files, so I think
this shouldn't be too hard. I want to write my program so that
it takes in a multi-hour IQ file and averages it like the
24-hour band averaging on the University of Twente WebSDR
site. This would allow users to average an IQ file over time
and see the most active frequencies and times. There’s no
utility for this yet, and I’d like to write it and release it
on my blog.

On a side note: is it possible to go “frame-by-frame” in an IQ
file? For example, to follow the hops of a 900-MHz FHSS device.

___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org 
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio



___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org 
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio




___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio


___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] Handling of IQ files

[Henry Barton]

So there’s no “read x samples, divide by y, do such-and-such, and you have a 
frequency-domain array” that I can average over time?






Sent from Windows Mail





From: Nikos Balkanas
Sent: ‎Saturday‎, ‎March‎ ‎19‎, ‎2016 ‎1‎:‎31‎ ‎PM
To: James Humphries
Cc: Henry Barton, discuss-gnuradio@gnu.org






Hi,




I missed your second part. gr-fosphor is realtime, so It will follow whatever 
frequencies you have. Frequency hops show as frequency bands in a frequency 
spectrum.

The frequency spread of a single plot, is your sampling frequency.




HTH,

Nikos​



On Sat, Mar 19, 2016 at 7:22 PM, James Humphries  
wrote:


Hi Henry,



There is a script, read_complex_binary.m, that is included with gnuradio. You 
can use that with Octave or Matlab to read the I/Q recordings from a file as a 
time vector.




-Trip



On Sat, Mar 19, 2016 at 12:43 PM, Henry Barton  wrote:




Is there any simple formula for plotting spectrum (finding the intensity of 
each frequency component, Hertz by Hertz) from IQ recordings? Specifically I 
need to know how to read an IQ file and somehow dissect clusters of samples. 
I’ve written programs that deal with large amounts of data from files, so I 
think this shouldn't be too hard. I want to write my program so that it takes 
in a multi-hour IQ file and averages it like the 24-hour band averaging on the 
University of Twente WebSDR site. This would allow users to average an IQ file 
over time and see the most active frequencies and times. There’s no utility for 
this yet, and I’d like to write it and release it on my blog.






On a side note: is it possible to go “frame-by-frame” in an IQ file? For 
example, to follow the hops of a 900-MHz FHSS device.

___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio



___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] Handling of IQ files

[Nikos Balkanas]

Hi,

I missed your second part. gr-fosphor is realtime, so It will follow
whatever frequencies you have. Frequency hops show as frequency bands in a
frequency spectrum.
The frequency spread of a single plot, is your sampling frequency.

HTH,
Nikos​

On Sat, Mar 19, 2016 at 7:22 PM, James Humphries 
wrote:

> Hi Henry,
>
> There is a script, read_complex_binary.m, that is included with gnuradio.
> You can use that with Octave or Matlab to read the I/Q recordings from a
> file as a time vector.
>
> -Trip
>
> On Sat, Mar 19, 2016 at 12:43 PM, Henry Barton  wrote:
>
>> Is there any simple formula for plotting spectrum (finding the intensity
>> of each frequency component, Hertz by Hertz) from IQ recordings?
>> Specifically I need to know how to read an IQ file and somehow dissect
>> clusters of samples. I’ve written programs that deal with large amounts of
>> data from files, so I think this shouldn't be too hard. I want to write my
>> program so that it takes in a multi-hour IQ file and averages it like the
>> 24-hour band averaging on the University of Twente WebSDR site. This would
>> allow users to average an IQ file over time and see the most active
>> frequencies and times. There’s no utility for this yet, and I’d like to
>> write it and release it on my blog.
>>
>> On a side note: is it possible to go “frame-by-frame” in an IQ file? For
>> example, to follow the hops of a 900-MHz FHSS device.
>>
>> ___
>> Discuss-gnuradio mailing list
>> Discuss-gnuradio@gnu.org
>> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>>
>>
>
> ___
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>
>
___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] Handling of IQ files

[James Humphries]

Hi Henry,

There is a script, read_complex_binary.m, that is included with gnuradio.
You can use that with Octave or Matlab to read the I/Q recordings from a
file as a time vector.

-Trip

On Sat, Mar 19, 2016 at 12:43 PM, Henry Barton  wrote:

> Is there any simple formula for plotting spectrum (finding the intensity
> of each frequency component, Hertz by Hertz) from IQ recordings?
> Specifically I need to know how to read an IQ file and somehow dissect
> clusters of samples. I’ve written programs that deal with large amounts of
> data from files, so I think this shouldn't be too hard. I want to write my
> program so that it takes in a multi-hour IQ file and averages it like the
> 24-hour band averaging on the University of Twente WebSDR site. This would
> allow users to average an IQ file over time and see the most active
> frequencies and times. There’s no utility for this yet, and I’d like to
> write it and release it on my blog.
>
> On a side note: is it possible to go “frame-by-frame” in an IQ file? For
> example, to follow the hops of a 900-MHz FHSS device.
>
> ___
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>
>
___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Re: [Discuss-gnuradio] Handling of IQ files

[Nikos Balkanas]

​Hi,

Yes. gr-fosphor will do that for you. It's main takes as input a complex
float I/Q file from usrp.

HTH
Nikos

On Sat, Mar 19, 2016 at 6:43 PM, Henry Barton  wrote:

> Is there any simple formula for plotting spectrum (finding the intensity
> of each frequency component, Hertz by Hertz) from IQ recordings?
> Specifically I need to know how to read an IQ file and somehow dissect
> clusters of samples. I’ve written programs that deal with large amounts of
> data from files, so I think this shouldn't be too hard. I want to write my
> program so that it takes in a multi-hour IQ file and averages it like the
> 24-hour band averaging on the University of Twente WebSDR site. This would
> allow users to average an IQ file over time and see the most active
> frequencies and times. There’s no utility for this yet, and I’d like to
> write it and release it on my blog.
>
> On a side note: is it possible to go “frame-by-frame” in an IQ file? For
> example, to follow the hops of a 900-MHz FHSS device.
>
> ___
> Discuss-gnuradio mailing list
> Discuss-gnuradio@gnu.org
> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>
>
___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio