Hello Mallesham, That will convert I-Q data (two numbers per a point) to a real number data (one number per a point), so you do save some storage when you do that. However, not enough to lift the storage concern. (Or maybe, if you have sufficiently large storage and you re-tune your receiver to different center frequencies frequently).
If you are collecting PSD data, then I guess it has to do something with spectrum sensing. If that is the case, I suggest try duty-cycling averaging or max-holding the PSD data to reduce number of PSD blocks that you have to store per a unit time. That is often sufficient. Or, maybe you can look at some compressive sensing techniques, if the signal is sparse (although I am not sure how easy would they be to implement with GNURadio + COTS sensors, as I never tried it out). Regards, Kyeong Su Shin On Fri, Jan 20, 2017 at 1:02 PM, Mallesham Dasari <[email protected] > wrote: > Hi Marcus, > > I am recording only power values(PSD), as you described in one of the > previous emails. Not raw IQ or FFT samples. Sorry for the confusion. > > On Fri, Jan 20, 2017 at 3:21 PM, Marcus Müller <[email protected]> > wrote: > >> Mallesham, >> >> you should __really__ look into what kind of data you need to store. Are >> you sure what you want to do in the end is analyze 24 h * 3600 s/h * 32·10⁶ >> S/s * 8B/s = 20 Terabyte of data? That is what you get when you store the >> FFTs of one day worth of samples. I know that you're probably tuning in >> between, but again, as we all stressed: the FFT does *not* reduce the >> amount of data! It is just a transform into a different base system. >> >> Maybe you can take a step back and describe your *application*. What is >> it that you need to *do*? I think your system requirements are not fully >> developed, yet. >> >> Best regards, >> >> Marcus >> On 01/20/2017 09:15 PM, Mallesham Dasari wrote: >> >> Hi All, >> >> Sorry, it was a typo. The sample rate is 32MHz. Thank you very much for >> the information. I will look for the larger storage or some other mechanism. >> >> Thanks! >> >> On Fri, Jan 20, 2017 at 3:01 PM, Kyeong Su Shin <[email protected]> wrote: >> >>> To whom it may concern: >>> >>> As like other people have explained, amount of information carried by >>> the FFT data equals that of the inputted raw I-Q data. At 32GS/s, that >>> would be 64Gbit per a second even if the ADCs are 1-bit. (I am actually not >>> aware of such fast SDR transceivers.. Is it possible?) >>> >>> However, do you really need all that data? What are you trying to >>> achieve? What I do (or some FFT-related blocks do) is duty-cycling the data >>> collection. For an example, you can do something like storing only 1 FFT >>> vector per every 1000 generated and still get a decent information about >>> the channel. If you really need to have all that data, then no, I am not >>> aware of any feasible implementation. >>> >>> Regards, >>> Kyeong Su Shin >>> >>> On Fri, Jan 20, 2017 at 11:42 AM, Mallesham Dasari < >>> [email protected]> wrote: >>> >>>> Hi Marcus, >>>> >>>> Thanks for the quick response. I am recording the FFT samples >>>> continuously. But, I am getting overflow after some time when the file size >>>> has become huge. My sample rate is high (32GHz) and hence writing to the >>>> file takes so long and hence the usrp_spectrum_sense getting overflow. >>>> >>>> On Fri, Jan 20, 2017 at 2:33 PM, Marcus Müller < >>>> [email protected]> wrote: >>>> >>>>> Hello Mallesham, >>>>> >>>>> I'm afraid not, since I'm afraid that to my current understanding, >>>>> what you want is mathematically impossible. Either you want much data – >>>>> and >>>>> that seems to be the case, since you want to record 24h of raw IQ data – >>>>> or >>>>> you can store it in what comparably little RAM modern computers have. >>>>> >>>>> Maybe, however, we haven't fully understood the problem. Can you, >>>>> mathematically, define what you want to observe and record? >>>>> >>>>> Best regards, >>>>> >>>>> Marcus >>>>> >>>>> >>>>> >>>>> On 01/20/2017 08:28 PM, Mallesham Dasari wrote: >>>>> >>>>> Hello everyone, >>>>> >>>>> Can anyone give some solution for this? Even writing to the ramdisk is >>>>> not enough for running the flow graph for so long. I am facing the same >>>>> issue. >>>>> >>>>> Thank you! >>>>> >>>>> On Thu, Jan 12, 2017 at 5:41 PM, Hasini Abeywickrama <[email protected] >>>>> > wrote: >>>>> >>>>>> Hi all, >>>>>> >>>>>> Thank you very much for the informative responses. >>>>>> >>>>>> My requirement is to run the flowgraph for a long time (ideally 24 >>>>>> hours) and store the FFT data in the memory (ramdisk) to they can be >>>>>> processed later or in chunks, not everything at the same time. >>>>>> >>>>>> So far, I have increased the size of the ramdisk and it works fine >>>>>> for a few hours. But it still is not the solution I'm looking for. >>>>>> >>>>>> Regards, >>>>>> Hasini >>>>>> >>>>>> On Thu, Jan 12, 2017 at 8:30 PM, Marcus Müller < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> But if you do a single 1024-FFT, you'd only operate on 1024 of the >>>>>>> input samples! >>>>>>> >>>>>>> And: the FFT doesn't just give you power values, but complex values; >>>>>>> mathematically, the FFT is a DFT, and the DFT is an invertible linear >>>>>>> operator [image: $\mathcal F$]: >>>>>>> >>>>>>> [image: $\mathcal F: \mathbb C^{N} \mapsto \mathbb C^{N}$] >>>>>>> >>>>>>> which maps complex vectors to complex vectors of size [image: >>>>>>> $N,\quad 0<N\in\mathbb N$]. It is, in fact, representable as >>>>>>> square matrix with column (and row) vectors being samples of the >>>>>>> orthogonal >>>>>>> complex sinusoids $e^{j\frac{2\pi}N nk},\, k=0,\ldots,N-1$; that is, it >>>>>>> can >>>>>>> also be understood as a *base change matrix*, that just represents >>>>>>> the "input vector" according to a different base, orthogonal base. >>>>>>> In the physical sense: the input vector base was represented by the >>>>>>> standard basis $\mathbf e_N$, meaning that each base vector represents a >>>>>>> single point in time – the sample time of the respective entry; the >>>>>>> "output" of the transform is represented on a base of orthogonal >>>>>>> frequencies. This is an invertible operation – really just another way >>>>>>> to >>>>>>> look at *the same signal*. I think this is really important to keep >>>>>>> in mind: >>>>>>> >>>>>>> The Fourier transforms are *not* magical by any means. What they do >>>>>>> is represent *the same signal* from a different point of view. It >>>>>>> can be *interpreted* as transform between time and frequency domain >>>>>>> (or space and impulse, or...). The DFT is still just a boring, old, >>>>>>> square, >>>>>>> orthogonal, invertible matrix that produces output of the same >>>>>>> dimensionality as it takes input. >>>>>>> >>>>>>> As you can see, the DFT/FFT itself never reduces the amount of data. >>>>>>> >>>>>>> What you might be referring to is some kind PSD estimate done by >>>>>>> first |·|² a lot of DFTed vectors and then averaging them. The data >>>>>>> reduction here lies in the magnitude square operation and the average, >>>>>>> not >>>>>>> in the DFT. >>>>>>> The point here is that you're throwing away a whole lot of >>>>>>> information, and I'm not convinced that's what Hasini needs! >>>>>>> >>>>>>> Best regards, >>>>>>> >>>>>>> Marcus >>>>>>> >>>>>>> On 12.01.2017 05:54, Mallesham Dasari wrote: >>>>>>> >>>>>>> Hi Marcus, >>>>>>> >>>>>>> Raw IQ samples take lots of memory because each sample will be >>>>>>> around 8Bytes. Suppose, if we 1Msps sample rate, just for 10 minutes of >>>>>>> data, we get 10*60*1M*8B = 4.8GB data. On the other hand, if you store >>>>>>> just >>>>>>> FFT with 1024 bin, we get 4.8GB/1024 power values right (which has very >>>>>>> less size)? >>>>>>> >>>>>>> Please correct me if I am wrong. >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> On Wed, Jan 11, 2017 at 7:32 AM, Marcus Müller < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi Mallesham, >>>>>>>> >>>>>>>> I don't understand – the raw IQ samples and their FFT have the same >>>>>>>> size, and data type. >>>>>>>> Maybe you've understood something that I (and Martin) didn't – >>>>>>>> could you elaborate? >>>>>>>> >>>>>>>> Best regards, >>>>>>>> Marcus >>>>>>>> >>>>>>>> >>>>>>>> On 01/11/2017 12:56 AM, Mallesham Dasari wrote: >>>>>>>> >>>>>>>> Hi Hasini, >>>>>>>> >>>>>>>> If you are trying to print just the FFT, it should not be an issue. >>>>>>>> If you print raw iq samples, then you will run out of memory. By long, >>>>>>>> you >>>>>>>> mean how long? Days? >>>>>>>> >>>>>>>> On Tue, Jan 10, 2017 at 3:16 PM, Martin Braun < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hasini, >>>>>>>>> >>>>>>>>> can you please re-state what you're trying to do? That might help >>>>>>>>> you >>>>>>>>> getting some answers. It is not quite clear from this email. >>>>>>>>> >>>>>>>>> Cheers, >>>>>>>>> Martin >>>>>>>>> >>>>>>>>> >>>>>>>>> On 01/02/2017 09:16 PM, Hasini Abeywickrama wrote: >>>>>>>>> > Hi all, >>>>>>>>> > >>>>>>>>> > I have a flowgraph that reads a signal and writes its FFT >>>>>>>>> samples to a >>>>>>>>> > file. I need to run this continuously (for a long time), without >>>>>>>>> running >>>>>>>>> > out of memory. >>>>>>>>> > >>>>>>>>> > I tired deleting the earlier FFT samples from the file but that >>>>>>>>> messes >>>>>>>>> > up with reading the data. I also tried starting writing to a >>>>>>>>> different >>>>>>>>> > file after some time so the initial file can be completely >>>>>>>>> deleted. But >>>>>>>>> > it did not work as well. >>>>>>>>> > >>>>>>>>> > What would be the best approach for this? Any thought would be >>>>>>>>> very much >>>>>>>>> > appreciated. >>>>>>>>> > >>>>>>>>> > Regards, >>>>>>>>> > Hasini >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > _______________________________________________ >>>>>>>>> > Discuss-gnuradio mailing list >>>>>>>>> > [email protected] >>>>>>>>> > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio >>>>>>>>> > >>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Discuss-gnuradio mailing list >>>>>>>>> [email protected] >>>>>>>>> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Best Regards, >>>>>>>> *Mallesham Dasari* >>>>>>>> Department of Computer Science >>>>>>>> Stony Brook University >>>>>>>> USA - 11794 >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Discuss-gnuradio mailing >>>>>>>> [email protected]https://lists.gnu.org/mailman/listinfo/discuss-gnuradio >>>>>>>> >>>>>>>> _______________________________________________ Discuss-gnuradio >>>>>>>> mailing list [email protected] >>>>>>>> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio >>>>>>> >>>>>>> -- >>>>>>> Best Regards, >>>>>>> *Mallesham Dasari* >>>>>>> Department of Computer Science >>>>>>> Stony Brook University >>>>>>> USA - 11794 >>>>>>> >>>>>>> _______________________________________________ Discuss-gnuradio >>>>>>> mailing list [email protected] https://lists.gnu.org/mailman/ >>>>>>> listinfo/discuss-gnuradio >>>>>> >>>>>> -- >>>>> Best Regards, >>>>> *Mallesham Dasari* >>>>> Department of Computer Science >>>>> Stony Brook University >>>>> USA - 11794 >>>>> >>>>> -- >>>> Best Regards, >>>> *Mallesham Dasari* >>>> Department of Computer Science >>>> Stony Brook University >>>> USA - 11794 >>>> _______________________________________________ Discuss-gnuradio >>>> mailing list [email protected] https://lists.gnu.org/mailman/ >>>> listinfo/discuss-gnuradio >>> >>> _______________________________________________ Discuss-gnuradio >>> mailing list [email protected] https://lists.gnu.org/mailman/ >>> listinfo/discuss-gnuradio >> >> -- >> Best Regards, >> *Mallesham Dasari* >> Department of Computer Science >> Stony Brook University >> USA - 11794 >> >> > > > -- > Best Regards, > *Mallesham Dasari* > Department of Computer Science > Stony Brook University > USA - 11794 >
_______________________________________________ Discuss-gnuradio mailing list [email protected] https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
